Our Data center have formally closed out this issue as resolved and provided an update on the cause.
Cause:
Private network traffic routing was interrupted by a failure during a routine maintenance event.
Description:
An error during a maintenance update caused routing configuration to become unstable. This behaviour hadn’t been observed during previous maintenance on other parts of this infrastructure and was not expected.
Resolution:
During the event the data center team had to attempt manual restoration of routing paths until the issue had been identified. Affected routes were then restored.
The supplier has been contacted for future mitigation strategies in case of any future maintenance requirements.
We don’t expect a repeat of this event and the maintenance has now been completed.
Our data center have advised us that restoration of all network services has now been completed.
Our engineering team have also now completed a thorough check of all systems and confirmed full service restore.
We’ll continue to monitor in case of any further issues.
Resolution of all network issues has almost been completed.
There may still be some network instability over the coming hour as work progresses but most affected UK clients should be able to access their simPRO builds without issue.
We are awaiting the final all clear from our data center.
Our data center team in the UK are still working on this issue and we hope to provide a further update in the next 30 minutes.
The data center have advised us that this issue is still ongoing on some parts of the network and they are working as quickly as possible to rectify.
During this time access to some simPRO builds could be impacted and so our engineering team are still engaged with the data center to assist where needed until complete resolution.
Our data center team have resolved the internal network fault and we are now progressing through each part of the network to ensure the network is reset correctly. simPRO builds will be back online but there could be moments of slowness as we restart network services on various components.
Our UK data center have identified the internal network fault affecting a considerable part of the UK data center and working to have this issue resolved as quick as possible.
We are currently awaiting updates from the network team on progress.
Our data center is currently experiencing an internal network fault and this is affecting some UK customer simPRO builds.
Our engineering team is working with the data center team to get this issue resolved as quickly as possible.