Categories
Outages

[Resolved]DataCentre Network Outage

We are currently facing a network outage in our UK DataCentre, once we have more information it will be posted here.

Update @ 01:49 am – At approximately 00:20 a network issue was detected affecting connectivity to the North cluster.

Our on call engineers were contacted and the problem has been resolved. Service was restored at approximately 01:35. Unfortunately we do not currently have any further details available at this time but will issue them as and when they become available.

We would like to express our sincere apologies for the inconvenience this will have caused. If you are experiencing any problems as a result of this please do not hesitate to contact our support team immediately and we will look into this for you.

Update @ 01:35 am – Connectivity has just come back, still awaiting any news as to what caused this.

Update @ 01:13 am – Just a quick update to say that we still cant get in touch with the DataCentre by any means, we are continuing to call them but as of yet all phone lines are completely dead.

Categories
Planned Maintainance

[Completed] CP1 Upgrade Status

Migrations have now completed! In the following 24 hours we will be doing some in-depth optimization of the machine in order to further improve performance.

Thanks.

Categories
Outages

[Resolved] cp1 crashed

cp1 has crashed (hence why we are upgrading) and is being checked.

We will update you ASAP

Update: The machine is doing an automatic forced FSCK on the filesystem. It’s already 43% complete so we will let this run its course.

Sincere appologies for the inconvenience caused, it couldnt happen at worse timing.

Categories
Planned Maintainance

[Resolved] vz1 issues

vz1 appears to have extremely high load and is being checked

Update: This isnt moving so giving it a hard reboot.

Update 2: FSCK is running on /vz

Update 3: VPS are now starting one by one, and we will be investigating which VPS was the cause, and take appropriate action.

Categories
Outages

[Resolved] cp1 load issues

cp1 is currently having high load issues which means some customers might see their services down.

We are working to restore service ASAP

Additional Info: The current CP1 will be retiring in the next 2 weeks, as we have a new replacement machine on order based on the latest Xeon Woodcrest platform to put a stop to these load issues. We really appreciate your patience.

Update: We are currently still waiting on on the DC for our KVM to be moved. Appologies for the delay.

Update 2: The machine is failing to boot, we are waiting for further information from onsite staff, on-call staff have been notified and are ready to go to site if required.

Update 3: Things are now back under control and the load will return to normal shortly.

Categories
Planned Maintainance

[Resolved] xn7 performance issues

xn7 is currently having some I/O performance issues.

In order to resolve this we are installing the latest kernel and Xen stack. We will be rebooting it shortly.

Update: Appologies for the delay, / needed a FSCK so we let it run. VPS are starting up one by one now.

Categories
Outages

[Resolved] xn9 Issues

xn9 appears to be suffering some filesystem issues. Currently getting some remote eyes.

Update: For some reason disk 3 decided it would belong to a new array, and caused the filesystem to go read only. This has been corrected and the RAID is rebuilding. We will continue to monitor.

Categories
Outages

[Resolved] xn7 issues

It appears XN7 has crashed, although it is responding to ping and as such was not picked up by our monitoring.

We will update you ASAP

Update: We have rebooted the machine and VPS are now starting up one by one. We are still investigating the cause of the crash.

Categories
Outages

[Resolved]Shared / Reseller Server Issues

There are currently issues with our Shared and Reseller Server CP1 which is causing sites to fail to load.

This is being investigated and all update will be here.

Update @ 15:58: This is now resolved

Update @ 15:46: This is looking like a grub issue, it is being corrected now

Categories
Outages

[Resolved] xn11 down

xn11 has crashed and is being checked

Update: Looks ok on the console but is unreachable via the network. This is rebooting

Update 2: It turns out there was a bit of fat finger syndrome and the port was taken out of the VLAN. The server is now up and VPS are starting up as well

Sorry folks!