We are aware of performance issues with our Xen VPS which has been introduced by security patches applied yesterday. You may see increased load averages and general sluggishness inside your VPS caused by CPU Steal.
We have a fix to resolve this, and will be rebooting each VPS node shortly to apply a fix. A further update will be posted here when complete.
Update: As of 19:00 UTC, this is completed and all VPS are operating normally.
We are currently carrying out essential unplanned maintenance on our VPS infrastructure, to address recently released security vulnerabilities. This will involve a reboot of all VPS host machines.
We apologise for the lack of notice, however this is for the benefit of all customers to ensure the ongoing security and stability of our infrastructure.
This is completed.
web1.cpservers.info is currently having some intermittent timeout issues due to a sophisticated peice of malware which goes after the Apache binary.
This malware takes control of httpd and inserts code to interrupt or redirect requests, for the last few days we have been mitigating the symptoms as best as possible.
We believe this has come across on a recent migration from another server, and as it’s incredibly difficult to pinpoint we will be reinstalling the system to ensure its fully removed. There is no risk to the security of individual accounts or the integrity of your data.
21/01/14 20:15 – We started generating a new backup set containing all account data at approx 6.30PM, and there are now only around 15% of accounts remaining. Assuming this completes within the next 45-60mins we will be proceeding with the reinstall.
21/01/14 20:47 – Backup completed, reinstall in progress.
21/01/14 21:26 – cPanel install is now well underway, should be approx 45mins then we can begin account restoration.
21/01/14 22:37 – The server has now been fully configured, and account restorations have been running for around 30mins. It will take several hours for all accounts to restore.
22/01/14 18:09 – Restores have almost completed, unfortunately this has taken a while as cPanel’s new backup system will only restore accounts one by one on a queue based system. The majority of clients should already be up at this point.
Restoration has been completed.
We currently have several nodes offline / with packet loss.
Xn4 and XN6 are restored.
Update: xn5 is now restored, please be aware of some I/O wait until the system settles down