Tech:2014 Datacenter Migration

From Orain Meta
Jump to navigation Jump to search

There will be no technical feature sacrifice. This is a major must. In the end; we will gain from a move with things such as;

  • More CPU cores available to us (3 instead of 1)
  • Private networking built in (10.x.x.x IPs)
  • Easier fail-over and emergency recover (we can recover a server to a different datacenter if necessary)
  • Fully native IPv6 support
  • Built in snapshot/backups (We will use this actively during upgrades)

In order to ensure minimum downtime for all services; there will be a general maintenance window for all of Orain. When this window begins, the migration will begin although in a specific order. The below has been planned in a way so that if server A requires B, B will move prior to A so we can start directing traffic out of the new location as soon as possible. There will be continuous updates via Twitter, IRC and here as time goes by.

The planned migration details are as below:

The window will open 15:00 UTC and close 19:00 UTC on Wednesday, 20th of August. We may not use the whole window but it gives leeway if necessary. The order of migration, with the servers current purpose and future purpose is as below:

  • prod6: Services. The purpose when moved will still be services.
  • prod3: Database server. The purpose when moved will still be a database server.
  • prod5: MediaWiki host. The purpose when moved will still be a MediaWiki host.
  • prod4: MediaWiki/Memcached/IRC RC bot service. The purpose when moved will solely be MediaWiki. Memcached will be moved to a new server and the IRC RC bot will be on prod6.

During the migration window, Orain will also be adding a new prod7 to the cluster. This new server will be dedicated to Memcached and may be purposed off as a loadbalancing host.

  • Memcached has been planned to be phased out in favour of Redis. When we move - this will happen as Redis can be repurposed as well for other cool stuff.

We have taken this time to ensure the infrastructure will be scalable both vertically and horizontally. The main objective is to increase redundancy. We want it to be so if a single host failure it will no longer is a 'nuclear war' type reaction but instead a 'we should fix that'. Lately we have been increasing our monitoring infrastructure and hope this migration will provide us with even more amazing opportunities to make it a single byte of discrepancy will get to us immediately.

When the time to migrate prod3, prod4 and prod5 comes - all wikis will be put into read-only mode and all databases will be locked. We do not expect this to last long and we will keep RamNode open so we can fail back if necessary. 'migrationwiki' will be created prior to this plan going up and will be served from the new datacenter immediately when available. Please keep an eye on this wiki for progress. Below are a few meta details:

  • Technical staff on hand for migration: John
  • Support staff on hand for migration: Dusti
  • Migration location checked for technical features by: John
  • Migration approved by: Dusti

Now, on behalf of Orain staff, John (talk) 15:05, 13 August 2014 (BST)

If you have any questions, please do not hesitate to ask them. You can ask your questions here or on IRC (webchat). Southparkfan (talk) 17:44, 13 August 2014 (BST)
Retrieved from "https://meta.orain.org/wiki/Tech:2014_Datacenter_Migration?oldid=13513"