Tech:Incidents/2014-07-07

From Orain Meta
Jump to navigation Jump to search

On Monday July 7th, Orain experienced frequent database issues for 4 hours.

Timeline

  • 14:09 JohnLewis: issues seem to be resolved
  • 14:08 JohnLewis: reboot prod3
  • 14:00 JohnLewis: prod3 seems a bit slow and unresponsive
  • 13:50 JohnLewis: I become a aware of the issue and start investigating
  • 11:24 Addshore: Investigates the issue briefly
  • 11:15 JohnLewis: Southparkfan emails sysadmin at orain.org reporting issues
  • 10:25 JohnLewis: Nagios - MySQL is returned as 'critical'
  • 10:02 JohnLewis: Pingdom reports Orain is down