Database borkage

We had a database master (for German Wikipedia and several other major languages) fill up its disk, breaking replication and causing some general havoc. Tim patched things up for the time being, but we still have to fix up some of the de-synced servers once they’re done rebuilding; in the meantime, you may see old data in some places or very long reported lag times for your bot tools.

See Tim’s summary of what lead to the failure and my follow-up notes about necessary fail-safe fixes and monitoring procedures.