Recoverability and awareness beat heroics. This guide covers backups, monitoring, alerting, and incident basics for sites and apps on Build Me Web.
Backups
Automated database and media backups run on a schedule appropriate to your plan. Retention should match legal and business needs. Test restores quarterly — an untested backup is a rumor.
Know your RPO (how much data you can lose) and RTO (how fast you must restore). Point-in-time recovery, when available, still needs a practiced runbook.
- Confirm backup schedules in Cloud Backups.
- Store restore credentials and steps in your ops wiki.
- Perform a restore drill into an isolated environment.
- Verify application boot and spot-check critical records.
Media backups are useless if only the database is restored without object storage. Drill both.
Monitoring
Track uptime, error rates, latency, and Core Web Vitals. Segment by template or route group so a blog regression does not hide a checkout issue. Synthetic checks should hit critical user journeys, not only /.
Logs: structured JSON with request IDs. Metrics: RED (rate, errors, duration) for services; USE (utilization, saturation, errors) for resources.
Alerting
Page on symptom thresholds that matter to customers. Route noise to digests. Include runbooks in alert payloads so on-call is not starting from a blank page at 2 a.m.
- Severity definitions agreed by the team.
- Inhibition rules so one outage does not page fifty times.
- Maintenance windows that silence expected deploy blips.
Incident basics
Acknowledge, mitigate, communicate, then review. Status page updates keep stakeholders informed without flooding support. Separate incident commander from people fixing systems when the event is large.
- Restore drill completed in the last 90 days
- On-call rotation and escalation path published
- Status page access tested
- Post-incident template ready
Troubleshooting
Alert fatigue: raise thresholds thoughtfully and fix flappy checks — do not mute forever.
Backup failures unnoticed: alert on backup job failure specifically.
Blind spots: add synthetics for login and checkout, not just homepage HTML.
Prerequisites & preparation
Before changing production settings for backups and monitoring, align the people who own content, DNS, analytics, and approvals. A fifteen-minute kickoff that names owners prevents multi-day Slack archaeology later. Capture decisions in the workspace notes so the next teammate inherits context instead of guesswork.
Gather credentials and access: workspace admin or editor role, DNS control when hostnames are involved, payment or API sandbox accounts when money paths are involved, and a shared checklist link. Confirm which environment you will rehearse in — preview first, production only after a green run.
Write RPO/RTO targets and name on-call rotations before enabling noisy alerts.
Detailed walkthrough
Work the happy path slowly the first time. Narrate what you expect to see after each click: a status badge, a DNS record, a webhook delivery, a Lighthouse metric. When reality diverges, stop and resolve the mismatch instead of clicking ahead — most outages begin as ignored yellow states.
- Confirm backup schedules and retention; store restore steps in the ops wiki.
- Run a restore drill into an isolated environment including media.
- Deploy synthetics for homepage, login, and checkout start.
- Tune alerts to customer symptoms with runbook links.
- Rehearse status page updates on a tabletop incident.
Edge cases & failure modes
Plan for partial failure. Networks drop, registrars delay, providers rate-limit, and humans approve the wrong revision. Your runbook should say what “abort” looks like: leave preview up, roll back the release, or freeze campaigns until metrics recover.
- Backup jobs failing silently without dedicated alerts.
- Alert fatigue causing ignored pages.
- Synthetics that only fetch `/` while checkout is down.
QA checklist before you announce
- Happy path verified on mobile and desktop
- Failure path messaging reviewed
- Owners named for the first hour after launch
- Rollback or freeze path documented
Operating the change
After launch, watch the metrics that prove the change worked — not vanity charts. Pair quantitative signals with one qualitative check (support ticket themes, sales feedback). Schedule a follow-up within a week to remove temporary flags, raise DNS TTLs, or archive the experiment.
Quarterly restore drills and post-incident reviews with action owners.
Treat backups and monitoring as a repeatable playbook. The second time your team runs it should be faster because the checklist and owners already exist.
Untested backups are rumors — drill until restore time is a known number.
Structured logs with request IDs turn “it felt slow” into actionable traces.
Severity definitions prevent every blip from becoming a SEV1.
Maintenance windows should silence expected deploy noise without hiding real outages.
Separate incident command from deep technical fixing when events grow large.
Media and database restores both matter; app boot is not enough without objects.
Implementation checklist
Confirm RPO and RTO targets written down on preview before you promote. Name who approves the change, which environment is authoritative, and what “done” looks like in measurable terms.
Document restore drills including media objects where the team already looks — workspace notes or the engineering handbook — not only in a meeting memory.
Operating it with your team
Rehearse synthetics for login and checkout against written acceptance checks. If you cannot name the verification event (order, DNS lookup, deploy health, or signed API call), you are not ready to announce.
When you finish this guide, you should be able to explain the happy path, the abort path, and the metrics that prove success. If any of those are fuzzy, revisit the walkthrough with your teammates before you scale traffic, spend, or automation.