Set Up Application Monitoring and Observability

Set up a complete monitoring and observability solution for the following application:

Application type: [web app / API / microservices / background workers / all]
Language/framework: [specify]
Infrastructure: [cloud provider and services used]
Current monitoring: [none / partial — describe]
Key pain points: [what goes wrong that you can't see / debug slowly / find out from users]
Team size: [solo / small / large — affects tooling choices]
Budget: [open source only / modest SaaS budget / enterprise budget]
Compliance requirements: [data residency / log retention rules]

Provide a complete observability implementation:

THREE PILLARS SETUP:

LOGGING:
- Logging library and configuration
- Log levels and when to use each
- Structured logging format (JSON schema)
- What to log at each level
- Log aggregation setup (ELK / Loki / CloudWatch / Datadog)
- Log retention policy
- Sensitive data handling in logs

METRICS:
- Metrics library setup (Prometheus / StatsD / OpenTelemetry)
- The 4 Golden Signals (latency, traffic, errors, saturation) implementation
- Custom business metrics to track
- Metrics storage and visualisation (Grafana / etc.)
- Dashboard designs for: engineering on-call / engineering daily / product/business

DISTRIBUTED TRACING:
- Tracing library setup (OpenTelemetry / Jaeger / Zipkin)
- Instrumentation of key code paths
- Trace propagation across services
- Sampling strategy

ALERTING:
- Alert rules for critical conditions (with thresholds)
- Alert severity levels and routing
- On-call rotation setup
- Alert fatigue prevention
- Runbook template for each critical alert

ERROR TRACKING:
- Error tracking setup (Sentry / Rollbar / etc.)
- Error grouping and prioritisation
- Release tracking
- User impact measurement

SYNTHETIC MONITORING:
- Uptime checks
- End-to-end user journey monitoring
- Performance benchmarking

ON-CALL RUNBOOKS:
- Template for the 5 most common alerts
- Escalation path documentation