The 3 a.m. error spike that was only a deploy Tells a traffic surge from a broken release before anyone is paged.
- CaptureErrors, traffic and the last deploy
- JudgeRules and AI rate the severity, 0 to 10
- ActSlack at 7 or more, once every 3 hours at most
- What it captures
- 5xx count for the last hour and the hour before from Kibana Discover, all requests for the last hour, the traffic overview dashboard, and the CI page with the last deploy time. The built-in Kibana AI Triage template captures all of these except the deploy page; you add the delivery.
- What the AI is asked
- Rate the severity from 0 to 10. More errors with more traffic is load. More errors with flat traffic is a failure. Errors that start within ten minutes of a deploy are a bad release. Everything at zero means the source stopped.
- What happens next
- Below 7 the run stops and nobody is paged. At 7 and above a second AI action writes a four-sentence briefing with the numbers, and Slack gets it with the dashboard attached.
Why a plain threshold is not enough. A threshold on the error count fires on the Black Friday surge and stays quiet on a broken release with low traffic. The AI sees the traffic curve next to the error count.