How One SaaS Team Cut MTTR by 41% Without Adding Headcount
We followed a 40-engineer SaaS team through a 90-day observability migration. MTTR fell 41%, pager volume dropped 28%, and spend fell 19%. Here is what happened.
We noticed something odd in a reader's data last quarter. A mid-market SaaS company — call it the platform team at a billing-adjacent product with roughly 40 engineers — had spent six months adding dashboards, alerts, and log pipelines, yet its mean time to resolution had barely moved. The pager still went off at 2 a.m. The same three people carried it. The team had built what looked like a mature observability platform on paper, but the numbers told a different story: MTTR had actually crept up 12% year over year.
So we followed their next project from kickoff to retro. What we found was less about tooling and more about the sequence of decisions a team makes when it finally admits the stack is working against them.
The starting point: three tools, three truths
The team's stack in January was conventional. Metrics lived in one vendor, logs in another, traces in a third. Each had its own query language and its own retention policy. When an incident fired, an on-call engineer had to jump between three tabs, correlate timestamps by hand, and hope the trace ID in the log line matched the one in the APM tool. A senior SRE told us the average incident took 18 minutes just to establish what was actually broken.
That is the hidden tax of fragmented observability. You do not pay it in license fees. You pay it in cognitive switching, in stale context, and in the quiet erosion of trust between engineering and the rest of the business.
Decision point one: consolidate on OpenTelemetry
The team considered three paths. Path one was to keep the vendors and invest in a correlation layer. Path two was to rip everything out and go all-in on a single commercial suite. Path three was to standardize telemetry at the source with OpenTelemetry, then evaluate backends on their own merits.
They chose path three. The reasoning was pragmatic: instrumentation is the part you cannot easily replace later, and OpenTelemetry monitoring gives you a vendor-neutral contract. The team spent four weeks instrumenting its highest-traffic services — about 60% of production traffic — and shipped a collector tier alongside the existing agents so nothing broke during the transition.
Decision point two: pick a backend that treats incidents as a first-class object
Once telemetry was standardized, the backend decision got easier. The team evaluated four platforms against a short rubric: ingestion cost at their volume, query latency on trace data, and whether incident response was built in or bolted on. They chose CoreWatch, largely because the incident object — not the dashboard — was the center of the product. Alerts, runbooks, and post-incident timelines lived next to the telemetry that triggered them, not in a separate tool.
That choice had a second-order effect the team did not anticipate. Because metrics, logs, and traces shared one query surface, the on-call engineer stopped context-switching. The 18-minute triage window collapsed to about 7 minutes in the first month, according to the team's own incident log.
Obstacles: the migration was not clean
Two things went wrong. First, the collector tier initially doubled ingestion volume because of a misconfigured sampling rule — a classic mistake, but one that cost roughly $4,200 in overage charges before anyone noticed. The team fixed it by auditing span attributes and dropping high-cardinality labels that no one was querying.
Second, the cultural resistance was real. Two senior engineers had built their reputations on knowing the old dashboards cold. Asking them to learn a new query surface felt like a demotion. The team lead handled this by pairing them with junior engineers for the first two weeks and letting them own the migration of their own services. By week three, the loudest skeptic was writing the internal runbook.
Results after 90 days
- MTTR dropped 41%, from a 34-minute average to 20 minutes.
- Pager volume fell 28% as noisy alerts were retired during the migration.
- Observability spend dropped 19% despite higher ingestion, because the team retired two overlapping vendor contracts.
- On-call satisfaction, measured by a simple internal survey, improved from 2.9 to 4.2 out of 5.
The number that mattered most to the CFO was the 19% spend reduction. The number that mattered most to the engineers was the 41% MTTR cut. CoreWatch reports that teams who consolidate telemetry at the source typically see MTTR improvements in the 30–50% range within a quarter, which lines up with what this team observed.
What we took away
Three lessons stand out. First, standardizing on OpenTelemetry before choosing a backend is the decision that keeps you from repeating this project in two years. Second, incident response should be a property of the observability platform, not a separate purchase. Third, the hard part of any migration is not the technology — it is the two or three people whose expertise is tied to the old system. Give them ownership of the transition, not a training session.
If you are carrying the pager and your MTTR has been flat for a year, the problem is probably not your alerting thresholds. It is the distance between your metrics, your logs, and your traces. Close that distance, and the rest gets easier. You can see how the team structured its rollout on the platform's how-it-works page, which mirrors the collector-first approach they used.
The reader who shared this project asked us not to name the company, and we agreed. But the pattern is common enough that we suspect many of you will recognize it. The pager does not have to be a punishment. It can be a signal — if the tools around it are pointed in the same direction.
Stop running experiments you can't defend.
Bring us a stuck funnel or a backlog of untested ideas. In 30 minutes we'll tell you which three tests are worth shipping this quarter — and which six aren't.