Most quarterly security reviews still open with uptime slides. They are tidy, familiar, and almost useless for a mid-market board trying to judge whether security spend is working. That is the author’s view, and it is the problem this guide addresses. When you work with a managed security services provider MSSP, the quarterly service review should be a working agenda, not a status presentation. This post shows you how to turn that meeting into a decision-making forum built on five numbers: detection coverage against the asset estate, mean time to contain, escalation accuracy in both directions, false-positive load, and the ageing of open remediation items. You will learn what to have in place before the first review, how to read each metric against baseline, and what a defensible remediation commitment looks like when a number misses. We will also cover why detection speed alone is not enough, why false positives carry hidden costs, and how a fixed quarterly cadence makes drift visible before it becomes an incident. Finally, you will see how to turn the review into a governance habit that holds up over time.
Why an Uptime Slide Cannot Tell a Board Whether Security Spend Is Working
This agenda rests on one argument: an availability dashboard measures the health of a platform, not the outcome of a security service. A provider can post near-perfect uptime while half your estate is not feeding detection at all.
Boards approve renewal budgets on outcomes: risk reduced, dwell time shortened, exposure retired. Activity reporting, meaning alerts triaged, tickets closed and tools deployed, describes effort rather than effect. NIST CSF 2.0 offers a taxonomy of high-level cybersecurity outcomes rather than prescribing how they are achieved, which is the register a board discussion needs.
This matters for managed security services provider relationships: the buyer must bring the outcome frame to the review. Moving from alerts to answers faster is the operational version of that shift.
Each of those five measures is defined in the sections that follow.
What to Have in Place Before the First Quarterly Review
Before metrics mean anything, five things need to be on the table.
An agreed asset inventory. Endpoints, servers, cloud accounts, identities, and operational technology, with one owner and a review date. Detection coverage is meaningless without a denominator both sides accept. Treat third parties with the same discipline: Supply Chain Cyber Risk: Auditing and Managing Third-Party Threats in Mid-Market walks through the audit workflow.
A written baseline. Record the first review’s numbers in the minutes, so every later quarter is measured against a recorded reference point rather than a memory of how things felt.
Documented service scope and escalation criteria in the managed security services agreement, so escalation accuracy is scored against an agreed standard rather than the provider’s judgment in the moment.
One accountable owner on your side, a CISO, IT director, or COO, with authority to sign remediation commitments. As a rule of thumb, send the data request at least five business days before the meeting.
Agreed reporting granularity and format, so your team can recompute every metric from raw data instead of accepting a summary number on trust.
Metric 1: Detection Coverage Against the Asset Estate
With inventory and baseline locked, the first metric to score is detection coverage: the share of the agreed inventory actually feeding detection, broken out by asset class and log source, not alert volume.
Ask for onboarding status per asset class, how new assets are discovered and ingested, and the reconciliation cadence against the asset register. A flat percentage while the estate grows means the absolute gap is widening.
Drift appears after cloud migrations, acquisitions, or unmanaged SaaS adoption, or when numbers improve only because the denominator shrank. NIST SP 800-137, Information Security Continuous Monitoring, provides a reference framework for ongoing monitoring as a voluntary discipline for non-governmental organisations.
Put these questions to your managed security services provider: which asset classes fall outside scope, what the discovery method is for shadow IT, and how often coverage is reconciled to the register.
Demand a dated coverage target for each named gap, with the reconciliation method documented so next quarter’s figure can be verified independently. A disciplined security risk assessment for mid-market organizations supplies the inventory that becomes that denominator.
Metric 2: Mean Time to Contain, and Why Detection Speed Alone Is Not Enough
Where coverage measures what is watched, mean time to contain measures what happens when a threat is found. Mean time to detect measures how long until your managed security services provider knows something happened; mean time to contain measures how long until the threat stops spreading or acting.
Never accept an average alone. Request p50 and p90 containment times, segmented by severity and detection source (endpoint, identity, cloud, email), because a falling average can mask a worsening long tail. Watch for identity incidents climbing while endpoint stays flat; that pattern usually points to an identity telemetry or privileged access gap, not staffing.
Ask how containment is defined and timestamped: first action (isolation, session revocation, account disable) or full eradication, and who records the clock. Demand severity-tiered containment targets plus a written escalation trigger if a critical incident breaches its target twice in a quarter. Our guide to MDR for Mid-Market: What to Demand From Your Provider Before Signing covers the contract language that makes those targets enforceable.
Metric 3: Escalation Accuracy, Measured in Both Directions
Containment time assumes the right incidents reach your team. Escalation accuracy tests that assumption in both directions: of everything escalated to you, how much was a genuine, correctly prioritized incident, and of everything that should have been escalated, how much reached you in time.
Score over-escalation and under-escalation separately. The first burns analyst and executive attention. The second is the failure mode that becomes a breach notification.
Judge both against the criteria written into the managed security services agreement, not against how urgent the alert felt that night. Criteria that live only in a provider’s workflow drift; identity and email alerts are where that drift often surfaces first, the same gaps phishing-resistant technical controls are designed to close.
Drift has a signature: escalation volume rising while the severity mix stays flat. That indicates criteria drift or a tuning backlog, not more threat activity.
A practical sample size is 10 to 20 sampled escalations per quarter, reviewed jointly by both parties, with any criteria changes recorded in writing before the next quarter.
Metric 4: False-Positive Load and the Hidden Cost to Your Team
The third metric scores whether the right alerts reach you; this one scores what the wrong ones cost.
Measure it as the share of escalations your team closes as benign, per 100 handoffs, then convert that into hours consumed monthly. Multiply those hours by your loaded analyst rate and the result sits directly against the managed security services invoice, where false positives stop reading as an annoyance and start reading as duplicated spend.
Read the trend, not the absolute rate. A jump after a tooling change, a scope expansion, or a new cloud environment is expected; a jump with no such change behind it points to tuning failure. That is the logic behind threat intelligence platforms that turn raw feeds into actionable defense: context separates signal from noise.
Drift looks like one detection rule firing quarter after quarter with no suppression or logic change logged, or tuning requests open without an owner. Demand a tuning backlog with named owners and target dates, plus a false-positive ceiling re-examined each review.
Metric 5: Ageing of Open Remediation Items
The fifth metric closes the loop on stalled work.
Track every finding in four buckets: 0 to 30, 31 to 90, 91 to 180, and 180-plus days. A single open-item count hides whether remediation is moving or stalled. Split the list by owner, because your team and your managed security services provider fix different things on different timelines.
Read the shape, not the total. If the 91-day-plus bucket grows while the headline count holds steady, work has stalled. Treat critical items moved to accepted risk without a documented decision owner, a review date, or a compensating control as drift; that is how a quarterly review becomes a formality.
Demand an ageing cap for critical and high findings, a monthly internal checkpoint between reviews, and a named owner for every item past 90 days. Closing findings starts with accurate scanning data, which is why a vulnerability scanner guide matters to this metric.
How to Read the Five Numbers Against Baseline
With all five metrics defined, the review now turns on what you compare them to.
Set the baseline, then calibrate
We recommend treating the next two quarters as calibration, not judgment; early variance usually reflects data quality, not service quality.
Compare like with like
If scope, estate size or tooling changed, restate the prior period before declaring improvement or decline.
Keep it on one page
One workable format is a scorecard with five columns per metric: current quarter, prior quarter, baseline, direction of travel, named owner.
Separate noise from drift
One quarter is a data point. Two consecutive quarters moving the same way is a trend worth a remediation commitment.
Avoid blended scores
Resist composite security scores; collapsing five metrics into one hides which part of the service is failing. If the board wants a single number, give them a quantified risk figure instead, as covered in Cyber Risk Quantification: What Your Board Actually Needs to See, and keep the operational scorecard unaggregated.
What a Defensible Remediation Commitment Looks Like
Drift confirmed, the commitment decides whether anything changes. Five tests separate a defensible commitment from a promise: it names the metric that missed, states a root cause, assigns one owner, carries a specific due date, and specifies how completion will be verified.
Illustrative template, not a client result: metric missed (coverage fell short of the agreed inventory target); root cause (cloud accounts onboarded without log ingestion); corrective action (ingest those accounts); owner (provider detection lead); due date; verification (buyer recomputes coverage from raw data); escalation path if the date slips.
Record every commitment in the quarterly minutes and tie it to the service review clause of the managed security services agreement, so it survives staff turnover on either side. Agree the consequence before you need it: a service credit, executive escalation to provider leadership, or a formal remediation plan review, never a negotiation after the miss. Name your own misses too, poor asset data or a change freeze; a review that only scores the provider will not fix the metric that broke.
Run It on a Fixed Quarterly Cadence So Drift Shows Up Early
A commitment only holds if the review that produced it recurs on schedule. Run it in the same week each quarter, so periods stay comparable and no one has to reconstruct changes from eight weeks back.
Keep a standing agenda: scorecard walkthrough, deep dive on any red metric, tracking of last quarter’s commitments, then next quarter’s commitments. As a rule of thumb, circulate the pack at least five business days out and discuss exceptions only; named owners handle detail offline. That typically keeps the meeting to 60 to 90 minutes.
A practical trigger is to escalate between quarters rather than waiting for the next session when a critical metric breaches its target twice.
Finally, maintain a rolling one-page history so a new CFO, CISO, or board member can aim to see at least two years of direction at a glance.
Treat the cadence itself as a control. A review that slips is the first sign the program is drifting.
Turning the Review Into a Governance Habit
The sections above have defined each element; this section names the habits that hold them together.
- Own the math. Ask for the five metrics in a format your team can recompute from raw data. NIST SP 800-55 Volume 1 describes a flexible approach to measure selection, and the guidance is voluntary for nonfederal organizations, so adapt this set to your estate rather than treating it as a standard handed down.
- Write the commitments down. Every remediation item gets one owner, a specific date, and a verification method in the minutes, with an agreed consequence ready before it is needed.
- Judge direction of travel. Show the board two consecutive quarters moving the same way as the trend that matters.
Together, these habits turn a recurring meeting into a control you can defend.
Conclusion
A quarterly review proves managed security value when it moves beyond uptime slides. Together, those five metrics and the cadence described above give the board a defensible account of where security spend is working and where it is not.
Next step: schedule the first review, lock the asset inventory, and bring the five metrics in a recomputable format. Treat the cadence as a control. A program that can show its direction of travel earns trust and improves quarter after quarter.
