The Inspector With No Fixed Agenda#
In 1971, the US Occupational Safety and Health Administration (OSHA) was established to regulate workplace safety. The inspection programme that OSHA implemented — and which has remained its primary enforcement mechanism — uses a variant of random targeting: workplaces are selected for inspection through a priority system that includes targeted high-hazard industries, complaint-triggered inspections, and programmed random inspections of establishments whose injury and illness rates suggest under-reporting. The exact date and specifics of an inspection are not known in advance to the employer being inspected.
The practical consequence is that an American workplace subject to OSHA jurisdiction cannot fully optimise for inspection conditions the way an automotive manufacturer can optimise for a published test cycle. The employer does not know what equipment will be examined, which procedures will be observed, or which documentation will be requested. The cost of reaching compliance only when inspected — and reverting to non-compliance between inspections — is higher when inspections are non-deterministic, because the non-compliance period is bounded only by the stochastic inspection arrival rate, not by a fixed known calendar. The MPDI between OSHA inspection results and actual workplace safety conditions is not zero, but it is substantially lower than it would be if OSHA published inspection schedules in advance.
The randomised inspection model is not universally applicable to product certification — a vehicle cannot be road-tested by a regulator following every car that leaves the factory. But the principle it embodies — that MPDI decreases when the tested entity cannot predict test conditions — is the design basis for the most effective regulatory responses to the Goodhart trap.
The Structural MPDI Reduction Toolkit#
Real-Driving Emissions: In-Use Testing at Scale#
The EU's Real Driving Emissions (RDE) programme, phased in from 2017 to 2021 for new vehicle type approvals, represents the most thoroughly developed implementation of post-market randomised in-use testing for automotive certification. Vehicles are equipped with Portable Emissions Measurement Systems (PEMS) — equipment that measures actual tailpipe emissions by weighing exhaust gases in real-time — and driven on public roads under conditions chosen by the test driver rather than prescribed by a fixed protocol. The route includes urban, rural, and motorway phases but is not a fixed sequence; the specific roads, traffic conditions, acceleration events, and ambient temperatures vary by test execution.
The PEMS-based RDE result is compared against the type-approval limit with a "conformity factor" that accounts for measurement uncertainty and the realistic variability of real-world driving. Current conformity factors allow a maximum of 2.1× the type-approval NOx limit in RDE testing — meaning a vehicle certified at 80 mg/km NOx may emit up to 168 mg/km in real driving and remain compliant. This is higher than environmental advocates prefer and represents an imperfect implementation. But the directional MPDI reduction from RDE is documented: ICCT analysis of the EU fleet shows that real-world NOx emissions from new diesel passenger cars fell from approximately 5–6× the certification limit in 2016 (under NEDC-only certification) to approximately 2–3× the limit by 2022 (under WLTP + RDE). The MPDI was nearly halved in six years through regulatory structure change alone, without any change in engine technology.
The mechanism is direct and consistent with the Goodhart analysis: manufacturers can no longer calibrate exclusively for a fixed-cycle result, because the RDE test observes the vehicle in a variety of non-predictable conditions. The optimisation target has become "perform well across a wide range of real driving conditions" — which is a closer proxy for "comply with the underlying air quality objective" than "perform well in the specific WLTP cycle."
Post-Market Surveillance in Medical Device Regulation#
The FDA's post-market surveillance requirements for Class III medical devices (high-risk implants and life-sustaining equipment) provide a regulatory model for managing MPDI in pharmaceutical and device certification. Following initial pre-market approval (PMA), Class III device manufacturers are subject to mandatory Medical Device Reporting (MDR) requirements — reporting of adverse events to an FDA database — and are required to conduct post-approval studies (PAS) that gather real-world performance data from implanted device populations.
The post-approval study requirement functions as a MPDI correction mechanism: certified performance based on the initial PMA clinical trial is supplemented by real-world outcome data from a representative deployment population. If the PAS reveals that real-world outcomes diverge significantly from certified trial outcomes — as has occurred for some Class III devices, including certain models of metal-on-metal hip restorations — the FDA can require labelling updates, restrict indications, or mandate device removal programmes.
The medical device PAS system is imperfect: PAS compliance rates have historically been less than 100%, and the FDA has occasionally struggled to enforce PAS completion within the mandated timelines. But the structural principle — that the certification record is not closed at first approval but is continuously updated with real-world performance data — creates an ongoing feedback mechanism that MPDI cannot permanently escape. The certified performance is compared, over time, against the actual performance, and the certified performance can be revised if the divergence exceeds acceptable limits.
Outcome-Based Contracting in Healthcare Procurement#
The NHS and several European health systems have experimented with outcomes-based contracts for pharmaceutical products — arrangements where the final payment to the drug manufacturer is conditional on the drug achieving a defined real-world outcome (typically a clinical endpoint in deployed patient populations) rather than payment at the time of prescription. The Novartis gene therapy Zolgensma for spinal muscular atrophy, priced at approximately $2.1 million per patient in the US, was structured around an outcomes-based payment arrangement in which Novartis would provide rebates if the treatment failed to maintain defined efficacy thresholds in the treated population.
Outcomes-based contracting converts the MPDI from an abstract regulatory concern into a direct financial liability for the manufacturer. A manufacturer confident that real-world performance will match certified trial performance has no financial incentive to resist outcomes-based payment. A manufacturer concerned that real-world performance will diverge significantly from certified performance will price the uncertainty into the payment arrangement. The pricing signal itself is information about the manufacturer's private assessment of their MPDI. Procurement systems that use outcomes-based contracts are, in effect, asking manufacturers to bet their own money on the accuracy of their MPDI.
The Meta-Requirement for Low MPDI#
The common structural features of regulatory contexts that achieve low MPDI — RDE testing, post-market surveillance, outcomes-based payment — are:
- In-use observation that the regulated entity cannot fully predict in advance, preventing optimisation specifically for the test.
- Post-certification feedback linking actual in-use performance to the certification record, creating ongoing accountability rather than one-time approval.
- Financial consequence for MPDI, ensuring that the gap between certified and actual performance falls on the entity that produced it rather than on the user or regulator.
These features do not require technologically sophisticated measurement apparatus. They require regulatory structures with the institutional commitment and resource allocation to sustain ongoing in-use monitoring — which is the harder constraint. Real-driving emissions testing requires PEMS equipment and trained testing teams operating on public roads; post-market surveillance requires analysis of adverse event databases and enforcement of study completion; outcomes-based contracting requires actuarial sophistication and contract enforcement infrastructure.
The MPDI is ultimately a statement about institutional design: how much resource and authority the measurement apparatus has to look at the real world rather than the test room. The gap between certified and actual performance is, in precise terms, the gap between what the measurement apparatus can see and what the product actually does. Closing that gap requires not a better test — it requires a different relationship between the regulator and the product across the full lifecycle of its deployment.
