Questions to Ask Before a Regulatory Impact Assessment
Frame an impact assessment around the problem, baseline, options, affected groups and evidence gaps before selecting an analytical method.
A regulatory impact assessment is easier to scope than to write. Before drafting, establish the problem being described and for whom, the counterfactual if nothing is done, the alternatives that deserve comparison, who bears the burdens and how they are distributed, the pathway through which effects are expected, the evidence that would be sufficient, and how uncertainty and review will be handled. These questions determine the shape of the assessment; asked late, they tend to be answered backwards from a preferred option.
What preparatory questions are for
Regulatory impact assessment works as a decision aid: it sets out a problem, compares options, and estimates likely effects with stakeholder input and analysis proportionate to the significance of the proposal. The OECD's regulatory policy guidance describes the task in roughly those terms (OECD, regulatory impact assessment guidance); the guidance is policy design material, not legal advice, and it does not determine what applies in any particular jurisdiction.
Two consequences follow. First, naming an option or costing a burden does not establish legal applicability or measured impact; the assessment states what is expected and on what basis. Second, the depth of analysis is a judgement that should be justified rather than assumed. A proposal with narrow, reversible effects may warrant a short assessment. One that changes reporting duties across a sector may warrant a longer one. The preparatory questions below are the ones that most often change the scope once answered honestly.
The questions
What problem is being described, and for whom?
Write the problem as a statement about observable conditions, not about the absence of the preferred instrument. "Response times for a service vary widely and are not published" is a problem statement. "There is no service standard" is a description of a missing policy. The distinction matters because the second version smuggles in the remedy.
Ask also whose problem it is. A problem experienced by service users, by providers, and by an oversight body may share a label but require different evidence and different remedies. Where the problem statement cannot be separated from a proposed solution, that is worth recording as a limitation.
What happens without intervention?
The counterfactual is the most frequently skipped question and the one that most affects reported effects. Without a baseline, any change after intervention can be attributed to the intervention.
Useful sub-questions: What are current levels, and how are they measured? Is the condition stable, improving, or worsening for reasons unrelated to policy? Are other changes already scheduled that would alter the baseline? If no intervention were taken, what would plausibly happen over the assessment period, and with what confidence?
Where current data are thin, the honest answer may be a range with stated assumptions rather than a point value. That is a legitimate output of scoping.
Which alternatives merit comparison?
The option set should include a genuine baseline (usually the counterfactual above) and at least one alternative that is not simply a weaker version of the preferred instrument. Options that differ only in stringency tend to produce a comparison of numbers rather than of mechanisms.
Ask of each option: What instrument does it use — information, prohibition, licensing, reporting, price? Whose behaviour must change for it to work? What would have to be true for this option to fail? Including a non-regulatory option, even where it is unlikely to be adopted, often clarifies what the regulatory option is actually contributing.
Who is affected, and how are burdens distributed?
Aggregate burden figures can conceal the distribution that determines whether a proposal is workable. Identify affected groups by their exposure and capacity: those who must change behaviour, those who must produce information, those who absorb costs, and those who receive benefits.
Then ask distributional questions specifically. Are costs concentrated on a small number of actors with limited capacity to absorb them? Are benefits diffuse and delayed while costs are immediate? Are there groups whose position would worsen even if total welfare improves? A burden that is small in aggregate can still be decisive for a particular category of organisation, and the assessment should say so rather than average it away.
Through what pathway is the effect expected?
An effect pathway states the sequence of expected change: the instrument alters a requirement, the requirement alters behaviour, the behaviour alters the outcome. Each link carries assumptions that can be tested.
For example: publication of response-time data leads providers to prioritise waiting times, which leads users to make better-informed choices, which leads to shorter waits. That chain assumes the published measure is comparable across providers, that providers attend to reputational exposure, and that users can act on the information. Naming those assumptions converts a claim into something the assessment can examine and the review can revisit.
What evidence exists, and what standard does it need to meet?
Evidence questions should be asked before evidence is gathered, because they determine whether the assessment can support the decision. Ask: What evidence is already held, and by whom? What would count as sufficient for this decision — administrative data, a small pilot, consultation responses, published research from another jurisdiction? What is the weakest link in the chain, and how much weight is the conclusion placing on it?
Transferred evidence deserves particular care. A finding from one jurisdiction may rest on licensing definitions, enforcement practice, or institutional arrangements that do not travel. Where evidence is transferred, record the conditions under which it was produced and whether they hold here.
What remains uncertain, and how will it be treated?
Uncertainty is not a residual category to be noted at the end. Decide at scoping how it will be handled: stated as a range, tested through sensitivity analysis, flagged as a question for consultation, or named as a reason for a staged approach or a review clause.
Unknowns should stay unknown. Filling a gap with a plausible figure and dropping the caveat is the failure mode this question exists to prevent.
How will the assessment be reviewed and revisited?
Decide early what the review will examine. Three kinds of evaluation answer different questions: process evaluation examines whether implementation occurred as designed; impact evaluation examines change attributable to the intervention; value-for-money evaluation weighs benefits against costs. A review clause that does not specify which of these it is commissioning tends to produce a report that answers none of them well.
Ask also what would count as evidence that the assessment was wrong, and when that evidence would be available.
A scoping sheet to adapt
The following is a suggested working format, not a validated instrument or an institutional requirement. It is designed to be completed before drafting and revised as evidence arrives.
| Field | Prompt | What to record |
|---|---|---|
| Problem | What observable condition is being addressed, and for whom? | A statement about conditions, plus whose experience it describes |
| Baseline | What would happen without intervention? | Current levels, trend, measurement source, and confidence |
| Options | Which alternatives are genuinely in play, including a non-regulatory one? | Instrument type and the behaviour each option must change |
| Affected groups | Who changes behaviour, who produces information, who bears costs, who benefits? | Groups identified by exposure and capacity, not only by size |
| Effect pathway | How is the instrument expected to produce the outcome? | The sequence of links and the assumption behind each |
| Evidence | What is held, what is missing, what standard does this decision need? | Sources, their origin, and conditions for transfer |
| Uncertainty | What is unknown, and how will that be handled? | Ranges, sensitivity, consultation questions, staged approach |
| Review | What will be evaluated, when, and against which question? | Process, impact or value-for-money focus, and trigger for revisiting |
Filling this sheet is not a substitute for the assessment. Its value is in exposing where an answer is missing before the drafting has created an impression of completeness.
A fictional worked example
The following example is invented to illustrate the questions. It is not a legal or compliance finding, and it offers no monetary estimates.
Consider a fictional proposal to introduce a service standard requiring regulated providers to publish quarterly response-time data.
Problem and baseline. The problem statement concerns variation in response times and the absence of public information about them. The baseline question exposes an early difficulty: some providers already report response times to the oversight body on a different definition, and complaint volumes are recorded but not published. Any comparison of future published figures against past internal figures would compare two different measures.
Options. Three are set out for comparison: voluntary publication with a shared definition; a mandatory standard with publication; and a mandatory standard with publication phased by provider size. A fourth, non-regulatory option (a supported benchmarking exercise) is included because it tests whether the same transparency could be achieved without a new duty.
Affected groups. Small providers with manual case systems would carry a reporting burden that larger providers with existing systems would not. Service users would gain information, but only if the published measure is comparable and they can act on it. The distribution matters more than the total burden: a modest cost per provider is concentrated among those least able to absorb it.
Effect pathway and evidence. The pathway runs from publication to provider attention to shorter waits. Evidence held includes complaint records and one small pilot. The pilot's conditions (a single provider, a self-selected participant group) are recorded as limits on transfer. The comparability of definitions across providers is identified as the weakest link.
Uncertainty and review. Two unknowns are recorded rather than estimated: whether providers would change prioritisation in response to publication, and what the reporting cost would be for manual systems. The review is specified as a process evaluation after the first two reporting cycles, with a defined question about whether comparability held in practice.
The sheet's value in this example is that it produced questions and a review question, not benefit estimates. That is the appropriate output at scoping stage.
Using the answers
Three conditions shape how far this sheet should be taken. The first is proportionality: the depth of analysis should follow the significance and reversibility of the expected effects, and the reasoning for that judgement should be recorded rather than left implicit. The second is access: some questions, particularly about current provider burdens, can only be answered with information the analyst does not hold, which makes consultation a source of evidence rather than a later formality. The third is ethics and confidentiality: where evidence comes from identifiable organisations or individuals, its use in a published assessment needs to be agreed in advance.
Where an answer cannot be obtained, the useful move is to record the gap, state what it affects, and identify how the review would close it. An assessment with named gaps is more defensible than one with an unbroken surface.
Researchers deciding whether this kind of scoping work fits their expertise may wish to review how project-based policy research is framed before accepting an engagement, as set out in the guide on what to check before accepting an expert project.