Checking a Commissioned Research Deliverable
Check a research deliverable against the agreed question, evidence trail and limitations, without requiring a preferred conclusion for acceptance.
Accepting a commissioned research deliverable is a test against the commission, not a verdict on whether the findings suit the reader. Work is acceptable when it answers the questions that were set, carries evidence a reader can follow from claim to source, and meets the agreed format and conditions. What remains after that (a different reading of the same evidence, a question nobody commissioned) should be recorded as disagreement or as new scope, not pressed into the report as a correction. The checklist below is a working format for making that judgement and for separating defects the author must fix from positions the reader may contest but should not edit away.
Fix the acceptance baseline before reading the draft
Acceptance needs a reference point that predates the deliverable. In most commissions that is a written brief or contract, supplemented by any agreed note of clarification issued during the work. Between them they should record the decision the research was meant to inform, the questions in scope, the exclusions, the required outputs and any format conditions.
If the baseline is thin (an exchanged email and a verbal agreement, for instance), write down your understanding of the requirements before you open the draft, and send it to the author with the draft. Acceptance then becomes a check against a stated position rather than a negotiation about taste conducted after the fact.
Checking should be proportionate. HM Treasury's Aqua Book, which sets out guidance on producing quality analysis for UK government, treats assurance as something scoped to the risk and importance of the analysis rather than applied uniformly to everything. The practical implication for a commissioner is that effort belongs where the decision turns: a claim that carries the recommendation deserves a full trace; a contextual paragraph that informs but does not determine the answer can be sampled.
Whether the deliverable answers the commissioned question
Start with coverage rather than quality. Take each agreed question and find the place in the document where it is answered. An answer that has to be assembled by the reader from three sections is a presentation problem worth raising; a question with no answer at all is a material gap, and the appropriate response is a correction request or an agreed variation, not silent acceptance.
Then check that the answer is stated and bounded. A reader should be able to quote the finding together with the conditions under which it holds — the population, period, jurisdiction and confidence the evidence supports. Reports that answer a neighbouring question, or answer the right question at a level of generality that could not guide the intended decision, fail on this test even when the underlying work is sound.
Exclusions matter as much as inclusions. If the brief ruled a topic out and the draft covers it anyway, that is a scope excursion. It may be welcome, but it should be recognised as new scope and either agreed as a variation or removed; uncommissioned material that has not been checked to the same standard creates a liability inside an otherwise acceptable report.
Whether the claims are traceable
Traceability is narrower than truth. The question is whether a reader can move from a sentence to the evidence that supports it, and then judge for themselves whether the evidence bears the weight placed on it. Four failures recur:
- Uncited load-bearing claims: a statement in the summary or conclusions that no note or annex supports.
- Citations that do not support the sentence: the source is real, but it addresses a different population, a different period, a different jurisdiction, or a weaker relationship than the sentence asserts.
- Superseded sources: a revised dataset, a later evaluation or a withdrawn guidance document exists and is not used, with no explanation of why the earlier version was preferred.
- Stacked or second-hand citation: several references listed for one claim where none individually supports it, or a figure attributed to a primary source that the author appears to have taken from a secondary summary.
Method transparency belongs to the same test. The reader does not need a full protocol, but the document should say enough about search strategy, inclusion rules and the treatment of conflicting evidence that the selection of material can be understood. Where the evidence base is divided, a report that presents only the studies that agree has a defect even if every citation is accurate.
Correction, disagreement and new scope
The distinction that most often goes wrong in acceptance meetings is between an error and a conclusion the reader dislikes. Three categories, plus a fourth for clarity, keep the conversation disciplined.
| Category | What it looks like | Appropriate response |
|---|---|---|
| Correction | The document misstates a source, relies on a superseded version, contradicts its own evidence, overstates confidence, or is internally inconsistent | Ask for the specific passage to be fixed and for any finding that depended on it to be revisited |
| Clarification | Accurate but ambiguous, under-explained or missing a stated limitation | Ask for wording or an added caveat; no change to findings expected |
| Analytical disagreement | Evidence accurately reported, but the reader would weight it differently, draw another inference or prefer another option | Record as a note on the acceptance record; ask for the reasoning to be made explicit if it is not; do not instruct the author to reach a preferred finding |
| New scope | A question outside the brief, whether omitted or answered without instruction | Note as a gap; commission separately if the decision needs it, or agree a variation |
The discipline matters in both directions. A correction that weakens a finding the reader happened to doubt is not vindication of the doubt, and a correction that strengthens a finding is not a reason to leave it in place. The test is whether the document is faithful to its evidence, not whether its conclusion survives.
An acceptance checklist
The worksheet below is a suggested working format. It is not a validated instrument, an institutional requirement or an audit standard, and it should be adapted to the size and risk of the commission.
| Requirement | Where to look | Check | Severity if it fails | Disposition |
|---|---|---|---|---|
| Every agreed question answered | Summary and section headings | Each question has a stated answer and a supporting section | Material | Correct or agree variation |
| Exclusions respected | Scope statement, annexes | No out-of-scope topic presented as if commissioned | Significant | Clarify or treat as new scope |
| Answer explicit and bounded | Executive summary, conclusions | Finding can be quoted with its conditions attached | Material | Correct |
| Claims traceable | Footnotes, references, data annex | A sample of load-bearing claims can be followed sentence → source → figure | Material | Correct |
| Sources current | Reference list | No superseded edition or withdrawn dataset relied on without explanation | Significant | Correct |
| Source fits the sentence | Cited passages beside the claims | Strength, population, jurisdiction and date match | Material | Correct |
| Conflicting evidence addressed | Methods annex, discussion | Contrary findings acknowledged and weighed | Material | Clarify or discuss |
| Method described | Methods annex | Search, inclusion rules and limitations intelligible to a non-specialist | Significant | Clarify |
| Confidence language proportionate | Throughout | Hedging and certainty track the evidence | Significant | Correct |
| Internal consistency | Tables against text | Totals reconcile; figures agree across sections | Significant | Correct |
| Agreed format met | Whole document | Structure, length and outputs as specified | Minor | Correct |
| Ethics and data handling as agreed | Statement or annex | Consent, anonymisation or approvals recorded where required | Material | Escalate before acceptance |
Severity should be assigned by consequence, not by how easy the fix is. A missing reference in a footnote is minor; a missing reference supporting the central comparison is material. Disposition should be one of a small set of words used consistently (correct, clarify, discuss, new scope, accept with note), so that the author can see what is being asked and the record shows what was decided.
Worked example (fictional)
The following example is invented to show the method; it does not describe a real commission or a real client.
A fictional charitable trust commissions a comparative report on three delivery models for a neighbourhood advice service. The draft is well structured and answers the two agreed questions. The acceptance review logs four issues.
| Ref | Location | Issue | Category | Severity | Disposition |
|---|---|---|---|---|---|
| 1 | Table 2 and accompanying text | Referral volumes supported by an evaluation of the service as it operated before a 2023 redesign; a later evaluation exists and is not cited | Correction | Significant | Replace source, confirm whether the finding holds |
| 2 | Conclusions | Summary describes the evidence as consistent, while the body reports that two of five studies found no effect | Correction | Material | Fix summary language; check whether the recommendation still follows |
| 3 | Section 4 | Report weights two small qualitative studies above a larger administrative dataset and concludes that Model B is strongest | Analytical disagreement | — | Record as a note; ask for the weighting rationale to be stated explicitly; conclusion unchanged |
| 4 | Whole document | Trust wishes to know cost per participant; not in the brief and not in the report | New scope | — | Note as a gap; commission separately if the decision depends on it |
After correction 1, the finding on referral volumes stands, so no conclusion changes. Had the outdated evaluation been the only support for Model B, the correct outcome would have been a revised finding, not a search for a replacement source that preserved the original conclusion. Correction 2 is a straightforward over-claim in the summary. Issue 3 is the reader's own analytical position and belongs in the acceptance record as a note, with a request that the reasoning be made visible to other readers. Issue 4 would have been a defect only if the report had answered it without instruction and without sourcing; as it stands it is a gap, and the right response is a separate commission or an explicit line in the record that the decision was taken without that evidence.
What the checklist cannot do
It cannot certify the underlying data, substitute for independent peer review, or establish legal or regulatory compliance. Those are separate activities with their own standards and their own limits. It also cannot resolve a genuine disagreement about interpretation; it can only make the disagreement visible and keep it out of the correction log.
Two practical constraints apply. First, sampling is unavoidable on long or multi-source reports, so choose the sample deliberately (every claim that carries a recommendation, plus a random selection from the rest), and say in the record what was and was not checked. Second, acceptance should be recorded: the version checked, the date, the person accepting, and the issue log with dispositions. Where the commissioning team is small, a second reader on material issues reduces the risk that acceptance turns on one person's reading of a contested paragraph.
If the commission itself has not yet been fixed in writing, the guidance on what to settle before accepting an expert project covers the decisions this checklist assumes are already on the record; the checklist is best used against a brief that specifies the questions, the outputs and the acceptance conditions in advance.