Reviews
Part of How to review and score marketing agencies before you shortlist
What a marketing agencies review methodology should cover before you shortlist
What a marketing agencies review methodology should cover, what sits outside scope, and how to separate vendor claims from independent evidence before shortlisting.
What to take away
- A credible marketing agencies review methodology states its scope: which agencies were assessed, on what evidence and against which criteria.
- Vendor claims, such as self-reported results and award entries, are not independent evidence and should be labelled as claims.
- Editorial judgement belongs in a review only when the method says how it was applied, and by whom.
- No directory can claim hands-on testing of an agency's delivery unless the method describes that testing.
- Verify legal status and filings separately, because a review scores positioning and evidence, not solvency.
What a review methodology should cover
Scope: assessed and not assessed
A useful method names the pool of agencies, the period covered and the criteria applied. It should also say what sits outside scope: pitch performance, staff retention and commercial terms are usually excluded.
Assessed vs Not Assessed
Assessed
- Agency pool
- Named
- Period
- Stated
- Criteria
- Applied
- Pitch performance
- Excluded
- Staff retention
- Excluded
- Commercial terms
- Excluded
Not Assessed
- Agency pool
- Unknown
- Period
- Unknown
- Criteria
- Unknown
- Pitch performance
- Outside scope
- Staff retention
- Outside scope
- Commercial terms
- Outside scope
If a review does not state its exclusions, treat the gaps as unknown rather than neutral. Two reviews with different scopes are not directly comparable. Our explanation of how marketing agencies reviews and comparison methods work sets out how scope shapes what a score can tell you.
Evidence types: claims, documents and independent data
Separate three evidence types. Vendor claims come from the agency or its marketing. Documents include filed accounts, published case studies and award entries. Independent evidence comes from third parties.
The IPA Effectiveness Awards 2022 winners page shows how a documented judging process makes campaign case studies citable. A review can cite that recognition, but not as proof of an agency's current team or pricing.
For legal status, use primary records. Registration, filing history and registered addresses sit on the Companies House register, the authoritative source for an agency's legal identity.
How to read scores and judgements
What editorial judgement is, and is not
Editorial judgement is a reasoned opinion formed from the evidence in scope. It is not a test result. A review scoring strategy work is scoring published outputs, not observing a live engagement.
Is the score verified?
Does the method state who applied judgement and against which criteria?
Treat score as reasoned editorial judgement
Treat score as summary of claims, not verified finding
Where a method uses judgement, it should say who applied it and against which criteria. Where it does not, treat the score as a summary of claims rather than a verified finding.
How to check advertising claims in a review
Agencies often describe results in ways that need checking. UK advertising rules are largely self-regulatory, and the ASA advice for businesses explains what advertisers must be able to substantiate. When a review repeats a claim, ask whether the underlying evidence would meet that standard.
If a claim cannot be substantiated on request, mark it unverified in your notes.
A practical method for buyers
Build a one-page evidence log
For each shortlisted agency, record the claim, the evidence type and the source. Add a column for what would change your mind. For example, a team paying £2,500 a month for a retainer should expect evidence of comparable work, not an assertion.
Log legal status from primary records, then note what the review could not assess. If your shortlist spans several regions, our guide to marketing agencies best providers in England explains how regional differences affect what you can verify.
Set a decision rule before you score
Decide in advance which criteria are pass or fail. Typical fails include unverifiable legal status, claims that cannot be substantiated, and scope gaps the agency will not close.
Then score only what is in scope. A methodology that mixes verified documents with vendor claims without labelling them will mislead you.
Glossary
- Vendor claim
- a statement the agency makes about itself or its work.
- Independent evidence
- material produced by a third party, such as a regulator or registrar.
- Scope
- the agencies, period and criteria a review covers.
- Editorial judgement
- a reasoned opinion based on the evidence in scope.
- Unverified
- a claim you could not check against a primary source.
Common questions
Does a review methodology replace due diligence?
No. It structures comparison. You still need to verify legal status, contracting entity and references before you sign.
Can a review claim hands-on testing?
Only if the method describes testing that took place. Most directory reviews assess published evidence, so treat an undescribed testing claim as unsupported.
How do I compare two reviews with different scopes?
Map each review's criteria and exclusions against your own decision rule. Compare only the overlapping criteria and treat the rest as gaps.
Should I trust an award listing as proof of quality?
Treat it as one data point. A documented judging process makes an award listing citable, but it does not confirm current team, pricing or fit for your brief.
