AI Assessment vs AI Pilot: Which Should Come First?
If you're deciding whether to diagnose the whole portfolio of candidate AI workflows or pick one and pilot it now, the answer depends on what you already know about your own situation, not on a universal rule. This page lays out the decision situation by situation — including the cases where a narrow pilot is exactly right and a full assessment would only slow you down.
Run an AI assessment first when you have more candidate workflows than certainty about which one to fund, no measured baseline, or no single sponsor committed to one project. Run a pilot first when you already have one well-scoped workflow, a named sponsor, and confirmed data access, and the open question is whether that specific approach works.
On this page
At a glance
- There's no universal order: an AI assessment ranks candidates across the operation, and an AI pilot tests one workflow already chosen — the decision matrix below sorts common situations into which comes first.
- An assessment wins when you have more candidates than certainty, no baseline, or no single committed sponsor.
- A pilot wins when the workflow, sponsor and data access are already settled and the open question is whether the approach works.
- Every row in the matrix is either sourced or marked as a decision factor you answer for your own organization.
The decision in one table
Should you run an AI assessment or a pilot first? It depends on what you already know, not on which one a vendor happens to sell. An assessment, such as the 360° AI Blueprint, diagnoses the whole operation and ranks candidates before anything gets built. A pilot tests one already-chosen workflow instead.
| Situation | An AI assessment | An AI pilot | Which comes first |
|---|---|---|---|
| You have more candidate AI workflows than certainty about which one deserves budget. | Ranks the candidates against each other. The Blueprint's stated deliverable is "AI and automation candidates ranked by ROI, cost, and time-to-value"[1]. | Tests one candidate's mechanics in depth, but a pilot is built to "work in a controlled, manageable environment"[2] around a single workflow — it says nothing about the ones you didn't pick. | Assessment. Rank the portfolio before you commit pilot budget to one guess. |
| You aren't sure yet whether the pain point is one workflow or a symptom of a bigger process. | As an analogy only: the GOV.UK Service Manual, written for public-sector digital services (not AI assessments or pilots), says teams should understand the problem before building anything: "you need to understand the problem that needs to be solved"[3], since "you'll often find the thing you're working on is part of a bigger process or user journey"[3]. | In that same manual's own sequence, the narrower prototyping (alpha) phase follows discovery rather than replacing it: "Alpha is where you try out different solutions to the problems you learnt about during discovery"[4], often focusing on "the areas you think will be most challenging"[4] rather than the whole journey. | Decision factor: if you don't yet know whether the pain point is one workflow or a symptom of a bigger process, diagnose that first — as an analogy, GOV.UK's own model runs discovery before its narrower alpha phase for the same reason. |
| Decision factor: do you already have one named sponsor committed to a single workflow, or several stakeholders each pushing a different idea? | Useful when nobody's agreed yet — a ranked portfolio gives multiple stakeholders one shared list to argue about, instead of arguing about which idea to try first. | Useful once the sponsor is already named and the workflow already chosen — there's nothing left to rank. | Decision factor: answer this for your own organization. Neither instrument supplies a missing sponsor. |
| Decision factor: is data access for your one candidate workflow already confirmed, or unknown across most of the portfolio? | An assessment's job is to surface where data is, and isn't, accessible across every function, before you commit build time anywhere. | Only tells you about the one workflow's data. A clean pilot run on good data says nothing about the other candidates. | Decision factor: known access on one workflow points to a pilot; unknown access across most of the portfolio points to an assessment. |
| Decision factor: is a bounded pilot already funded for this quarter, or are you setting next year's AI investment plan? | Fits a planning cycle — it's the input to deciding what to fund next, not something you run alongside a pilot that's already approved. | Fits money already approved for one narrow test. | Decision factor: match the instrument to the budget you actually have. |
| You want organization-level oversight of how AI risk is governed, not sign-off on one project. | As an analogy only: NIST's AI Risk Management Framework — itself voluntary guidance, not a mandatory standard[5] — separates a cross-cutting Govern function — "Govern is a cross-cutting function that is infused throughout AI risk management and enables the other functions of the process"[5] — from a system-level Map function (next column). NIST doesn't address assessments, portfolios, or pilots, and doesn't recommend this sequencing. | NIST's Map function works at the level of one system: "The Map function establishes the context to frame risks related to an AI system"[5] — one system, not a portfolio. | Decision factor: treat NIST's Govern/Map split only as an analogy for organization-level vs. system-level oversight, not as NIST recommending an assessment-before-pilot sequence. Match it to your own governance structure. |
Every row above is either backed by a citation or marked "Decision factor" for you to answer against your own organization — never an unsourced rule of thumb.
When an AI assessment should come first
An assessment's job is to compare candidates against each other, something one pilot can't do. The 360° AI Blueprint's own description states it produces "AI and automation candidates ranked by ROI, cost, and time-to-value"[1] — a ranked list across the operation, not a verdict on a single workflow.
- You have several plausible AI candidates and no ranked view of which one is worth the most, or the least risky to start with.
- Nobody has measured where hours, errors, or dollars actually concentrate across departments — surfacing that is what an assessment is designed to do, not a pilot.
- Different stakeholders are each championing a different project, and you need one shared, defensible list to choose from.
- You're planning a budget cycle or board conversation about AI as a program, not approving one team's test.
When a pilot should come first
A pilot should come first when the workflow is already agreed, a sponsor is already named, and the open question is whether that approach works in practice — not which of several candidates deserves attention.
- One workflow is already the obvious candidate, and nobody is seriously proposing an alternative.
- A sponsor and a bounded budget already exist for a single test, not a portfolio-wide review.
- Data access for that one workflow is already confirmed, so there's no discovery work left to do before you start.
- The organization has already ranked its candidates (through an earlier assessment or its own analysis) and this workflow is the one it picked.
Once the workflow is already scoped, narrowing further is reasonable too. As an analogy, GOV.UK's own alpha phase — which by design follows its discovery phase, not a substitute for it — still narrows further: "often it makes sense just to focus on the areas you think will be most challenging"[4] rather than the whole journey. A pilot's narrow, controlled design[2] is a feature at this stage, not a shortcut around the scope decision it comes after.
What each costs you in time and attention
This section covers time and attention, not money: this page states no fees or durations for either instrument. What differs is whose time each instrument needs, and for how long.
- Executive time: an assessment needs a sponsor's attention across every function in scope, over the diagnosis; a pilot needs the same sponsor's attention, but only for the one workflow being tested.
- Data-access approvals: an assessment needs read access across the systems behind every candidate workflow; a pilot needs access to the single system it touches.
- Stakeholder interviews: an assessment interviews people across departments to find where hours and errors concentrate; a pilot needs input mainly from the few people who run the one workflow.
- Decision risk while waiting: an assessment takes time before anything gets built at all; a pilot risks the opposite problem, covered next, where it runs indefinitely without anyone deciding what its result means.
Avoiding pilot purgatory
Pilot purgatory is what happens when a pilot's result never triggers a decision: it isn't killed, isn't funded to scale, and just keeps running as a permanent "test" nobody is accountable for closing out.
In Foresight's view, this follows from what a pilot is built to be, not from a claim IBM's research makes about why pilots stall: pilots are designed to "work in a controlled, manageable environment"[2] around one workflow, and, separately, "scaling AI across an enterprise involves complexities that can't be captured in a single trial run"[2]. A pilot scoped without an end condition has no natural reason to end — that reasoning is ours, not IBM's.
- No decision was attached to the pilot's result before it started, so no result — good or bad — has anywhere to go.
- The pilot quietly expanded past its original workflow boundary instead of staying scoped to the question it was meant to answer.
- Nobody owns the go/kill/redesign call, so a mediocre result just keeps running rather than getting decided.
- If you've already chosen to pilot, how to scope an AI automation pilot covers the worksheet for setting a decision, a boundary, and exit criteria before you start — the detail this page doesn't repeat.
Frequently asked questions
Should you do an AI assessment or a pilot first?
It depends on what you already know. Do an assessment first if you have several candidate workflows and no ranked view of which deserves budget, no measured baseline, or no single committed sponsor. Do a pilot first if the workflow, sponsor, and data access are already settled and the open question is whether that specific approach works.
When is a pilot the better first step?
When one workflow is already the clear candidate, a sponsor and bounded budget already exist, and data access for that workflow is already confirmed. In that situation, an assessment would mostly re-diagnose a decision your organization already made, rather than adding new information.
What does an AI assessment deliver that a pilot doesn't?
A ranked comparison across every candidate workflow — the 360° AI Blueprint's stated deliverable is candidates "ranked by ROI, cost, and time-to-value." A pilot only tests the mechanics of the one workflow it runs; it can't tell you how that workflow compares to others you didn't test.
What is pilot purgatory?
Pilot purgatory is a pilot whose result never triggers a decision: not killed, not scaled, just running indefinitely as a permanent test. In Foresight's view, it follows from pilots being scoped to one controlled workflow with no built-in end condition, so nobody is ever forced to call it done.
Can you run an assessment and a pilot at the same time?
Yes, if they're scoped to different questions: an assessment ranking the rest of the portfolio while a pilot tests the one workflow you've already committed to. Running both on the same undecided workflow just duplicates the ranking work the assessment already does.
Related reading
- The 360° AI Blueprint is the assessment referenced throughout this page: how it ranks candidates across the operation.
- What is a 360° AI Blueprint? covers what the assessment contains and how the engagement runs, in more depth than this comparison does.
- Before either instrument, AI readiness assessment: 12 questions is a self-check for the sponsorship and data-access gaps this page treats as decision factors.
- Once you've chosen to pilot, how to scope an AI automation pilot is the worksheet for its boundary, baseline, and exit criteria.
Sources
- Foresight AI Partners, "The 360° AI Blueprint" (homepage) (accessed 2026-09-29)
- IBM, "AI pilots are fun. AI at scale is a business." (accessed 2026-09-29)
- Government Digital Service, "How the discovery phase works" — GOV.UK Service Manual (accessed 2026-09-29)
- Government Digital Service, "How the alpha phase works" — GOV.UK Service Manual (accessed 2026-09-29)
- National Institute of Standards and Technology, AI RMF Core, Section 5 — AI Risk Management Framework 1.0 (2023) (accessed 2026-09-29)