Separate genuine process change from marketing language
Almost every production company now mentions AI somewhere in its material, which makes the term close to meaningless as a differentiator. The useful question is where machine assistance actually sits in their process and what a human is accountable for at each point.
Ask a supplier to walk through a recent piece of content they delivered and identify exactly where assistance was used. You are looking for specifics such as transcription and caption drafting, first-draft script expansion from an outline, tag and keyword suggestion, image or illustration generation, translation draft, or automated test generation against a learning management system. Vague claims such as used AI to improve quality are not an answer.
A second and more revealing test is what the supplier refuses to delegate. A credible partner will have a clear position on what must remain human, for example anything involving legal or regulatory accuracy, anything that shapes the learning objective, and anything that a learner will interpret as authoritative.
Understand what the technology changes about cost and speed
The realistic benefit of AI-assisted production is most visible in first-draft and transcription workloads. Turning recorded video into a captioned transcript, or expanding a bullet outline into a readable script draft, can compress a stage that used to dominate a schedule. It does not remove the review load, because a draft still needs to be verified before it is publishable.
The other place the effect shows is at scale. Machine-assisted translation drafting, subtitle generation, and content summarisation become valuable when you are producing or updating large catalogues rather than a handful of flagship courses. If you are buying three courses, the technology is unlikely to change the commercial outcome much.
Ask suppliers to be specific about what they have actually measured. Has a stage been reduced from a number of days to a number of hours in practice, and has the total review burden changed or simply moved? A supplier who can talk about total elapsed time including human review is giving you a useful answer.
Test quality with a like-for-like sample
The only reliable way to evaluate AI-assisted production is to run a sample and compare it against a human-led approach on the same input. Give two shortlisted suppliers the same outline and ask each to deliver the same small module, one with machine assistance and one with a purely human workflow, at a comparable cost.
Then assess the things that matter. Check factual accuracy against your source material. Check whether the tone reads as one coherent author rather than a patchwork.
Check the learning design, because machine-assisted drafts often produce fluent prose that is cognitively dense and poorly sequenced, a common failure that reviewers mistake for efficiency. Check accessibility output, particularly captions, transcripts, and alt text.
Check how the work handles the specific things your organisation cares about, such as your product terminology, your internal policy language, and your brand voice. Machine drafts often read acceptably and miss the house style consistently.
Govern data, rights, and disclosure
Establish the data position before anything else. Ask which models and services are used, whether your content, source files, or learner data are transmitted to any third-party service, whether that data is used for training, and how it is deleted. Data residency and protection commitments should be written down, not just assured verbally.
Rights and provenance need attention too, particularly for generated imagery, generated voice or synthetic presenters, and for music. Ask who owns and licenses the output, whether your organisation can use it indefinitely, and whether the supplier indemnifies you against third-party claims. Synthetic presenters also raise a consent and disclosure question: if a viewer cannot tell a person is not real, that is a material transparency issue, and you should decide your position before it becomes a controversy.
Also establish the disclosure position for learners and for your own reviewers. Some organisations require that AI-assisted content is identified internally for review purposes. That is a reasonable practice, and it changes how the supplier needs to work.
Build review and provenance into the workflow
The best contracts and workflows treat AI assistance as a drafting aid inside a human-controlled process. That means a named human signs off on every published asset, source material is retained for all published content, and there is a way to trace any published claim back to its approved source. This is what makes a course defensible in an audit.
Ask how the supplier handles hallucinations in generated material, particularly fabricated citations, invented statistics, and plausible-sounding regulatory detail. In compliance content, a confidently wrong number is far more damaging than an obvious gap.
Finally, decide how you will monitor quality over time. If a content catalogue is maintained with machine assistance, establish a periodic audit sample, and a refresh trigger when the underlying tooling changes. Our guide on how to assess content quality before full production describes a front-loaded review model, and our guide on evaluating instructional design quality covers the design-specific failure modes that machine drafts tend to produce.
Commonly overlooked pitfalls
- Accepting AI capability claims without testing output quality on your own subject matter and terminology.
- Assuming machine drafts remove review effort, when they often relocate the burden to accuracy checking rather than eliminating it.
- Leaving data handling, retention, and model training terms on the provider's standard paper without checking them.
- Overlooking provenance and disclosure requirements for generated imagery, synthetic presenters, or generated text.
- Not building a periodic re-audit into a catalogue that is maintained with machine assistance.
Next steps
- Require each shortlisted supplier to state exactly where assistance is used and where humans remain accountable.
- Run a like-for-like sample with and without machine assistance and score both against the same criteria.
- Get data handling, retention, and third-party model terms in writing before any content is uploaded.
- Set an explicit policy on synthetic presenters, generated imagery, and disclosure, and require it in the contract.
- Require traceability from every published claim back to an approved source, with human sign-off on each asset.
Related guides
Put this into practice
The vendor evaluation checklist and the RFP template turn a guide into a process you can run this week.