When I visit a marketing team that runs AI every day, I ask who checks the output and what that person knows about the subject. The answer usually tells me more about the team’s AI maturity than any usage dashboard.
Anthropic’s education team has now studied this question. In September, it published a report on how people check Claude’s work, based on interviews with Claude Academy participants. The researchers said they had expected time pressure to top this list of barriers to checking Claude. But lack of subject matter expertise topped it instead.
The report summed it up: “It’s difficult to judge an answer when you’re not sure what good looks like.”
The sample for the report’s data is small and self-selected, and the authors say so. But I still think marketing teams should pay attention to the results of the survey.
Marketing copy has no test suite
The report gives a reason for the problem. AI lets people take on work outside their own field, and that is where they struggle to judge the result.
That describes much of B2B marketing today. A content manager writes about a pump’s efficiency class without an engineering degree. A campaign manager drafts for a market she has never sold into. An AI tool gives both of them fluent copy in minutes.
Software developers have an advantage here. When the task involved code, the report found, “tests were usually involved.” A broken function fails the test.
Marketing copy has no equivalent. A wrong claim about a product reads as smoothly as a right one. According to the report, writing tasks “leaned on domain knowledge and a careful read by a peer for fit.” In most marketing teams, that peer is the product manager, and the product manager is busy.
A wrong claim about a product reads as smoothly as a right one.
What people changed after a miss
The part of the report I found most useful is about mistakes. Participants described 18 cases in which they missed an error Claude had made. Half of those people said it changed how they work.
Their changes were structural. Some now make Claude quote its source text before it gives a recommendation. Some added approval gates at each step of a multi-step task. Some run several AI agents at once, with one producing the work and others checking parts of it.
The researchers note that these people didn’t “check their work harder” or become experts overnight. They redesigned the workflow.
I see the opposite reflex in many companies. After a weak AI pilot, the first plan is another training and another brand document. In my experience, those don’t reach the moment when someone looks at a draft and has to decide whether it is right.
The panel already exists
The report ends with five suggestions. The fourth is to build a panel of experts: Know who you’d call for a second read outside your own field, and reach out before you press publish.
In most B2B companies, that panel is already on the payroll. It’s the product manager, the application engineer, the sales lead who knows the market. The problem is their calendar. They can’t read every draft.
In most B2B companies, that panel is already on the payroll.
So we write their judgement down as an AI instruction package. These ‘skill sets’ are a senior practitioner’s way of doing a task, with its inputs, its steps and its quality checks, running inside the AI the team anyhow uses. It’s the same move the report’s participants made after a miss: They improved the workflow.
The expert still gets the call when a claim matters. But the first expert check now happens within every drafting procedure.
Back to my opening question. Who checks your team’s AI output, and what do they know about the subject? If the answer is “whoever has time,” start there: Pick one content type, name the expert who should read it, and write down what they would check.