Point the AI the other way: a Red Team agent for the learning ground

AI now does the document reviews, reference checks and first drafts that used to build judgement in junior people. A Red Team agent points AI the other way: it argues against your work and never writes it. How it works, the two rules that make it stick, what the research says, and a starter you can use this week.

By Richard Wood · · 10 min read

A page of typed lines with one sentence circled in orange pen, an orange question mark in the margin and a black pen with an orange cap lying beside it

In 2025, Deloitte Australia agreed to partially refund the Australian government for a $440,000 AUD report that included a fabricated quote from a federal court judgment and references to research papers that don't exist (AP via CTV News).

A few weeks later, a $1.6 million CAD Deloitte health workforce plan for the government of Newfoundland and Labrador was found to contain at least four citations to research papers that do not exist (Canadian HR Reporter).

Every resource in the world to run the checks. And the checks didn't fire. Twice.

I told that story at GROWTH:SUMMIT:26 in Manchester last week, and I promised the room my read on why it happened. This is the longer version, plus the thing we built to stop it happening to us. It's called a Red Team agent, and it's the first move in what we call "Protect the learning ground" in From Hype to Habit.

The work that used to teach people

AI doesn't just change the work. It changes the people doing it.

The grunt work that used to train junior people, the document review, the reference checks, the first drafts, the things most of us cut our teeth on early in our careers, is now done by AI. That was the learning ground. And the instinct to spot a fake reference, a poorly scoped proposal or key client communications only gets built by working through hundreds of real ones. Rinse and repeat. Reps and sets.

It's worth saying I'm not against firms using AI to take that work off people's plates. Most of it was tedious, and nobody misses it. But the apprenticeship was hidden inside the tedium. You couldn't reach the next career stage unless you could do the work, and doing the work was how you learned to judge it.

The hiring numbers suggest the gap is already opening. Stanford economists using ADP payroll data found that employment of 22- to 25-year-olds in the most AI-exposed occupations now stands 19% below where it would otherwise be, in their August 2026 revision of the "Canaries in the Coal Mine" paper (Stanford Digital Economy Lab). In the UK, the Big Four cut graduate intakes in 2025, with the cuts attributed "in part to the integration of AI tools" (HRreview).

That looks great on this year's P&L. But those graduates were the next wave of managers and, eventually, partners. The bill for that gap arrives in about five years, and it arrives as a shortage of people who can check the work.

We learned this the hard way

In the talk, I confessed to our own version, using the analogy of learning to drive.

We built the road: AI skills for sales scoping and quoting. We put in traffic lights: every quote went through delivery validation before a client saw it. And everyone passed their test: we trained the whole team.

Then the delivery validations started coming back as AI responses. People pasted them into Claude and sent back whatever it said. Two machines agreeing with each other, and nobody in the middle asking what was right for the client.

Every light was green. And a quote went out to a prospect at more than half as much again as it should have been. We only caught it because one of our directors happened to be on the deal. I'll let you guess how that one ended.

It wasn't laziness. If something looks right enough, and speed and ease are in abundance, why would anyone question it?

Losing the deal wasn't the worst part. We run an agency; that happens. The worst part was what our own tools had done to the learning ground. The team weren't learning to scope. They were learning to paste... which, when I last checked, isn't a skill many businesses are hiring for.

Agreement is cheap. Judgement is built by disagreement

Most assistants are built to be helpful, and helpful drifts very easily into agreeable. In April 2025, OpenAI rolled back a GPT-4o update because it was "overly flattering or agreeable", which it described as sycophantic (OpenAI). That got fixed. The underlying pull, towards telling you your work is good, didn't go away.

Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers and found that "higher confidence in GenAI is associated with less critical thinking", while "higher self-confidence is associated with more critical thinking" (Lee et al., CHI 2025). It's a self-reported survey, so hold it loosely. But it describes our quote problem almost exactly: the more people trusted the tool, the less they checked it.

That's the pattern we kept seeing, not just for us, but in the agents our clients were using too. The ordinary assistant makes agreement cheap, and judgement is built by disagreement.

Point the AI the other way

Banning the tools doesn't work. People will use them anyway, just less visibly. And, being frank, why would we want to, the potential gains from AI use are there to see. So our answer was to point AI the other way.

Think of the teacher with the big red pen, marking your homework. A Red Team agent does the opposite of what most assistants do. It doesn't draft, summarise or agree. It takes your work, a memo, a proposal, a forecast, a quote, and argues against it. It finds the weakest claim. It asks for the evidence behind your most confident sentence. It names the assumption you haven't noticed. And it refuses to be satisfied by a reworded version of the same point.

You have to defend your work to a machine before you defend it to a person.

Before the summit, I gave my own talk to our Red Team agent and asked it to assess it. It came back with three lines.

"Your weakest claim is 'every story skipped a step'. You drew that map after the fact. What stops a sceptic calling it hindsight?"

"You open with four companies failing. Where's yours? Until you show one, you're the judge, not one of them."

"I won't rewrite it. Defend it."

The first one was fair, so I changed how I framed the stories: I wasn't claiming FLAIR would have saved anyone, just asking which question nobody asked. The second one stung, and it's the reason the quoting story made it into the talk at all. The third is the whole design. It wouldn't fix any of it for me. I had to defend it, and I learned more from that than I would have from a tidier draft.

Two rules that keep it honest

We've run this long enough to know where it goes wrong.

It challenges, and it never rewrites. The moment it offers a better version, it quietly becomes another drafting tool with a sceptical tone, and you're back to pasting. Its output is never the deliverable. The deliverable is the person's revised work.

It's the first stop for junior work, before a senior sees it, and the senior expects to see the argument. The exchange travels with the work. In the early-career case, what the line manager reads isn't just the memo. It's how the person handled the challenge.

Get those two right and nobody has to remember to think critically. The agent asks the questions a good manager would have asked ten years ago, when there was time. That's the check that didn't fire at Deloitte, built into the way the work gets done.

What the research says, and where it doesn't help yet

The closest evidence comes from a classroom, not an office. In a field experiment with high school maths students, Hamsa Bastani and colleagues gave some students an ordinary AI assistant and others a tutor version that gave hints rather than answers. Practice grades went up 48% with the ordinary version and 127% with the tutor. But when the AI was taken away for the exam, students who'd had the ordinary version did 17% worse than those who never had it. (Bastani et al., PNAS 2025; Knowledge at Wharton).

That supports the core design choice. An AI that does the work for you makes you look better until it's taken away. One that makes you do the thinking leaves you better off.

There's also early work on AI as a devil's advocate in group decisions. A small CHI 2025 study found AI counterarguments "fostered a more flexible atmosphere" and improved satisfaction for everyone in groups with senior and junior members, without a significant rise in workload (Lee et al., arXiv 2025). Encouraging, but it measured psychological safety and satisfaction, not whether people's judgement got better.

So to be frank about the gap: nobody has yet run a controlled study showing that a Red Team agent builds better judgement in junior professionals over a career. Our evidence is what we've seen in our own work, and the tutoring research pointing the same way. And there's an obvious way it fails. If someone pastes the agent's challenge into another assistant and pastes the answer back, you've rebuilt the quote problem with extra steps. That's why the second rule matters more than the first. A senior reading the exchange will spot a pasted defence long before a policy would.

A starter you can use this week

Obviously, you can go deep with a Red Team. The more aligned with your company knowledge and processes it is, the better contextual challenges you'll get. But you don't have to start there. You also don't need a platform and you don't need to build out a plugin or skill to try this. Set up a project or custom assistant in whatever tool your team already uses, we use Claude, and start with something like this:

You are a Red Team adversarial reviewer. The person sharing work with you wants it challenged, not improved. Do not agree for agrements sake.

1. Identify the three weakest claims and say why a sceptical reader would reject them.

2. Find the most confident sentence and ask what evidence supports it.

3. Name three assumptions the author hasn't stated.

4. Ask what the reader or client would push back on first.

Never rewrite, redraft or suggest replacement wording. If asked to, refuse and ask the author to defend the point instead. If the author's reply only rewords the same point, say so and ask again. Keep each challenge to two sentences. End by asking which challenge they'll address first. Continue the conversation until the sharer has confidently defended your challenges.

Then make three decisions that matter more than the wording.

Pick the work. Name the early-career output that used to build capability in your function, the first draft of a proposal, a forecast narrative, a scoping document, and make the agent the default first stop for it.

Make the exchange visible. The senior reviewer asks to see it, every time, until it's habit. If they don't ask, it stops happening within a month.

Give it context. A Red Team agent that knows what was promised to the client, or what the board actually reads, asks much sharper questions than one working from the document alone.

Where it sits in FLAIR

The Red Team agent is the first of three Protect moves in From Hype to Habit. The other two are unaided first drafts on high-stakes, decision-shaped work, so the human writes first and consults AI second, and deliberate rotation through automated work, so people who no longer do a task still run it manually once in a while to keep the muscle memory alive. Underneath all three sits career structure: demonstrated standalone capability as a checkpoint for progression, so the incentive does the governing rather than a policy nobody reads.

None of it works layered on as new admin. It works when it's built into the tooling, the careers and what senior people visibly do.

I'm pro AI. But save the time and lose the judgement, and you've built a firm that can't check its own work.

The firms that skip this won't feel it this year. They'll feel it when they need someone who can spot a wrong quote or a fake reference, and nobody coming up has ever had to.

Take the FLAIR Scorecard

If you want to know whether your AI programme is quietly hollowing out the learning ground, start with the free FLAIR Scorecard. It takes a few minutes, needs no sign-up and scores the five FLAIR pillars out of 25, with your lowest-scoring pillar and a first move. Take it this month and you're in next month's numbers.

Take the free FLAIR Scorecard: five pillars, a few minutes, no sign-up

Rich

Sources

Questions and answers

What is a Red Team agent?
An AI assistant set up to argue against your work rather than produce it. It finds the weakest claim, asks for the evidence behind your most confident sentence and names the assumption you haven't noticed.
Why would you use AI to protect people from AI?
Because banning the tools doesn't work. Pointing AI at critique rather than drafting keeps the friction that builds judgement, at the speed people now expect.
What are the rules for a Red Team agent?
It challenges and never rewrites, or it quietly becomes another drafting tool with a sceptical tone. And it's the first stop for junior work before a senior sees it, with the senior expecting to see the argument.
Is there evidence this works?
Indirectly. A 2025 PNAS study found students who had an unrestricted AI assistant did 17% worse once it was removed, while a version that gave hints rather than answers largely avoided the harm. There's no controlled study of Red Team agents at work yet.
Who should use a Red Team agent first?
People early in their careers, on the work that used to build capability in your function: memos, proposals, forecasts, quotes and client deliverables.
Where does the Red Team agent sit in FLAIR?
It's the first move in Protect, part of the Protect, Redesign, Prove playbook in From Hype to Habit that runs alongside the five FLAIR phases.

Comments

No comments yet.

Leave a comment

Loading the comment form