Test It Before You Trust It: A 20-Minute Checklist for Any AI Agent

 

Here’s what I teach my team before anyone uses an AI agent on real client work: test it first, on something where you already know the answer.

You wouldn’t hand a new hire your biggest client on day one, and you shouldn’t hand one to an AI agent either. You check the work first.

This is the same 20-minute test my team runs on any AI agent, ours or anyone else’s, before trusting it with real work.

WANT ONE TO PRACTICE ON? Claim a FREE agent, then run this checklist on it this week.

The 20-minute test

Minutes 0 to 5: Set up a fair test

1. Pick a task you finished recently. Last month’s bank rec, a document chase you already sent, a close checklist you already ran. You know what the right answer looks like, so you can grade the agent.
2. Make a fictional twin of the client. Swap the name for a code (“Client 12”), round the numbers, keep the shape of the problem. No Social Security numbers, EINs, account numbers or health information, ever.
3. Plant three errors you already know about. This is the best trick on the list. In a bank rec test, slip in a duplicate deposit, an uncleared check and a bank fee nobody booked. Now you’re not wondering if the agent is good. You’re checking whether it found your three.

Minutes 5 to 10: Run it like a new hire

1. Give it only what a new hire would get. Fill in the facts the agent asks for and nothing more. A good agent asks for what it needs: the period, the threshold, what’s missing.
2. Watch what it asks you. If it never asks about something important (a deadline, a materiality threshold, the tone for a client email), that’s a gap you’ll need to cover yourself.
3. Time it. Write down how long the draft took versus how long the task took you last month. That number is the whole business case.

Minutes 10 to 20: Grade the work

1. Did it find your three planted errors? All three is a pass. Two is “useful with review.” One or none, and it’s not ready for that task.
2. Compare it to your finished work. Same totals? Same conclusions? Anything it added that you missed? (This happens more than you’d think.)
3. Try one edge case. A missing statement, a negative balance, a client who ignored the first two emails. Good agents handle the messy version, not just the tidy one.
4. Check what it assumed. Tax rates, legal rules and thresholds should be entered and checked by you, not quietly built in. If it “knew” a rate, verify it.
5. Ask: would I send this? Read the client-facing parts out loud. If you’d be comfortable sending it after a light edit, it passed.

Five red flags

• It invents numbers instead of telling you something is missing.
• It wants your bank or software login to “make it easier.” Our agents never connect to anything; you stay in control of the data.
• It hard-codes tax rates or legal rules with no way to see or change them.
• It never shows its work. You should be able to see why it flagged what it flagged.
• It sounds sure about everything. Real work has judgment calls. A good agent marks them for you.

Decide where the human always stays

Even after an agent passes, pick your checkpoints in advance so nobody has to decide in the moment. A simple rule that works for most practices:

• A human always reviews anything with a dollar amount, anything a client will read, and anything that gets filed.
• The agent can run unreviewed on internal prep: checklists, first drafts of notes, follow-up schedules and to-do lists.
• Re-test every few months, and any time the agent, the law or your process changes.

Write those checkpoints down once. Now your team can use the agent with confidence instead of guessing how much to trust it.

A quick example

Say you test the Document Collection Chase Agent. Your fictional client is missing three documents, the deadline is in ten days and the client ignored your first email. A passing result gives you a ready-to-send email that lists exactly what’s missing, a firmer follow-up, a schedule for the next two weeks and phone talking points. You check the dates, change one sentence to sound like you, and you’re done. That’s a pass, and now you know exactly what to review every time.

Your scorecard

1. Found all three planted errors?
2. Matched (or beat) your finished work?
3. Handled the messy edge case?
4. No hidden rates, rules or logins?
5. Would you send it after a light edit?

Five yeses: put it to work, with your checkpoints. Three or four: use it with closer review. Fewer: try a different agent for that task.

Start FREE

• Claim a FREE AI agent and run this test on it: https://aiaccounting.legacysbc.com/claim-agent
• Join our FREE community and tell us how your test went: https://www.facebook.com/groups/1024923770013348
• See every bundle and the agents inside: https://aiaccounting.legacysbc.com/pro-bundles
• Want a second pair of eyes? Grab 15 minutes with me. No pitch, just your questions.

Your turn: what’s the first task you’d test an AI agent on? Tell us in the FREE community. Every agent we build works with any LLM (ChatGPT, Claude, Gemini, Copilot), so you can test with the one you already use.

Here’s to agents that earn their spot on the team, one test at a time.

Yvonne