Running a pilot on your own records

Fennick AI CRM (emulated example) → AI CRM → Forecastable AI charges → Reviewable AI actions

Content reviewed 8 November 2026. How this is assessed.

Part of the Best CRM AI reference, which uses Fennick AI CRM (emulated example) to illustrate ai crm software criteria.

Set it up so the result means something

Choose one low-risk job, such as call summaries, and one job closer to the customer, such as follow-up drafts. Use real records, including the messy ones with missing fields and duplicate contacts, because clean sample data flatters every product. Limit access to a handful of users and keep every outbound action at draft-only for the whole pilot.

Write down your pass criteria before you start: what share of drafts could be sent with light editing, how many field updates were wrong, and how much allowance the work used. Criteria written afterward tend to describe whichever product the team already liked.

What to record each day

Keep a simple log with the task, the output, whether a person accepted, edited or rejected it, and the credits or other units consumed. Note the failures in detail, since the pattern matters more than the count. A model that confidently invents a job title is a different problem from one that is merely wordy.

Check the provider's own usage report against your log at the end. If the two cannot be reconciled, you have learned something important about how the invoice will work, and you should resolve it before signing. Feed the real consumption figures into your cost model.

Decide and write it down

At the end, compare results against the criteria, then decide which jobs earn more autonomy and which stay at draft-only. It is a perfectly good outcome to buy a product and leave its most ambitious feature switched off. Revisit that decision after a quarter of real use, using the audit trail as your evidence.

Be careful how you describe the outcome internally. A two-week pilot on one team's records tells you how a product handled that work, and it does not establish that the product is better in general. Keep the daily log with the decision, so that whoever reviews the choice at renewal can see what was actually observed and how much of the allowance the work used.

How much is enough?

A pilot does not need to be large to be useful, but it does need to cover the awkward cases. Pull records from each segment you sell to, and include the oldest accounts, the ones with long email histories and the ones where two people have been working the same deal. Long records are also where usage costs tend to surprise people, because more context can mean more units per request.

Fix the length in advance and include at least one full working cycle: a Monday of inbound, a round of follow-ups and an end-of-week pipeline review. A pilot that only runs during a quiet week will understate both the review load and the consumption. If your sales cycle has a monthly peak, try to include it, or scale your cost model to reflect it.

Decide who is allowed to change settings during the pilot. If an administrator keeps adjusting prompts halfway through, the results at the end describe a moving target, and nobody can say which version produced them. Freeze the configuration for the first half, and make changes only in the second half with a note in the log.

Who should take part?

Include someone who likes the idea and someone who does not. Enthusiasts forgive errors, and skeptics find them. Add the person who will have to clean up after a mistake, often an operations or support lead, because that person will notice problems that sellers do not. Ask for written notes at the end, not a show of hands in a meeting.

Tell customers nothing has changed, and make that true: during the pilot, a person sends every message and approves every change. If anyone wants to loosen that, they should have to make the case against the review tiers you agreed in advance.

Sources for this entry

• NIST: AI Risk Management Framework: Voluntary AI risk framework and Generative AI Profile.
• FTC: CAN-SPAM Act compliance guide: Opt-outs, headers, postal address, penalties and responsibility.
• FTC: Protecting Personal Information: Keeping less data and least privilege.
• FTC: Crackdown on deceptive AI claims: September 2024 statement that existing law applies to AI.
• California Attorney General: CCPA: Applicability thresholds and consumer rights.
• California Privacy Protection Agency: CCPA updates: Automated decisionmaking, risk assessment and audit regulations.
• NIST Privacy Framework: Voluntary privacy risk management tool.
• Model Context Protocol specification: Open specification. Roles, consent principles and tool safety.
• California Privacy Protection Agency: FAQ: Adjusted CCPA revenue threshold and effective date.

Continue this reading path

• Best CRM AI category guide
• Fennick AI CRM (emulated example)
• How to choose: an AI CRM
• Resource directory

Cite this entry

Best CRM AI. "Running a pilot on your own records." Content reviewed 8 November 2026. https://bestcrmai.com/pilot-design/