Case study · AI, data & growth
Piloting AI With Honest Judgement
I helped mature two AI products while identifying where automation added real value—and where it did not.
The context
Several emerging AI tools promised to automate work ranging from consumer calls to manager coaching.
The challenge
I needed to determine where AI could genuinely add value without overpromising its capabilities or forcing it into unsuitable use cases.
How I approached it
As a pilot customer of a calling software vendor, I worked closely with its team to mature an AI voice agent. We first tested it for pre-qualifying consumer leads and later explored its use for outbound sales calls to prospective partners.
I also led the internal rollout of an AI manager coach as a pilot customer, giving managers continuous feedback on their one-to-one meetings. Alongside these pilots, I supported the adoption of generative AI and AI-powered advertising tools across the business.
Throughout the experiments, I focused on narrow pilots, comparison with the human baseline, and honest assessment of whether each tool matched the problem it was intended to solve.
What it delivered
Voice-agent pre-qualification did not progress beyond the pilot because the fit was not strong enough. The agent proved better suited to cold outbound calling, which moved into a follow-up pilot in early 2026. The AI manager coach helped managers receive continuous feedback on their one-to-one meetings. The experience reinforced a practical approach to AI adoption: pilot narrowly, measure against human performance, and move the technology to the use case where it actually works.
Looking back
I learned that AI adoption depends less on the promise of a tool than on its fit with a specific problem. I appreciated how openly colleagues and vendor teams tested that fit. I would now begin with an even narrower pilot and compare performance with the human baseline from the outset.
The voice agent showed me that stopping one use case can reveal a better one, while the manager coach demonstrated value through continuous feedback. The principle I carry forward is simple: judge AI honestly, measure what it improves, and move it toward the work where it genuinely performs well.