← Back to Projects

Phonx AI – Real-Time Voice AI Platform

Problem

Insurance agencies were overwhelmed with manual calls to answer complex benefit questions. Existing systems offered limited automation and could not maintain regulatory compliance in long-form conversations, leading to bottlenecks and inconsistent customer experiences.

Context / Business Need

Phonx AI set out to build a voice automation system that would reduce human call load, ensure compliance with insurance regulations and deliver responses in under one second. The platform needed to handle long-form interactions, preserve context across multiple turns and integrate seamlessly with CRM and telephony infrastructure.

Constraints

  • Regulatory constraints requiring compliance and auditability of all conversations.
  • Sub-second response times for real-time voice interactions.
  • Accurate long-form conversation handling across multiple call turns.
  • Integration with legacy telephony and CRM systems.
  • Strict data privacy and security requirements in the insurance domain.

My Role

As AI Product Engineer and Head of Development, I led requirements elicitation, designed the system architecture and oversaw the development team. I ensured regulatory compliance by embedding a domain-specific knowledge layer and collaborated closely with legal and insurance experts.

System-Thinking Approach

We mapped the user journey from the moment a policyholder called to final resolution. Each component — speech-to-text, conversation manager, knowledge retrieval, compliance layer, CRM connector and telephony gateway — was designed to support one another, enabling fluid, compliant conversations.

MVP Design

The MVP focused on core elements: high-accuracy STT/TTS pipelines, a proprietary knowledge engine for insurance content, a compliance module, CRM integration and conversation memory to track context. We avoided feature creep to ensure speed and reliability.

Architecture Breakdown

The system comprised:

  • A telephony gateway handling SIP calls and routing.
  • STT and TTS microservices converting audio to text and back.
  • A conversation manager orchestrating LLM reasoning and maintaining state across long calls.
  • A compliance module enforcing insurance regulations and logging decisions.
  • A CRM connector syncing policy data and recording interactions.
  • A proprietary knowledge retrieval engine providing accurate answers in real time.
  • A microservice event bus enabling sub-second response times.

Final Solution & Results

The final platform achieved average response times under one second and 90%+ intent accuracy. It reduced human call volume by over 60%, provided consistent, compliant answers and allowed insurance agencies to scale support without growing staff.

Tech Stack

  • Python, Go and Node.js microservices
  • LLM orchestration with custom insurance knowledge engine
  • Docker & Kubernetes for scalability
  • gRPC, event buses and real-time telephony APIs
  • Twilio, proprietary STT/TTS engines