Founding AI Engineer
AimhireAbout the role
Founding AI Engineer
đź“Ť San Francisco, CA | đź•’ Full-Time, On-Site |
About the Company
Our client is a Y Combinator–backed startup transforming how teams test and monitor AI voice agents. Founded by IIT Bombay alumni with deep research and trading expertise, they’re solving the pain of manual voice testing by building automation tools that simulate thousands of real-world conversations.
Their platform uses dynamic personas, AI-generated datasets, and detailed workflows to uncover edge cases, identify failure points, and ensure that conversational agents perform reliably — before going live. Their mission is simple: help teams ship voice agents that work every time.
About the Role
As a Founding AI Engineer, you’ll work closely with the founders to design and build the core AI infrastructure behind their voice agent testing and monitoring platform. You’ll drive experimentation, build scalable systems, and collaborate directly with customers to incorporate feedback and improve performance.
This is a hands-on, high-impact role where you'll help define both product and technology direction from day one.
Key Responsibilities
Build AI tools for testing, benchmarking, and monitoring conversational agents
Develop scalable pipelines and evaluation methods for LLMs
Implement systems for real-time monitoring and feedback
Optimize deployments for high availability and performance
Engage with customers to understand needs and shape product direction
Continuously experiment and improve AI reliability and accuracy
What We’re Looking For
1+ years of experience as an AI or ML Engineer
Strong proficiency in Python
Experience working with LLMs or conversational AI systems
Familiarity with evaluation pipelines and production deployment
Customer-focused mindset and startup experience a plus
Responsibilities
- Build AI tools for testing, benchmarking, and monitoring conversational agents
- Develop scalable pipelines and evaluation methods for LLMs
- Implement systems for real-time monitoring and feedback
- Optimize deployments for high availability and performance
- Engage with customers to understand needs and shape product direction
- Continuously experiment and improve AI reliability and accuracy
Qualifications
- 1+ years of experience as an AI or ML Engineer
- Strong proficiency in Python
- Experience working with LLMs or conversational AI systems
- Familiarity with evaluation pipelines and production deployment
- Customer-focused mindset and startup experience a plus
Skills mentioned
About Aimhire
We connect great candidates with great companies, but we’re not just a hiring marketplace. We’re a team of recruiting veterans, and we’ve developed an in-house proprietary CultureMatrix to make sure every candidate fits every role.