Senior Software Engineer
HaystackAbout the role
A world-leading AGI infrastructure company is accelerating model development for frontier AI labs through high-quality training data, evaluations, and engineering talent. They are seeking a highly skilled individual to contribute to cutting-edge AI projects.
The Role
Design, refine, and iterate on rubrics and evaluation criteria for preference data
Review and label data with a high bar for quality, catching subtle issues
Build and maintain pipelines and infrastructure supporting data generation, collection, and evaluation workflows
Synthesize findings from data work into clear write-ups, updates, or recommendations
Collaborate closely with researchers and engineers to translate qualitative judgment into scalable processes
What You'll Need
5+ years of proficiency in Python (especially building/maintaining pipelines and infrastructure)
Experience contributing to or navigating large TypeScript codebases (like VSCode is a plus)
Hands-on experience with preference data collection and/or rubric creation
Strong written and verbal communication, able to distill technical work into clear, actionable summaries
Demonstrated high-taste judgment, able to articulate why something is good
What's On Offer
Work in a fully remote environment
Opportunity to work on cutting-edge AI projects with leading LLM companies
Flexible contract engagement (minimum 20 hours/week, 4 hours overlap with PST)
Apply via Haystack today!
Responsibilities
- Design, refine, and iterate on rubrics and evaluation criteria for preference data
- Review and label data with a high bar for quality, catching subtle issues
- Build and maintain pipelines and infrastructure supporting data generation, collection, and evaluation workflows
- Synthesize findings from data work into clear write-ups, updates, or recommendations
- Collaborate closely with researchers and engineers to translate qualitative judgment into scalable processes
Qualifications
- 5+ years of proficiency in Python, especially building/maintaining pipelines and infrastructure
- Experience contributing to or navigating large TypeScript codebases
- Hands-on experience with preference data collection and/or rubric creation
- Strong written and verbal communication skills
- Demonstrated high-taste judgment
Benefits
- Work in a fully remote environment
- Opportunity to work on cutting-edge AI projects with leading LLM companies
- Flexible contract engagement
Skills mentioned
About Haystack
Haystack combines AI & expert vetting to deliver world-class tech candidates who are engaged, aligned, and ready to interview. We're trusted by over 400,000+ tech candidates, working in Software Engineering, Data, Design, DevOps, Cloud, Tech Management, Testing, Product & Delivery, Architecture and more. 100s of employers from startups and scale-ups like Atom Bank, DuckDuckGo and Goodlord to established enterprises like American Express, Dunelm and AWS use Haystack to connect with qualified tech talent that they can't find anywhere else.