Staff Applied AI Engineer, Agentic Systems

About the role

About Beacon AI

We’re a fast-moving team of aviators, engineers, and operators building an AI platform to make flying safer, more efficient, and more capable. Backed by top investors, we’ve secured a dozen Department of Defense contracts and partnered with major airlines to deliver mission-critical systems. We operate without silos or heavy processes. Small, focused teams own what they build, ship quickly, and learn fast, pushing the boundaries of how humans and AI work together in aviation.

What you'll do

At Beacon AI, we're building AI-powered systems that help aviation professionals navigate complex, high-stakes operational decisions.

This role sits at the intersection of agentic AI, information retrieval, and production software engineering. We build systems that reason over diverse sources of information, use tools to investigate problems, and produce reliable, explainable results.

We're looking for engineers who can think critically about how these systems should work, design their underlying information and orchestration architectures, and take them from experimental prototypes to dependable production services.

Design and build agentic systems

Design and implement agent architectures involving tool use, planning, orchestration, state management, and memory.

Develop effective context engineering strategies, balancing relevant information, model capabilities, latency, and cost.

Make thoughtful architectural decisions about when to use autonomous agents, explicit workflows, conventional algorithms, or combinations thereof.

Design tool interfaces and structured model interactions that support reliable, inspectable behavior.

Turn complex information into usable knowledge

Develop an understanding of complex document collections and data sources: their structure, semantics, authority, relationships, and scope.

Translate domain knowledge into effective retrieval systems, structured representations, and agent-accessible tools.

Apply information retrieval, information extraction, and knowledge modeling techniques to heterogeneous structured and unstructured data.

Design strategies for handling temporal relevance, provenance, conflicting evidence, and incomplete information.

Evaluate and improve AI behavior

Develop rigorous evaluations for AI-powered systems, including representative datasets, human annotations, automated assessments, and targeted failure analysis.

Investigate system-level failures across retrieval, context construction, reasoning, tool execution, and final outputs.

Experiment with alternative architectures, models, and prompting strategies, using empirical results to guide decisions.

Help establish engineering practices for measuring and improving the reliability of agentic systems.

Build systems that work in production

Develop reliable services and asynchronous workflows with attention to concurrency, fault tolerance, observability, and maintainability.

Integrate model inference, retrieval systems, and external data services into production applications.

Build and operate containerized workloads and cloud infrastructure.

Collaborate with engineers and domain experts to translate complex operational requirements into working systems.

Areas of specialization

Applied AI is a broad discipline. We don't expect every candidate to have deep expertise in every part of the stack.

We particularly encourage candidates with substantial experience in one or more of the following areas:

Agentic systems and context engineering: Agent orchestration patterns, planning and execution, tool-use architectures, persistent and transient memory, and context management.

Information retrieval and knowledge representation: Semantic and structured retrieval, information extraction, document understanding, entity and relationship modeling, and knowledge-driven tool design.

AI evaluation and reliability: Evaluation methodology, benchmark and dataset construction, human and model-based assessment, statistical analysis, and systematic failure characterization.

Model adaptation and inference optimization: Fine-tuning, model selection, inference serving, quantization, batching, and performance optimization.

We value candidates who bring genuine depth to at least one of these areas, along with the curiosity and engineering judgment to work effectively across disciplinary boundaries.

What will make you successful

Strong software engineering fundamentals. You write maintainable production code, reason about system architecture, and understand testing, debugging, and operational reliability.

Substantive applied AI experience. You've worked on AI systems beyond basic model API integrations and can explain their architecture, tradeoffs, limitations, and observed behavior.

Information modeling instincts. You can understand a new domain, identify its important entities and relationships, and design software abstractions that reflect how its information is actually used.

Experimental discipline. You test assumptions, measure results, investigate failure modes, and use evidence to guide technical decisions.

Independent technical judgment. You can evaluate alternative approaches, recognize unnecessary complexity, and make appropriate tradeoffs.

Clear communication. You can discuss technical decisions with both technical and non-technical stakeholders and domain experts.

We care more about demonstrated technical understanding and the quality of your engineering work than familiarity with any particular AI framework.

Our technology environment

Our systems primarily use Python, LangGraph, BAML, AWS Bedrock, PostgreSQL/pgvector, and event-driven AWS infrastructure.

Experience with these technologies is helpful but not required. We're more interested in your understanding of the underlying concepts and ability to work effectively with unfamiliar tools.

Especially valuable experience

Aviation, aerospace, or another technically complex, safety-critical domain.

Computational linguistics, NLP, information science, or knowledge representation.

Designing systems that reason over technical documentation, regulations, or heterogeneous operational data.

Developing or operating distributed, event-driven, and containerized services.

Building AI systems in which traceability, evidence, and human oversight are important.

Examples of problems you might work on

Design an agentic workflow that identifies operational risks from aviation weather, airspace restrictions, aircraft-specific information, and regulatory guidance.

Develop retrieval and tool interfaces that allow agents to navigate multiple information sources while preserving source authority, provenance, and temporal relevance.

Build an evaluation framework that measures the factual completeness, correctness, and operational relevance of AI-generated analyses.

Investigate when an agent should perform additional research, use a specialized tool, or terminate its investigation.

Improve the reliability, latency, and cost of a multi-stage AI workflow through architectural changes informed by evaluation results.

What we're looking for in your application

We'd be particularly interested in hearing about an AI system you've designed or meaningfully improved, including the engineering decisions you made, the alternatives you considered, and how you determined whether the system was successful.

We value thoughtful technical work over the number of frameworks or models listed on a résumé.

Work Location

This is a hybrid role based in San Carlos, CA, with 3+ days per week onsite and the option to work remotely on remaining days.

Perks & Benefits (Full-Time Employees)

Healthcare: 100%* of employee medical premiums covered; 25% for dependents

Time Off: 3 weeks PTO plus 13+ paid company holidays

401(k): Offered (no current employer match, but we are committed to enhancing this benefit in the future)

Due to U.S. export control regulations, we can only hire U.S. Persons (U.S. citizens, Green Card holders, lawful permanent residents, or individuals granted asylum or refugee status). We are unable to provide visa sponsorship or support visa transfers. All work must be performed in the United States.

Beacon AI is an equal opportunity employer and does not discriminate based on race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, veteran status, or any other protected characteristic. We prohibit harassment or discrimination of any kind in the workplace and comply with all applicable federal, state, and local employment laws.