Senior Staff Engineer - AI ADC

About the role

A global technology client of Insight Global is seeking a Senior Staff Engineer - AI ADC to join their team. This company specializes in high-performance application delivery, network security, and AI-ready infrastructure solutions for enterprises, service providers, and government organizations.

This role combines deep domain expertise in Application Delivery Controllers (ADC), Server Load Balancing (SLB), DNS/GSLB, with practical experience in AI/LLM systems, to build intelligent routing, inference‑aware load balancing, and Layer‑8 contextual decision engines for AI workloads. The ideal candidate is a senior staff engineer who can drive architecture, design, prototyping, and execution—working closely with product management, platform teams, and AI strategy groups.

Key Responsibilities Include:

Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI application traffic.

Design and implement LLM‑aware load balancing, incorporating semantic, token, latency, and model‑level insights into traffic decisions.

Develop enhancements for ADC and GSLB platforms to support AI workloads, including token‑aware rate limiting, inference latency‑based routing, AI model endpoint discovery and health checks, and integration with vector databases, model registries, and ML observability systems

Contribute to advanced features including Layer‑8 context‑aware routing, adaptive traffic shaping, and AI‑driven anomaly detection.

Drive system‑wide architecture: data plane, control plane, configuration, and distributed state management.

This role is permanent and will be worked hybrid on-site in San Jose, CA. We are looking to pay between $200,000-$220,000 base salary. Apply today!

Advanced degree in Computer Science, Networking, or related field (MS/PhD preferred)

Deep knowledge of L4–L7 protocols (TCP/TLS/HTTP/2/3, QUIC), load balancing algorithms and GSLB strategies (DNS‑based, HTTP redirect, anycast)

Strong programming skills in C/C++ (data plane), Go/Python (control/ML), and performance profiling (perf, eBPF, flamegraphs, VTune/nvprof)

Hands‑on experience with AI/ML systems, including: model inference pipelines, LLM API integrations; token‑level behavior and performance characteristics, and understanding of AI workload routing challenges (latency, caching, batching, multi‑model orchestration)

Exact compensation may vary based on several factors, including skills, experience, and education.

Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.

We are an Equal Opportunity Employer and are committed to fostering a diverse and inclusive workplace. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, age, or veteran status. We believe that diversity drives innovation and strengthens our ability to serve our clients and communities.