Cerebras
More roles at CerebrasEngineering Manager, Inference Router
Part remote, part office, you need to live within commuting distance of a named location.
Hybrid · Sunnyvale, CA
Employer listed it 5 weeks ago · Added 3 days ago
Been open since 5 weeks ago, still being checked, but it has been live a while.
Salary
Not stated
Location
Hybrid · Sunnyvale, CA
Timezone
Not stated
Contract
Full-time
Experience
Mid
Category
Software
The employer did not publish a salary, and we have too few comparable roles to estimate one.
What the advert says
Quoted word for word from the employer's advert. Where it says nothing, we say so.
- Working hours
- Not mentioned in the advert
- Where you can work from
- Location eligibility not confirmed
- Contract
- Not mentioned in the advert
- Experience asked for
8+ years of experience in large-scale software engineering, distributed systems, or infrastructure.
- Pay
- Not mentioned in the advert
Remote flexibility
Hybrid
This role is only partly remote, the employer expects time in the office around Sunnyvale, CA, Toronto, CAN, Hybrid, so you need to live within commuting distance.
What the employer says
- Source listing states candidate location: "Sunnyvale, CA, Toronto, CAN, Hybrid"
- Listing mentions "Hybrid"
What Nomaders makes of it
- Not suitable if you plan to move between countries
The quotes above are the employer's own words; the reading is ours. Always check the original listing and employment terms before working from another country.
Before you apply
Questions worth asking: Nomaders interpretation
- This role involves office time, but the advert does not say how often. Ask how many days a week are expected on site.
- No pay is stated. Ask for the salary range before investing time in interviews.
- The advert does not say whether this is employment or a contractor arrangement. Ask, as it affects tax and benefits.
Produced automatically from this advert's text; not reviewed by a person. The employer's own page is the final word.
About the role
Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.
About the Role
The Inference Router team at Cerebras builds the systems that decide where every inference request runs. We route traffic across all of Cerebras’ data centers and third-party integrations, balancing heterogeneous compute, maximizing cache utilization, and delivering the industry’s lowest time-to-first-token (TTFT) at massive scale. As an Engineering Manager, Inference Router , you will lead the team responsible for the intelligent control plane of the fastest generative AI inference platform in the world. Your team owns global request routing, prefill/decode (P/D) disaggregation, prefix-aware routing for high cache hit rates, autoscaling, and the high-availability infrastructure that keeps the platform fast and reliable under real-time, bursty load. This role operates at the intersection of distributed systems, Kubernetes-based infrastructure, routing-algorithm design, and machine learning. It combines technical leadership, people management, and execution ownership, with direct impact on the latency, throughput, and reliability experienced by every Cerebras inference customer.
What You’ll Do
Technical Leadership
Own the architecture and evolution of the inference routing layer across all Cerebras data centers and third-party inference integrations.
Guide the design of:
global, multi-region request routing and load balancing;
routing for heterogeneous compute, including prefill/decode (P/D) disaggregated serving;
prefix-aware / cache-aware routing to maximize KV-cache reuse and utilization;
low-TTFT, high-throughput request scheduling under concurrent, bursty workloads.
Drive the routing algorithms that trade off cache locality, queueing, load, and latency in real time.
Invest in simulation: build high-fidelity models of the fleet and its workloads so the team can validate routing policies, failover behavior, and scaling decisions under a wide range of conditions without tying up precious accelerator capacity.
Partner with inference runtime, cloud/platform, hardware, and ML teams to optimize end-to-end performance.
Team Leadership
Build, manage, and grow a team of distributed systems, infrastructure, and ML systems engineers.
Provide technical direction, mentorship, and career development.
Foster a culture of ownership, velocity, and operational excellence.
Recruit top talent in distributed systems, Kubernetes/infrastructure, and routing/ML systems.
Execution & Delivery
Drive execution of complex, cross-functional initiatives spanning inference runtime, cloud/infrastructure, and ML teams.
Own delivery of capabilities such as:
prefix- and cache-aware routing policies;
P/D-disaggregation-aware scheduling and placement;
Requirements
- ·8+ years of experience in large-scale software engineering, distributed systems, or infrastructure.
- ·2+ years of engineering management experience.
- ·Strong programming skills in C++ and/or Go for performance-critical, concurrent production systems.
- ·Deep experience operating production services on Kubernetes : deployments, autoscaling, rollout strategies, and multi-cluster/multi-region topologies.
- ·Proven track record building and operating highly available, scalable systems: load balancing, failover, scale-out, and reliability engineering.
Benefits
No benefits package published with this listing. Ask about it at first interview.
How to apply
- 1Check the flexibility label above, hybrid, matches where you plan to live and work.
- 2Tailor your CV to the role at Cerebras, mentioning your remote working experience.
- 3Apply directly on the employer's careers page using the button below. Nomaders never handles your application.
Found 4d ago. Last checked today. Always confirm the details on the original posting, salary and location can change after publication.
Listing sourced from Company boards.
Similar roles
Other open software roles with comparable remote rules.
Browse all open rolesPay not stated · You'll be taken to the employer's careers page.