Cerebras logo

Cerebras

More roles at Cerebras

Engineering Manager, Inference Router

Hybrid

Part remote, part office, you need to live within commuting distance of a named location.

Hybrid · Sunnyvale, CA

Employer listed it 5 weeks ago · Added 3 days ago

Been open since 5 weeks ago, still being checked, but it has been live a while.

Salary

Not stated

Location

Hybrid · Sunnyvale, CA

Timezone

Not stated

Contract

Full-time

Experience

Mid

Category

Software

The employer did not publish a salary, and we have too few comparable roles to estimate one.

What the advert says

Quoted word for word from the employer's advert. Where it says nothing, we say so.

Working hours
Not mentioned in the advert
Where you can work from
Location eligibility not confirmed
Contract
Not mentioned in the advert
Experience asked for
8+ years of experience in large-scale software engineering, distributed systems, or infrastructure.
Pay
Not mentioned in the advert

Remote flexibility

Hybrid

This role is only partly remote, the employer expects time in the office around Sunnyvale, CA, Toronto, CAN, Hybrid, so you need to live within commuting distance.

What the employer says

  • Source listing states candidate location: "Sunnyvale, CA, Toronto, CAN, Hybrid"
  • Listing mentions "Hybrid"

What Nomaders makes of it

  • Not suitable if you plan to move between countries

The quotes above are the employer's own words; the reading is ours. Always check the original listing and employment terms before working from another country.

Before you apply

Questions worth asking: Nomaders interpretation

  • This role involves office time, but the advert does not say how often. Ask how many days a week are expected on site.
  • No pay is stated. Ask for the salary range before investing time in interviews.
  • The advert does not say whether this is employment or a contractor arrangement. Ask, as it affects tax and benefits.

Produced automatically from this advert's text; not reviewed by a person. The employer's own page is the final word.

About the role

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

About the Role

The Inference Router team at Cerebras builds the systems that decide where every inference request runs. We route traffic across all of Cerebras’ data centers and third-party integrations, balancing heterogeneous compute, maximizing cache utilization, and delivering the industry’s lowest time-to-first-token (TTFT) at massive scale. As an Engineering Manager, Inference Router , you will lead the team responsible for the intelligent control plane of the fastest generative AI inference platform in the world. Your team owns global request routing, prefill/decode (P/D) disaggregation, prefix-aware routing for high cache hit rates, autoscaling, and the high-availability infrastructure that keeps the platform fast and reliable under real-time, bursty load. This role operates at the intersection of distributed systems, Kubernetes-based infrastructure, routing-algorithm design, and machine learning. It combines technical leadership, people management, and execution ownership, with direct impact on the latency, throughput, and reliability experienced by every Cerebras inference customer.

What You’ll Do

Technical Leadership

Own the architecture and evolution of the inference routing layer across all Cerebras data centers and third-party inference integrations.

Guide the design of:

global, multi-region request routing and load balancing;

routing for heterogeneous compute, including prefill/decode (P/D) disaggregated serving;

prefix-aware / cache-aware routing to maximize KV-cache reuse and utilization;

low-TTFT, high-throughput request scheduling under concurrent, bursty workloads.

Drive the routing algorithms that trade off cache locality, queueing, load, and latency in real time.

Invest in simulation: build high-fidelity models of the fleet and its workloads so the team can validate routing policies, failover behavior, and scaling decisions under a wide range of conditions without tying up precious accelerator capacity.

Partner with inference runtime, cloud/platform, hardware, and ML teams to optimize end-to-end performance.

Team Leadership

Build, manage, and grow a team of distributed systems, infrastructure, and ML systems engineers.

Provide technical direction, mentorship, and career development.

Foster a culture of ownership, velocity, and operational excellence.

Recruit top talent in distributed systems, Kubernetes/infrastructure, and routing/ML systems.

Execution & Delivery

Drive execution of complex, cross-functional initiatives spanning inference runtime, cloud/infrastructure, and ML teams.

Own delivery of capabilities such as:

prefix- and cache-aware routing policies;

P/D-disaggregation-aware scheduling and placement;

Requirements

  • ·8+ years of experience in large-scale software engineering, distributed systems, or infrastructure.
  • ·2+ years of engineering management experience.
  • ·Strong programming skills in C++ and/or Go for performance-critical, concurrent production systems.
  • ·Deep experience operating production services on Kubernetes : deployments, autoscaling, rollout strategies, and multi-cluster/multi-region topologies.
  • ·Proven track record building and operating highly available, scalable systems: load balancing, failover, scale-out, and reliability engineering.

Benefits

No benefits package published with this listing. Ask about it at first interview.

How to apply

  1. 1Check the flexibility label above, hybrid, matches where you plan to live and work.
  2. 2Tailor your CV to the role at Cerebras, mentioning your remote working experience.
  3. 3Apply directly on the employer's careers page using the button below. Nomaders never handles your application.

Found 4d ago. Last checked today. Always confirm the details on the original posting, salary and location can change after publication.

Listing sourced from Company boards.

Similar roles

Other open software roles with comparable remote rules.

Browse all open roles

Pay not stated · You'll be taken to the employer's careers page.