Cerebras
More roles at CerebrasSenior Software Engineer, ML API – AI Inference
Part remote, part office, you need to live within commuting distance of a named location.
Hybrid · Toronto, CAN
Employer listed it 3 weeks ago · Added 3 weeks ago
Been open since 3 weeks ago, still being checked, but it has been live a while.
Salary
Not stated
Location
Hybrid · Toronto, CAN
Timezone
Not stated
Contract
Full-time
Experience
Senior
Category
Software
The employer did not publish a salary, and we have too few comparable roles to estimate one.
What the advert says
Quoted word for word from the employer's advert. Where it says nothing, we say so.
- Working hours
- Not mentioned in the advert
- Where you can work from
- Location eligibility not confirmed
- Contract
- Not mentioned in the advert
- Experience asked for
- Not mentioned in the advert
- Pay
- Not mentioned in the advert
Remote flexibility
Hybrid
This role is only partly remote, the employer expects time in the office around Toronto, CAN, Sunnyvale, CA, Hybrid, so you need to live within commuting distance.
What the employer says
- Source listing states candidate location: "Toronto, CAN, Sunnyvale, CA, Hybrid"
- Listing mentions "Hybrid"
What Nomaders makes of it
- Not suitable if you plan to move between countries
The quotes above are the employer's own words; the reading is ours. Always check the original listing and employment terms before working from another country.
Before you apply
Questions worth asking: Nomaders interpretation
- This role involves office time, but the advert does not say how often. Ask how many days a week are expected on site.
- No pay is stated. Ask for the salary range before investing time in interviews.
- The advert does not say whether this is employment or a contractor arrangement. Ask, as it affects tax and benefits.
Produced automatically from this advert's text; not reviewed by a person. The employer's own page is the final word.
About the role
Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.
About the Role
Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.
We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes this heterogeneous serving system accessible, reliable, and easy to use. You will own significant features and components across our inference APIs, model integration layer, and request-routing services, delivering a consistent experience across models and accelerator backends.
This role sits at the intersection of machine learning systems, API design, model serving, and distributed systems. You will translate evolving model capabilities and customer requirements into clear designs, lead implementation across multiple components, and carry features through validation, rollout, and production operation. Your work will help ensure that streaming, sampling, tool use, structured outputs, multimodal inputs, and model configuration behave correctly and consistently.
You will work closely with model enablement, compiler, runtime, cloud infrastructure, product, and customer-facing teams, as well as customers directly. This is a hands-on individual-contributor role with opportunities to lead small project teams, mentor engineers, and improve the quality and reliability of production inference software.
Responsibilities
Own features end to end. Design, implement, test, and operate features for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference capabilities.
Lead multi-component delivery. Write clear design documents, evaluate technical tradeoffs, break projects into deliverable milestones, and coordinate implementation with a small group of engineers and partner teams.
Deliver consistent API behavior. Implement clear request and response semantics across GPU prefill, Cerebras decode, and other inference backends. Maintain compatibility with widely adopted inference interfaces and evolve owned APIs through validation, versioning, and deprecation best practices.
Enable new models and capabilities. Integrate foundation models, tokenizers, chat templates, generation configuration, sampling methods, and model-specific features into the serving platform.
Integrate serving components. Extend inference services and their integration with frameworks and runtimes such as vLLM, PyTorch, Hugging Face libraries, LLGuidance, and Cerebras runtime components.
Support disaggregated inference. Implement routing, state-transfer integration, error handling, retries, cancellation, and request lifecycle management to coordinate GPU prefill with Cerebras decode.
Deliver measurable performance improvements. Diagnose bottlenecks and improve time to first token, streaming latency, throughput, and resource efficiency through changes to tokenization, serialization, batching, scheduling, and communication.
Own correctness and release quality. Build conformance tests, model-validation suites, numerical comparisons, integration tests, and performance benchmarks that catch regressions in owned features and support safe releases.
Strengthen production reliability. Improve service metrics, tracing, logging, alerts, and diagnostic tools. Investigate production incidents, resolve root causes, and follow through on improvements to resilience and incident response.
Improve customer and developer experience. Partner with product, support, and solutions teams to understand user needs and make practical tradeoffs. Improve configuration, documentation, examples, and debugging workflows.
Raise the engineering quality bar. Provide thoughtful design and code reviews, mentor less-experienced engineers, share domain knowledge, and proactively remove delivery roadblocks.
Minimum Qualifications
5+ years of software engineering experience, including independent ownership of production services, distributed systems, or substantial software components.
Strong programming ability in Python and proficiency in Go, C++, Rust, or another language used to build production services.
Experience designing and delivering features that span multiple components, including documenting designs, evaluating tradeoffs, and coordinating implementation with other engineers.
Experience building and evolving production APIs, with sound practices for validation, error handling, compatibility, and testing. Familiarity with REST, gRPC, or streaming protocols.
Experience integrating software across service, framework, runtime, or infrastructure boundaries.
Requirements
- ·5+ years of software engineering experience, including independent ownership of production services, distributed systems, or substantial software components.
- ·Strong programming ability in Python and proficiency in Go, C++, Rust, or another language used to build production services.
- ·Experience designing and delivering features that span multiple components, including documenting designs, evaluating tradeoffs, and coordinating implementation with other engineers.
- ·Experience building and evolving production APIs, with sound practices for validation, error handling, compatibility, and testing. Familiarity with REST, gRPC, or streaming protocols.
- ·Experience integrating software across service, framework, runtime, or infrastructure boundaries.
Benefits
No benefits package published with this listing. Ask about it at first interview.
How to apply
- 1Check the flexibility label above, hybrid, matches where you plan to live and work.
- 2Tailor your CV to the role at Cerebras, mentioning your remote working experience.
- 3Apply directly on the employer's careers page using the button below. Nomaders never handles your application.
Found 22d ago. Last checked today. Always confirm the details on the original posting, salary and location can change after publication.
Listing sourced from Company boards.
Pay not stated · You'll be taken to the employer's careers page.