Cerebras logo

Cerebras

More roles at Cerebras

Staff Software Engineer, Inference API

Hybrid

Part remote, part office, you need to live within commuting distance of a named location.

Hybrid · Sunnyvale, CA

Employer listed it 8 weeks ago · Added 8 weeks ago

Been open since 8 weeks ago. Long-running listings are sometimes left up after the role is filled.

Salary

Not stated

Location

Hybrid · Sunnyvale, CA

Timezone

Not stated

Contract

Full-time

Experience

Lead

Category

Software

The employer did not publish a salary, and we have too few comparable roles to estimate one.

What the advert says

Quoted word for word from the employer's advert. Where it says nothing, we say so.

Working hours
Not mentioned in the advert
Where you can work from
Location eligibility not confirmed
Contract
Not mentioned in the advert
Experience asked for
Not mentioned in the advert
Pay
Not mentioned in the advert

Remote flexibility

Hybrid

This role is only partly remote, the employer expects time in the office around Sunnyvale, CA, Toronto, CAN, Hybrid, so you need to live within commuting distance.

What the employer says

  • Source listing states candidate location: "Sunnyvale, CA, Toronto, CAN, Hybrid"
  • Listing mentions "Hybrid"

What Nomaders makes of it

  • Not suitable if you plan to move between countries

The quotes above are the employer's own words; the reading is ours. Always check the original listing and employment terms before working from another country.

Before you apply

Questions worth asking: Nomaders interpretation

  • This role involves office time, but the advert does not say how often. Ask how many days a week are expected on site.
  • No pay is stated. Ask for the salary range before investing time in interviews.
  • The advert does not say whether this is employment or a contractor arrangement. Ask, as it affects tax and benefits.

Produced automatically from this advert's text; not reviewed by a person. The employer's own page is the final word.

About the role

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

About the Role

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.

We are hiring a Software Engineer to build and evolve the ML API layer that makes this heterogeneous serving system accessible, reliable, and easy to use. You will work across our inference APIs, model integration layer, request-routing services and Cerebras inference platform to deliver a consistent experience across models and accelerator backends.

This role sits at the intersection of machine learning systems, API design, model serving, and distributed systems. You will enable new model architectures and inference capabilities, define stable user-facing behavior, and ensure that features such as streaming, sampling, tool use, structured outputs, multimodal inputs, and model configuration behave correctly and consistently in production.

You will work closely with model enablement, compiler, runtime, cloud infrastructure, product, customer-facing teams, and customers directly. This is a hands-on software engineering role for someone who enjoys turning rapidly evolving ML capabilities into durable, production-quality APIs.

Responsibilities

Build production ML inference APIs. Design, implement, and maintain APIs for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and other emerging inference capabilities.

Deliver a unified serving experience. Create consistent request and response semantics across GPU prefill, Cerebras decode, and other heterogeneous inference backends.

Enable new models and capabilities. Integrate emerging foundation models, tokenizers, prompt formats, sampling methods, attention variants, multimodal inputs, and model-specific features into the serving platform.

Own API compatibility and evolution. Maintain compatibility with widely adopted inference interfaces while designing Cerebras-specific extensions. Establish clear versioning, deprecation, validation, and backward compatibility practices.

Integrate with model-serving runtimes. Extend and integrate custom inference services with vLLM, PyTorch, Hugging Face libraries, the AMD ROCm stack, and Cerebras runtime components.

Support disaggregated inference. Build the control and data paths required to coordinate GPU prefill with Cerebras decode, including request routing, state transfer, error handling, retries, and lifecycle management.

Improve serving performance. Optimize streaming behavior, time to first token, request latency, throughput, batching, serialization, tokenization, scheduling, and communication between serving components.

Ensure functional and numerical correctness. Build validation systems for tokenization, sampling, logits, generated outputs, precision changes, model upgrades, determinism, and compatibility across serving backends.

Strengthen reliability and observability. Define end-to-end service indicators and build structured logging, tracing, metrics, dashboards, health checks, and diagnostic tooling for production inference traffic.

Develop testing and qualification infrastructure. Create conformance tests, workload-replay tools, model-validation suites, performance benchmarks, integration tests, and release gates.

Improve developer experience. Build intuitive configuration, SDKs, documentation, examples, debugging tools, and self-service workflows for internal developers, customers, and partners.

Collaborate across the stack. Partner with compiler, runtime, kernel, cloud, product, and solutions teams to translate model and customer requirements into scalable serving capabilities.

Minimum Qualifications

5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems.

Strong programming ability in Python and Go plus experience developing performance-sensitive or highly concurrent services in C++, Rust, or a similar systems language.

Experience building stable APIs with clear validation, error handling, observability, compatibility, and versioning practices.

Experience integrating software across service, framework, runtime, and infrastructure boundaries.

Requirements

  • ·5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems.
  • ·Strong programming ability in Python and Go plus experience developing performance-sensitive or highly concurrent services in C++, Rust, or a similar systems language.
  • ·Experience building stable APIs with clear validation, error handling, observability, compatibility, and versioning practices.
  • ·Experience integrating software across service, framework, runtime, and infrastructure boundaries.
  • ·Experience designing or maintaining OpenAI-compatible, gRPC, REST, or streaming inference APIs.

Benefits

No benefits package published with this listing. Ask about it at first interview.

How to apply

  1. 1Check the flexibility label above, hybrid, matches where you plan to live and work.
  2. 2Tailor your CV to the role at Cerebras, mentioning your remote working experience.
  3. 3Apply directly on the employer's careers page using the button below. Nomaders never handles your application.

Found 2mo ago. Last checked today. Always confirm the details on the original posting, salary and location can change after publication.

Listing sourced from Company boards.

Similar roles

Other open software roles with comparable remote rules.

Browse all open roles

Pay not stated · You'll be taken to the employer's careers page.