Ramp logo

Ramp

More roles at Ramp

Research Engineer, Benchmarks & Evaluation

Hybrid

Part remote, part office, you need to live within commuting distance of a named location.

Hybrid · New York, NY (HQ)

Employer listed it 2 weeks ago · Added 2 weeks ago

Been open since 2 weeks ago, still being checked, but it has been live a while.

Salary

$204k to $352k per year

Location

Hybrid · New York, NY (HQ)

Timezone

Not stated

Contract

Full-time

Experience

Mid

Category

Software

Published by the employer

What the advert says

Quoted word for word from the employer's advert. Where it says nothing, we say so.

Working hours
Not mentioned in the advert
Where you can work from
In-office perks: lunch, snacks, drinks, and more
Contract
Benefits available to all full-time Ramp employees (Global)
Experience asked for
Not mentioned in the advert
Pay
Fertility HRA (up to $10,000 per year)

Remote flexibility

Hybrid

This role is only partly remote, the employer expects time in the office around New York, NY (HQ), San Francisco, CA, Hybrid, so you need to live within commuting distance.

What the employer says

  • Source listing states candidate location: "New York, NY (HQ), San Francisco, CA, Hybrid"
  • Listing mentions "Hybrid"

What Nomaders makes of it

  • Not suitable if you plan to move between countries

The quotes above are the employer's own words; the reading is ours. Always check the original listing and employment terms before working from another country.

Before you apply

Questions worth asking: Nomaders interpretation

  • This role involves office time, but the advert does not say how often. Ask how many days a week are expected on site.

Produced automatically from this advert's text; not reviewed by a person. The employer's own page is the final word.

About the role

About Ramp

Ramp is building the smart infrastructure for finance teams, embedded in the transaction flow of every dollar a business spends. We automate how over $275B in annualized spend flows in and out of 85,000+ companies: authorizing payments, flagging risk, categorizing spend, and closing books.

The problems are high-stakes, data-dense, and unforgiving.

We hire people with high agency and high urgency. We look for slope over intercept. We care less about where you trained and more about what you’ve built. At Ramp, everyone is a builder who owns problems end to end and makes consequential decisions that shape the outcome.

The median Ramp customer saves 7.5% and grows revenue 16% in their first year – far in excess of businesses operating without Ramp. We believe every ambitious company deserves the same.

If you want to build systems that directly shape how companies move and manage billions, Ramp is the place to do it.

About Ramp Labs

Ramp Labs is Ramp's AI research group. We post-train models on finance work, build the environments and evals they learn from, and ship them into the products 70,000+ companies use to move money. We think the defensible part of an AI product is the loop: proprietary environments, evals that track what customers care about, and models trained against both.

About the Role

Every model decision at Ramp Labs runs through a benchmark. Which model to route to. Whether a post-training run worked. Whether an agent is ready to post a corrective journal entry to a customer's ledger without a human in the loop.

You'll own Ramp's benchmarking effort. You'll turn real finance work (expense policy review, invoice coding, month-end close, spreadsheet Q&A) into benchmarks and RL environments, build the graders, and tell the team what is actually improving.

What You’ll Do

Build benchmarks from real workflows: turn agent traces, customer edge cases and expert labels into versioned benchmark suites for long-horizon agents that run for hours across tools and systems, plus tool use, multi-step reasoning and financial judgment

Own the benchmarking effort: decide what we measure, run benchmark suites across models and agent versions, and report results the team makes decisions on

Design graders: rubrics, programmatic checks and llm-as-judge, calibrated against human agreement, with a clear view of where each one breaks

Close the loop with post-training: turn evals into reward functions and RL environments, and catch reward hacking and contamination before they ship

Publish external benchmarks and write-ups on how models perform at real finance work

What You Need

Hands-on experience building LLM benchmarks or evals that people used to make real decisions

Strong Python and software engineering; your eval code is reproducible and fast

Statistical judgment: variance, confidence intervals, sample size, inter-rater agreement, and knowing when a delta is noise

Working knowledge of post-training (SFT, RL with verifiable rewards) and how evals feed into it

Comfort with SQL and large datasets; you'll work directly from production traces

Taste for what is worth measuring

Requirements

The employer hasn't listed requirements separately, they're described in the role summary above and on the original listing.

Benefits

  • ·Centralized home-office equipment ordering
  • ·Health and wellness stipend
  • ·Budget for intra-office travel
  • ·Weekly coffee stipend
  • ·100% medical, dental & vision insurance coverage for you, with partial coverage for dependents

How to apply

  1. 1Check the flexibility label above, hybrid, matches where you plan to live and work.
  2. 2Tailor your CV to the role at Ramp, mentioning your remote working experience.
  3. 3Apply directly on the employer's careers page using the button below. Nomaders never handles your application.

Found 15d ago. Last checked 17 Nov. Always confirm the details on the original posting, salary and location can change after publication.

Listing sourced from Company boards.

Similar roles

Other open software roles with comparable remote rules.

Browse all open roles

$204k to $352k per year · You'll be taken to the employer's careers page.