OpenAI
People Research Data Scientist, AI Fairness & Bias
Part remote, part office, you need to live within commuting distance of a named location.
Hybrid · San Francisco
Employer listed it 3 months ago · Added 2 months ago
Listed for 56 days before it closed.
This job has closed
OpenAI has taken this listing down, so it is no longer accepting applications. The details below are kept for reference only.
Browse open rolesSalary
$170k to $220k per year
Location
Hybrid · San Francisco
Timezone
Not stated
Contract
Full-time
Experience
Mid
Category
Data
Published by the employer
What the advert says
Quoted word for word from the employer's advert. Where it says nothing, we say so.
- Working hours
- Not mentioned in the advert
- Where you can work from
This role is preferred to be based in San Francisco, CA.
- Contract
- Not mentioned in the advert
- Experience asked for
- Not mentioned in the advert
- Pay
- Not mentioned in the advert
Remote flexibility
Hybrid
This role is only partly remote, the employer expects time in the office around San Francisco, Hybrid, so you need to live within commuting distance.
What the employer says
- Source listing states candidate location: "San Francisco, Hybrid"
- Listing mentions "Hybrid"
What Nomaders makes of it
- Not suitable if you plan to move between countries
The quotes above are the employer's own words; the reading is ours. Always check the original listing and employment terms before working from another country.
Before you apply
Experience: employer states
Written as a requirement
- “AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.”
Questions worth asking: Nomaders interpretation
- This role involves office time, but the advert does not say how often. Ask how many days a week are expected on site.
- The advert does not say whether this is employment or a contractor arrangement. Ask, as it affects tax and benefits.
Produced automatically from this advert's text; not reviewed by a person. The employer's own page is the final word.
About the role
About the Team
OpenAI’s People team hires, engages, and retains world-class talent to safely build and deploy AGI that benefits all of humanity. The People Analytics team helps leaders make rigorous, evidence-based talent decisions and ensures that the systems supporting those decisions are valid, reliable, fair, and accountable.
About the Role
As a People Data Scientist focused on AI fairness and bias testing, you will help establish how OpenAI evaluates AI-assisted People systems and high-impact talent processes. You will design and conduct rigorous assessments to identify, measure, and mitigate potential bias across the lifecycle of models, agents, decision-support tools, and automated workflows.
Your work will span the entire employee life-cycle, such as hiring, performance, promotion, employee development, workforce planning, etc. You will evaluate both technical systems and the broader human-AI decision processes in which they operate, examining not only model performance but also data quality, measurement validity, differential outcomes, human oversight, and unintended consequences.
We’re looking for an experienced data scientist or applied researcher who can translate complex fairness questions into defensible evaluation strategies, scalable testing infrastructure, and clear recommendations for technical teams and senior leaders.
This role is preferred to be based in San Francisco, CA.
In this role, you will:
Define and lead fairness and bias-testing strategies for AI-assisted People processes, models, agents, and decision-support systems from development through deployment and ongoing monitoring.
Design rigorous algorithmic audits and validation studies, including adverse-impact analysis, subgroup and intersectional evaluation, error-rate analysis, calibration, measurement invariance, reliability, criterion-related validity, and sensitivity testing.
Identify the appropriate fairness criteria for each use case, evaluate tradeoffs among competing definitions of fairness, and clearly document the assumptions, limitations, and residual risks of each approach.
Evaluate end-to-end human-AI decision systems, including model outputs, user behavior, human overrides, escalation pathways, and whether AI assistance changes the quality, consistency, or equity of decisions.
Develop evaluation approaches for generative and agentic AI, including test-set design, counterfactual testing, behavioral evaluation, human-rating studies, robustness testing, and analysis of disparate performance across populations and contexts.
Investigate the sources of observed disparities, including data representation, label and measurement bias, proxy variables, model design, decision thresholds, workflow design, and differential adoption or usage.
Partner with engineering, People Operations, Legal, Privacy, Security, and People Systems teams to recommend and evaluate mitigations such as data improvements, model changes, threshold adjustments, workflow redesign, monitoring controls, and additional human oversight.
Build scalable fairness-evaluation infrastructure, including reusable datasets, automated validation pipelines, regression tests, monitoring systems, self-service tools, and standardized reporting.
Establish research and documentation standards for fairness test plans, dataset and model documentation, validation reports, limitations, monitoring plans, and decision records.
Translate complex findings into concise, decision-ready narratives, helping leaders understand the significance of identified risks, the strength of the evidence, available mitigation options, and remaining uncertainty.
You might thrive in this role if you have:
Deep expertise in algorithmic fairness, bias measurement, responsible AI, psychometrics, applied statistics, or the evaluation of high-impact decision systems.
Exceptional strength in research design, measurement, experimentation, causal inference, and statistical modeling.
Hands-on experience applying methods such as subgroup and intersectional analysis, adverse-impact testing, equalized-odds and equal-opportunity analysis, demographic-parity assessment, calibration analysis, counterfactual testing, measurement invariance, reliability analysis, and validation studies.
Strong judgment about the limitations of fairness metrics, including the ability to determine which measures are appropriate for a particular decision context rather than applying a single universal definition of fairness.
Experience evaluating machine-learning models, generative AI systems, agents, or human-AI workflows using quantitative and qualitative evidence.
Requirements
- ·Deep expertise in algorithmic fairness, bias measurement, responsible AI, psychometrics, applied statistics, or the evaluation of high-impact decision systems.
- ·Exceptional strength in research design, measurement, experimentation, causal inference, and statistical modeling.
- ·Experience evaluating machine-learning models, generative AI systems, agents, or human-AI workflows using quantitative and qualitative evidence.
- ·High proficiency in Python or R and SQL, with experience working across complex, sensitive, and imperfect datasets.
- ·Experience building reproducible evaluation pipelines, automated testing frameworks, analytical tools, monitoring systems, or governed research workflows.
Benefits
No benefits package published with this listing. Ask about it at first interview.
Applying
This listing has closed, so there is no application link. The details above are kept for reference only.
Found 2mo ago. Last checked 30 Oct. Always confirm the details on the original posting, salary and location can change after publication.
Listing sourced from Company boards.
Similar open jobs
This role is closed, applications are no longer accepted.