Senior Data Scientist — Model Evaluation
- Role
- Data Science
- Experience
- Senior
Open to US only. Set where you work from to check your eligibility.
No BS summary
Senior Data Scientist focused on model evaluation and quality for football ML models. Strong statistical foundations and production ML pipeline experience required; role is Remote (United States).
Core skills
Required skills
Optional skills
SumerSports is a leading football intelligence technology company that specializes in providing an innovative suite of products for football fans and NFL clubs. We are a collection of executives, engineers, data scientists, and visionaries from NFL clubs, technology startups, finance, and academia.
Our data-driven platform empowers teams with insights and tools to make informed decisions within salary cap constraints. The platform also serves the NCAA, offering insights around the transfer portal and more.
What sets us apart is our unique blend of big tech talent, data scientists, and former NFL personnel, who have a combined 600+ years of NFL experience. Our domain knowledge is augmented by AI and machine learning technologies to create a unique view into many aspects of Football.
**Position Summary
The Senior Data Scientist will be focused on model evaluation and quality for our football ML models. You'll
contribute to the work that sits between research and production, making sure the models we ship measure what we think they measure. We believe good research is less about having the perfect idea and more about building a rigorous
system for testing ideas. Ideas are plentiful. The real work is designing experiments, learning quickly,
and following the evidence.
**Responsibilities
- Contribute to model evaluation across our football models. Drive one or two threads end-to-end.
- Build and extend the tooling we use to test, compare, and ship models.
- Work with football subject-matter experts to design evaluation that reflects how the models will actually get used.
- Catch silent regressions before they ship.
- Pushback when a stakeholder asks for a metric that doesn't measure what they think.
- Build the experimentation harness that lets the team test many ideas fast.
**Qualifications
- Strong statistical foundations. Calibration (reliability and resolution), bootstrapping, inter-rater reliability.
- Track record contributing to model evaluation on a production ML team.
- Production ML pipeline experience. You've seen schema drift, label noise, and leakage.
- You build tooling, not just consume it.
- Self-starter. Takes strategic direction and shapes their own path.
- AI-native. Builds skills, loops, and tooling to accelerate evaluation throughput.
- Stays current with research. Papers, releases, new techniques.
- Domain absorption capacity. You've gotten fluent on a complex domain (sports, finance, biotech, medical) on a prior job
**Bonus Points
- Football or sports analytics exposure.Experience evaluating tracking-data models.
- Skill scores for probabilistic forecasts (Brier, etc.).
- Deep-learning fluency to read training code.
- Human-in-the-loop ML experience.
**Benefits
- Competitive Salary and Bonus Plan
- Comprehensive health insurance plan
- Retirement savings plan (401k) with company match
- Remote working environment
- A flexible, unlimited time off policy
- Generous paid holiday schedule - 13 in total including Monday after the Super Bowl
SumerSports is committed to fair and equitable compensation practices.
Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to skill set, depth of experience, certifications, and specific work location. This may be different in other locations due to differences in the cost of labor.
The total compensation package for this position may also include annual performance bonus, benefits and/or other applicable incentive compensation plans.
The pay range for this role is:
135,000 - 165,000 USD per year (Remote)
What you'll do
- Contribute to model evaluation across our football models; drive one or two threads end-to-end.
- Build and extend the tooling we use to test, compare, and ship models.
- Work with football subject-matter experts to design evaluation that reflects how the models will actually get used.
- Catch silent regressions before they ship and push back when a metric doesn't measure what stakeholders think.
- Build the experimentation harness that lets the team test many ideas fast.
What they require
- Strong statistical foundations: calibration (reliability and resolution), bootstrapping, inter-rater reliability.
- Track record contributing to model evaluation on a production ML team.
- Production ML pipeline experience; familiarity with schema drift, label noise, and leakage.
- Experience building evaluation tooling and experimentation systems.
- Ability to learn complex domains quickly (domain absorption capacity).
Benefits
- Competitive Salary and Bonus Plan
- Comprehensive health insurance plan
- Retirement savings plan (401k) with company match
- Remote working environment
- A flexible, unlimited time off policy
SumerSports is a leading football intelligence technology company that specializes in providing an innovative suite of products for football fans and NFL clubs. We are a collection of executives, engineers, data scientists, and visionaries from NFL clubs, technology startups, finance, and academia. Our data-driven platform empowers teams with insights and tools to make informed decisions within salary cap constraints. The platform also serves the NCAA, offering insights around the transfer portal and more. What sets us apart is our unique blend of big tech talent, data scientists, and former NFL personnel, who have a combined 600+ years of NFL experience. Our domain knowledge is augmented by AI and machine learning technologies to create a unique view into many aspects of Football.