Assessment Psychometrist

Pearl Talent - Psychometrician We're hiring a Psychometrician for Pearl, which finds exceptional talent from around the world, trains them to be AI-native, and places them into operational roles at startups as managed contractors: from client-facing roles to software engineers to executive assistants. We're 3x founders who've bootstrapped our company to a couple million in ARR and are adding six to seven figures in net new annualized revenue each month. Our clients span venture-backed tech and healthcare, including fast-growing startups and phenomenal US-based businesses that have raised over $3B in funding from Sequoia, a16z, Founders Fund, Y Combinator, and other top VC firms. Today we're roughly 50 people managing a few hundred talents, growing fast and into new verticals. We started Pearl because we believe that even though opportunity isn't created equal in the world, ambitious talent is. Location: Fully remote Purpose of Your Role You'll be the scientific backbone of our AI voice-model initiative. We're building a model that infers psychological constructs from voice, and its quality will only ever be as good as the measurement system behind it. This is not a machine-learning engineering role. You won't build the model itself. Instead, you'll define what we should measure, design and validate the instruments that measure it, and convert those measurements into defensible ground-truth labels and evaluation criteria for model training. You'll also be the person in the room with the authority — and the responsibility — to say what voice can't responsibly tell us about a person. What You'll Own 1. The Psychometric Framework Review the literature and evaluate established frameworks (Big Five/OCEAN, HEXACO, DISC, and other relevant models) for validity, usefulness, and fit for our use case Recommend which constructs and dimensions to include, exclude, or treat cautiously — and explicitly identify traits that cannot be responsibly inferred from voice Build and maintain a clear construct map connecting constructs, dimensions, indicators, survey items, and resulting scores, documented as the single source of truth for Research, Data, and AI teams 2. Instrument Design and Validation Design psychometric surveys that serve as a high-quality measurement source for participants who also provide voice samples: items, response scales, scoring rules, reverse-coded items, and attention checks, while minimizing fatigue and response bias Decide when to use validated existing scales, adapt them, or develop fit-for-purpose measures — then pilot and iterate based on empirical performance Run the full validation battery: reliability (McDonald's omega, Cronbach's alpha, item-total analysis), EFA and CFA to test dimensional structure, construct/convergent/discriminant/criterion validity, test-retest where appropriate, and IRT where useful Recommend sample sizes, pilot methodology, and evidence thresholds a measure must clear before it's used for training 3. Ground Truth and Model Evaluation Partner with AI and Data teams to transform psychometric responses into defensible training labels: continuous scores, categories, normalization, and confidence/reliability information Define rules for missing, inconsistent, or low-quality responses, and the criteria for when a label is reliable enough to train on Design the framework for comparing survey-based ground truth against voice-model predictions, and define evaluation metrics that reflect psychometric validity — not just ML performance Analyze where the model performs well, where it fails, and whether prediction quality differs by construct or population 4. Bias, Fairness, and Scientific Guardrails Evaluate measurement invariance and potential bias across languages, cultures, and populations, including the impact of translation and sampling choices Define the scientific limits around what conclusions may and may not be drawn from voice-based predictions, and partner with Product and AI to prevent unsupported or misleading interpretations Translate complex statistical findings into practical decisions for technical and non-technical stakeholders, keeping methodology aligned with current peer-reviewed research What Success Looks Like Within your first cycle, the full measurement pipeline runs through you: framework selected, construct map built, Survey V1 designed and piloted, statistically validated, refined into Survey V2, and converted into a scoring system that produces training labels the AI team trusts. When the voice model ships predictions, they're validated against ground truth you built — and every claim the product makes about a person is one your framework can scientifically defend. Requirements Advanced degree (MSc or PhD preferred) in Psychometrics, Quantitative Psychology, Psychological Measurement, I/O Psychology, Behavioral Science, or a closely related quantitative field Hands-on instrument experience. You've designed, validated, and refined psychometric instruments yourself — not just used them A strong statistical foundation in factor analysis, reliability, validity, and measurement theory; IRT experience is a strong plus Comfort with messy, real-world data. You can work with imperfect behavioral datasets and still make evidence-based recommendations Cross-functional fluency. You're comfortable collaborating with AI/ML engineers, data engineers, and product teams; Python or R experience is highly desirable Scientific backbone. You'll challenge unsupported assumptions, define responsible limits for AI-based psychological inference, and hold the line when it matters Benefits Build and Grow Quickly - We’re scaling fast, and we trust that you’ll know best on the ground what needs to be done. You’ll have the opportunity to step into leadership early and own decisions that shape how our company grows. Fully Remote. Forever. - We’ve built Pearl with a multicultural DNA and teammates across 23 countries. We trust that the best work isn’t done behind a cubicle Unlimited PTO - We trust that you’ll get your work done, and we want to create space for you to take time away with the people you care about. Global Retreats - We create space for our teammates to get to know each other as people, rather than just to-do lists. We’ve shared meals, laughs, and sunrises across the world in places like Cancun, El Nido, Boracay, and Siargao**.** Ambitious and Kind Team - We build with the most competent people we know, and we maintain a low-ego, no-assholes policy. We’re looking for people who are sharp, kind, and open to being vulnerable when it matters.