Overview
Design and solve complex Physics problems to evaluate AI model performance.
About the hiring company
Based in San Francisco, California, the hiring company is the world’s leading research acceleratorfor frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. The hiring company supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligencewith systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.
Role Overview
- Design and solve challenging Physics problems to evaluate and identify the limitations of large language models.
- Develop clear, high-quality, step-by-step solutions with precise and well-articulated reasoning.
- Collaborate closely with LLM researchers to align problem design with evaluation objectives, particularly in areas where models typically struggle, such as abstraction, multi-step reasoning, and symbolic manipulation.
- Contribute to the development of new evaluation benchmarks based on Physics curricula ranging from early undergraduate concepts to advanced PhD-level topics.
- Analyze model outputs, identify reasoning gaps, and provide insights to improve model accuracy and performance.
- Support the advancement of AI research by combining domain expertise in Physics with analytical and problem-solving methodologies.
Requirements
Analytical Skills
Strong research and analytical abilities, with the capability to evaluate complex information and identify meaningful insights.
Feedback Skills
Ability to provide constructive feedback, detailed annotations, and clear evaluations to support continuous improvement.
Creative Thinking
Strong creative and lateral thinking skills, with the ability to approach problems from multiple perspectives and develop innovative solutions.
Communication
Excellent written and verbal communication skills, with the ability to collaborate effectively in a remote and cross-functional environment.
Independence
Self-motivated, proactive, and capable of working independently while managing tasks and deadlines efficiently in a remote setting.
Technical Setup
Access to a reliable desktop or laptop and a stable high-speed internet connection to support remote collaboration and project work.
Prefered Qualification:
- Candidates who have completed a Master’s, Ph.D., or Postdoctoral degree in Physics, Applied Physics, or a closely related field are eligible and strongly encouraged to apply.
- Strong ability to analyze and solve complex Physics problems using a structured, logical, and methodical approach.
- Ability to explain Physics concepts clearly and effectively through simplified explanations, visual representations, and strong conceptual reasoning.
Offer Details
- Commitments Required: at least 4 hours per day and up to 40 hours per week with 4 hours of overlap with PST.
- Engagement type: Contractor
Evaluation Process
- Shortlisted candidates will be sent a Job Interest Form.
- After the profile review, an assessment will be shared, which must be completed within 24 hours.
- Based on the assessment outcomes, shortlisted candidates will be contacted to schedule an interview.