AI Benchmarking Specialist, SP Support - German, International Seller Growth
Technology, Data & Digital · Data, AI & Analytics · Data Science · Artificial Intelligence · QA & Testing
In short
The AI Benchmarking Specialist supports the Seller AI team in evaluating AI and LLM models by designing and executing benchmarking and audit activities. The role involves content accuracy checks, data validation, report preparation, and contributing to process efficiencies and automation, requiring fluency in German and English.
Responsibilities
- Assist in planning and executing benchmarking exercises for AI models, defining test plans, metrics, and acceptance criteria.
- Support content accuracy, relevancy, and privacy checks by reviewing datasets, model outputs, and data handling practices.
- Validate data based on specific annotation guidelines, ensuring accuracy and quality.
- Prepare clear audit and benchmarking reports, including error ratings, root-cause analysis, and recommendations.
- Contribute to presentations for senior stakeholders.
- Maintain organized audit documentation, evidence, and benchmarking datasets.
- Work closely with team members and managers to drive process efficiencies and explore opportunities for automation.
- Enhance productivity and effectiveness of data generation by contributing to the development of AI audit methodologies, checklists, and test frameworks.
Requirements
- Speak, write, and read fluently in German and English at B2 level or above.
Desired Qualifications
- Speak, write, and read fluently in German.
#Editorial#Writing#Content Management
Company
Amazon
Job Posted
1 week ago
Employment Type
Full Time
WorkMode
On Site
Experience Level
Associate
Locations
Bengaluru, India
Applicants
Be an early applicant