AI Benchmarking Specialist, SP Support - Italian, International Seller Growth
Technology, Data & Digital · Data, AI & Analytics · Data Analysis · Artificial Intelligence · QA & Testing
In short
The Seller AI team is seeking an AI Benchmarking Specialist to support the evaluation of AI and LLM models. This role involves designing and executing benchmarking and audit activities, reviewing datasets, and preparing reports to ensure model quality, compliance, and fairness. Fluency in Italian is required.
Responsibilities
- Assist in planning and executing benchmarking exercises for AI models, including defining test plans, metrics, and acceptance criteria.
- Support content accuracy, relevancy, and privacy checks by reviewing datasets, model outputs, and data handling practices.
- Validate data based on specific annotation guidelines, ensuring the accuracy and quality of the collected information.
- Prepare clear audit and benchmarking reports, including error ratings, root-cause analysis, and recommendations.
- Contribute to presentations for senior stakeholders.
- Maintain organized audit documentation, evidence, and benchmarking datasets.
- Work closely with team members and managers to drive process efficiencies and explore opportunities for automation.
- Enhance the productivity and effectiveness of data generation by contributing to the development and continuous improvement of AI audit methodologies.
Requirements
- Speak, write, and read fluently in Italian
Desired Qualifications
- Experience using managed ML/AI solutions
#AI#benchmarking#specialist#seller growth#Italian#International#Amazon#Gen-AI#LLM#auditing#quality assurance#content management
Company
Amazon
Job Posted
1 month ago
Employment Type
Full Time
WorkMode
On Site
Experience Level
Associate
Locations
Bengaluru, India
Applicants
Be an early applicant