AI Benchmarking Specialist, SP Support , International Seller Growth
Technology, Data & Digital · Data, AI & Analytics · Artificial Intelligence · Data Analysis · QA & Testing
In short
The Seller AI team is seeking an AI Benchmarking Associate to support the evaluation of AI systems by designing and executing benchmarking and audit activities. This role involves reviewing datasets, model outputs, and data handling practices to assess model quality, compliance, robustness, and fairness, contributing to the improvement of AI and LLM models for seller growth on Amazon.
Responsibilities
- Assist in planning and executing benchmarking exercises for AI models, defining test plans, metrics, and acceptance criteria.
- Support content accuracy, relevancy, and privacy checks by reviewing datasets and model outputs, escalating potential regulatory risks.
- Validate data based on specific annotation guidelines, ensuring accuracy and quality.
- Prepare clear audit and benchmarking reports, including error ratings, root-cause analysis, and recommendations.
- Contribute to presentations for senior stakeholders.
- Maintain organized audit documentation, evidence, and benchmarking datasets.
- Work with team members to drive process efficiencies and explore automation opportunities.
- Enhance productivity and effectiveness of data generation by contributing to AI audit methodologies and frameworks.
Requirements
- Experience with computer skills, including proficiency in MS Office (Word, Excel, PowerPoint).
- Bachelor's degree or equivalent.
Desired Qualifications
- Experience in natural language data labeling, data annotation, linguistic annotation, or other forms of data markup.
Benefits
- Opportunity to play a pivotal role in shaping the future of selling on Amazon.
- Inclusive culture empowers Amazonians to deliver the best results.
#AI#LLM#Benchmarking#Auditing#Quality Assurance#Data Annotation
Company
Amazon
Job Posted
3 weeks ago
Employment Type
Full Time
WorkMode
On Site
Experience Level
Associate
Locations
Bengaluru, India
Qualification
Bachelor
Applicants
Be an early applicant