Data Engineer II, Prime Video Advertising

Technology, Data & Digital · Data, AI & Analytics · Data Engineering

In short

Prime Video Advertising is seeking a skilled Data Engineer II in Bengaluru, India, to design, build, and maintain scalable data pipelines and infrastructure. You will leverage AWS technologies and internal tools to handle data ingestion, transformation, and ensure data quality, while also contributing to data governance and continuous improvement efforts. This role requires collaboration with cross-functional teams to deliver tailored data solutions.

Responsibilities

Design and implement scalable, fault-tolerant data pipelines using AWS technologies and internal Amazon tools to extract, transform, and load data from multiple sources.
Collaborate cross-functionally with BIEs, Data Scientists, PMs, and SDEs to understand data requirements and deliver customized data solutions.
Automate infrastructure deployment with CI/CD pipelines and ensure streamlined processes for deployment and maintenance.
Ensure data quality through robust validation, cleansing, and deduplication techniques.
Implement data governance standards, including access control, encryption, data retention, deletion policies, and audit mechanisms to ensure compliance and security.
Continuously improve and optimize data pipelines and infrastructure, staying up to date with emerging technologies and implementing automation and monitoring tools.
Build a scalable and reliable data platform supporting analytics and experimentation for intuitive, self-service data products.
Write high quality code and build scalable applications that interface with critical services and APIs to extract and process unstructured data.
Work with a range of data technologies, including Python, EMR, Spark, Iceberg, Airflow, and many AWS data services like Glue, Athena, Redshift to create end-to-end pipelines that consolidate data from disparate systems.

Requirements

Bachelor's degree
3+ years of data engineering experience
Experience with data modeling, warehousing and building ETL pipelines
Experience with SQL
Knowledge of professional software engineering & best practices for full software development life cycle, including coding standards, software architectures, code reviews, source control management, continuous deployments, testing, and operational excellence
Knowledge of distributed systems as it pertains to data storage and computing
Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam
Experience as a data engineer or related specialty (e.g., software engineer, business intelligence engineer, data scientist) with a track record of manipulating, processing, and extracting value from large datasets
Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
Experience with Apache Spark / Elastic Map Reduce

Desired Qualifications

Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
Master's degree in computer science, engineering, analytics, mathematics, statistics, IT or equivalent
Experience programming with at least one modern language such as C++, C#, Java, Python, Golang, PowerShell, Ruby
Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets