Data Engineer - II (FTC), DAVENGERS(Data Avengers)
Technology, Data & Digital · Data, AI & Analytics · Data Engineering
In short
We are seeking a Data Engineer to design, implement, and support an analytical data infrastructure for BI and ML applications. You will manage AWS resources, collaborate with scientists and analysts to build data pipelines, and explore new AWS technologies.
Responsibilities
- Design, implement and support an analytical data infrastructure serving both business intelligence and machine learning applications
- Manage AWS resources including EC2, Redshift, EMR-Spark etc
- Collaborate with applied scientists to integrate and build data pipelines as necessary for building and training machine learning models
- Collaborate with Product Managers, Financial and Business analysts to recognize and help adopt best practices in reporting and analysis: data integrity, test design, analysis, validation, and documentation
- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies
- Explore and learn the latest AWS technologies to provide new capabilities and increase efficiency
- Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning
- Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers
Requirements
- 3+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with SQL
Desired Qualifications
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
- Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
#Data Engineering#ETL#Data Modeling#AWS