Data Engineer - Azure Data Factory (ADF), Databricks

Job Summary

The Developer will be responsible for managing and optimizing data workflows using Snowflake, Azure Data Factory (ADF), and Data Bricks. The role involves developing and maintaining data pipelines, ensuring data quality and integrity, as well as supporting data transformation and processing tasks.

Key Responsibilities

1. Develop, and implement data pipelines using snowflake, azure data factory (adf), and data bricks.
2. Optimize existing data workflows to improve efficiency and performance.
3. Collaborate with data engineers and data scientists to support data analytics and reporting requirements.
4. Monitor data pipelines and troubleshoot issues to ensure smooth operations.
5. Implement best practices for data governance, security, and compliance.
6. Document data pipeline processes and maintain clear documentation for reference.

Skill Requirements

1. Knowledge of snowflake architecture, sql queries, and performance tuning techniques.
2. Knowledge of azure data factory (adf) including creating pipelines, data flows, and data sets.
3. Knowledge of data bricks for data processing, transformation, and analytics.
4. Understanding of data warehousing concepts, etl processes, and data modeling.
5. Ability to work collaboratively in a team environment and communicate effectively with stakeholders.
6. Problem-solving skills and attention to detail for ensuring data accuracy and reliability.

Other Requirements

1.Relevant certifications in Snowflake, Azure Data Factory, or DataBricks will be a plus.