Senior Data Lead - Azure Data Factory (ADF), Databricks

Job Summary

Description
CPT 260400333
Require a Senior Data Architect for the Data Enablement Factory Area (DEF). DEF is responsible for handling ingestion of Data into UDP which is primarily on the Azure Tech stack
echnical Skills

• Proven expertise in data ingestion and integration architectures across diverse data ecosystems.

• Deep experience in cloud-based architectures and DW/BI/Analytics domain knowledge with experience in Microsoft Azure technologies like ADF, AEH, Databricks and Azure storage.

• Strong understanding of ETL/ELT concepts, streaming, and event-driven pipelines.

• Exposure to data modelling and transformation concepts, primarily to support consistent data structures post-ingestion.

• Expertise in security, authentication, and access control within Azure cloud (Managed Identities, OAuth 2.0, Key Vault integration).

• Experience with message queues and integration technologies such as Kafka, IBM MQ, or Event Hub is preferred.

• Experience defining ingestion SLAs, throughput optimization, and monitoring strategies.
Should be ready to work from office

Key Responsibilities

1. Lead and manage technical teams, providing guidance and support in the implementation of snowflake, azure data factory (adf), and data bricks projects.
2. Design, develop, and maintain data pipelines using snowflake, azure data factory (adf), and data bricks to support business requirements.
3. Work closely with stakeholders to gather requirements, identify opportunities for data analytics, and propose data driven solutions.
4. Monitor and troubleshoot data pipelines, ensuring data quality, reliability, and performance.
5. Stay updated with the latest trends and technologies in data management and analytics, incorporating best practices into projects.
6. Collaborate with cross functional teams to integrate data solutions with existing systems and applications.
7. Provide technical expertise and mentorship to team members, promoting a culture of learning and innovation.

Skill Requirements

1. Proficiency in snowflake, including data modeling, querying, and performance optimization.
2. Strong experience in azure data factory (adf) for data integration and orchestration.
3. Handson knowledge of data bricks for data engineering, data processing, and machine learning.
4. Ability to design, develop, and optimize complex data pipelines for etl processes.
5. Strong problem-solving skills and the ability to troubleshoot data pipeline issues.
6. Excellent communication skills to interact with technical and nontechnical stakeholders effectively.
7. Strong leadership skills to guide and motivate technical teams towards project delivery and success.
8. Experience in agile methodologies and project management practices for efficient project execution.

Other Requirements

1.Relevant certifications in Snowflake, Azure Data Factory (ADF), DataBricks are a plus.