Azure Senior Data Lead

Teknik, data och digitalt · Data, AI och analys · Data engineering

I korthet

We are looking for an experienced Azure Data Lead to design and build scalable data platforms on Microsoft Azure, utilizing services like Azure Data Factory, Databricks, and ADLS Gen2. The role involves developing data pipelines, implementing data models, and optimizing performance and costs.

Ansvarsområden

Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Data Factory (ADF) and Azure Databricks.
Build and manage enterprise data platforms using ADLS Gen2, Medallion Architecture (Bronze, Silver, Gold layers), and Lakehouse principles.
Develop robust ETL/ELT solutions using PySpark, Spark SQL, and Python.
Implement and optimize batch and streaming data processing solutions.
Design and maintain Delta Live Tables (DLT) for dependable and automated data transformation workflows.
Build streaming pipelines using Auto Loader, Structured Streaming, and Delta Lake technologies.
Establish and govern data assets through Unity Catalog for centralized security, governance, and metadata management.
Create and maintain dimensional and enterprise data models to support analytics and reporting requirements.
Develop and optimize Power BI datasets and semantic models for business intelligence and reporting.
Implement CI/CD pipelines for data engineering solutions using Azure DevOps or similar tools.
Monitor and improve data platform performance through workload optimization, partitioning strategies, caching, and query tuning.
Drive cloud cost optimization initiatives across Azure Data Lake, Databricks, storage, and compute resources.
Ensure adherence to data governance, security, compliance, and best practices.
Collaborate with architects, business stakeholders, analysts, and application teams to understand requirements and deliver data solutions.

Krav

Azure Data Factory (ADF)
Azure Data Lake Storage Gen2 (ADLS)
Azure Databricks
Delta Lake
Unity Catalog
Medallion Architecture
Batch Processing
Streaming Data Processing
Delta Live Tables (DLT)
Auto Loader
ETL/ELT Frameworks
PySpark
Spark SQL
Python
Data Modelling
SQL
Power BI
Data Visualization
Semantic Models
CI/CD Pipelines
Azure DevOps
Git
Infrastructure Automation
Query Optimization
Spark Performance Tuning
Partitioning Strategies
Workload Management
Cost Optimization
Experience with large-scale Azure Data Platform implementations.
Strong understanding of Lakehouse Architecture and Delta Lake.
Experience working in Agile/Scrum environments.
Knowledge of data governance and security frameworks.

Önskade kvalifikationer

Microsoft Azure Data Engineering certifications are highly desirable.

Färdigheter

Azure Data FactoryAzure DatabricksADLS Gen2PySparkSpark SQLPythonDelta Live TablesAuto LoaderUnity CatalogPower BIAzure DevOpsGitSQLData ModellingETLELTBatch ProcessingStreaming Data ProcessingData VisualizationCI/CDQuery OptimizationCost Optimization

#Azure#Data Engineer#Data Lake#Lakehouse#ETL#ELT#PySpark#Databricks#Power BI#DevOps#Cloud#Cost Optimization

Om HCLTech

Supercharging progress through technology and innovation

HCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered around digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending September 2022 totaled $12.1 billion. To learn how we can supercharge progress for…

Se alla lediga jobb hos HCLTech Läs mer om HCLTech Webbplats