Track Lead - PaaS Azure, Azure IaaS, Azure DevOps
Technology, Data & Digital · Data, AI & Analytics · Data Engineering · DevOps
In short
We are seeking an experienced Azure Databricks Platform Support & Data Engineer for a hybrid role. You will manage platform operations (50%), including monitoring, workflow support, and administration of Databricks, Unity Catalog, and Azure services. You will also contribute to data engineering (50%), focusing on data ingestions, PySpark/Spark SQL pipelines, and data asset development.
Responsibilities
- Monitor Databricks jobs, workflows, and scheduled processes to ensure operational status and SLA adherence.
- Proactively identify and resolve workflow failures, investigate production incidents, and perform root cause analysis.
- Manage Azure Databricks workspaces, configure and maintain Unity Catalog, including catalogs, schemas, tables, and external locations.
- Administer clusters, compute resources, and support job scheduling and workflow orchestration.
- Manage RBAC and Unity Catalog permissions, provision/deprovision user access, and manage service principals.
- Support Azure Data Lake Storage Gen2 and Synapse Analytics integrations, troubleshooting connectivity and API issues.
- Develop and enhance PySpark-based data pipelines and optimize Spark SQL processes.
- Build and maintain ingestion frameworks for various source systems, supporting batch and incremental loads.
- Create and maintain Delta Lake tables and managed/external Unity Catalog tables, supporting Bronze, Silver, and Gold architecture layers.
Requirements
- Strong expertise in Azure Databricks
- Strong expertise in Unity Catalog
- Strong expertise in PySpark
- Strong expertise in Spark SQL
- Strong expertise in Azure Data Lake
- Strong expertise in Synapse Analytics
- Strong expertise in access governance
- Strong expertise in production support
- Strong expertise in data pipeline development
Benefits
- Global technology company
- Opportunity to supercharge potential and find your career spark
- Focus on putting people first
- Work with industry-leading capabilities in digital, engineering, cloud, and AI
- Collaborate with clients across major verticals including Financial Services, Manufacturing, Life Sciences & Healthcare, Technology & Services, Telecom & Media, Retail & CPG, and Public Services.
#Azure#Databricks#PaaS#IaaS#DevOps#Platform Support#Data Engineering#PySpark#Spark SQL#Unity Catalog#Data Lake#Synapse Analytics
Company
HCLTech
Job Posted
3 weeks ago
Employment Type
Full Time
WorkMode
Hybrid
Experience Level
Senior
Locations
Noida, India
Applicants
Be an early applicant