Track Lead - Extract Transform Load (ETL), Data Lakes

Technology, Data & Digital · Data, AI & Analytics · Data Engineering

In short

We are seeking a Senior Data Engineer with expertise in Databricks, SQL, and data warehousing to lead the migration of an Oracle data warehouse to Databricks on the Allianz Data Platform. This role involves building and optimizing ETL/ELT pipelines, translating SQL logic, and ensuring data quality for downstream analytics and AI/ML use cases.

Responsibilities

Analyse the current Oracle data warehouse and design the target model in Databricks.
Build and optimise ETL/ELT pipelines in Databricks (Spark SQL, PySpark) to migrate data from Oracle to Databricks.
Translate Oracle SQL/PL-SQL logic into Databricks Spark SQL and PySpark, ensuring functional equivalence and performance optimisation.
Ensure data quality, performance, and reliability of the new Databricks-based data warehouse.
Collaborate closely with the Enterprise Architect, ADP platform team and business stakeholders.
Document migration patterns, data models, and operational procedures.

Requirements

Graduate or Postgraduate in Engineering or a Post graduate in non-engineering disciplines.
5-9 years of experience in ITIL process/application support/Deployment activities.
Basic knowledge on database.
Strong hands-on experience with Databricks (Spark SQL, PySpark, Delta Lake, jobs, clusters).
Proven background in data warehousing (dimensional modelling, ETL/ELT, performance tuning).
Solid experience with Oracle DWH (SQL, PL/SQL) and migrations from Oracle or similar legacy DWH platforms.
Experience integrating PostgreSQL as a data source into a central DWH/Lakehouse.
Ability to reverse-engineer existing ETL logic (e.g. SAS jobs) and translate it into Databricks pipelines.
Strong communication and stakeholder management skills in an international, multi-vendor setup.
Good understanding of how high-quality, well-modelled data underpins advanced analytics and AI/ML capabilities.

Desired Qualifications

Experience with modern data engineering practices and AI-enabling data platforms is highly desirable.

Benefits

Supercharge your potential.
Find your career.
Find your spark.

Skills

DatabricksSQLData WarehousingETLELTPySparkDelta LakeOraclePL/SQLPostgreSQLSAS

#ETL#Data Lakes#Databricks#Data Warehousing#SQL#PySpark#Oracle#PostgreSQL

About HCLTech

Supercharging progress through technology and innovation

HCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered around digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending September 2022 totaled $12.1 billion. To learn how we can supercharge progress for…

See all open jobs at HCLTech Read more about HCLTech Website