The Job

Senior Technical Lead

Technology, Data & Digital · Data, AI & Analytics · Data Engineering

Join for updates

In short

Join HCLTech as a Senior Technical Lead in Bengaluru, India, to design, build, and operate secure and cost-efficient data pipelines on AWS. This role focuses on batch, streaming, and CDC ingestion, embedding testing, governance, observability, and CI/CD.

Responsibilities

  • Implement pipelines using AWS Lambda, EMR for batch/streaming, managing schema evolution and backfills.
  • Build orchestration workflows with Step Functions and/or MWAA (Airflow), including retries, alerts, and SLAs.
  • Model datasets on S3 (Parquet/Iceberg), Glue Data Catalog, Athena, focusing on partitioning, compaction, and performance tuning.
  • Add automated data quality tests (Great Expectations/Deequ, PySpark unit tests), data contracts, and CI gates.
  • Implement security and governance features like IAM, KMS, VPC endpoints, Lake Formation policies, PII handling, audit trails, and RBAC/RLS.
  • Establish observability through CloudWatch metrics/alerts, lineage, usage dashboards, and implement FinOps for cost optimisation.
  • Provision infrastructure with Terraform/CloudFormation/CDK and build CI/CD pipelines with GitHub Actions.
  • Maintain pipeline diagrams, SLAs/SLOs, and incident playbooks.
  • Adopt capabilities from other teams and contribute back to the community.

Requirements

  • 10+ years of experience building data platforms on AWS.
  • Strong proficiency in Python (including PySpark) and SQL.
  • Hands-on experience with DMS, Kinesis/MSK, Glue/EMR, Lambda, Step Functions/MWAA, S3/Parquet/Iceberg, Glue Catalog, Athena, Redshift/Serverless.
  • Experience with IaC (Terraform) and CI/CD (GitHub Actions).
  • Familiarity with data quality frameworks (Great Expectations/Deequ), schema evolution, and backfills.
  • Knowledge of security and compliance aspects: IAM/KMS, private networking, Lake Formation, audit/lineage.
  • Experience with performance and cost tuning across storage and compute.

Desired Qualifications

  • Experience with Snowflake, Apache Flink, Redshift Spectrum, OpenLineage, or OPA/policy as code is a plus.

Benefits

  • Supercharge your potential at HCLTech.
  • Find your career and spark at a people-first company.
  • Global technology company with a broad portfolio of services and products.
  • Work with clients across major verticals including Financial Services, Manufacturing, Life Sciences, Healthcare, Technology, Telecom, Media, Retail, and Public Services.
#Data Engineering#AWS#Data Pipelines#Batch Processing#Streaming#CDC#ETL#Data Lakehouse#Data Warehouse#Data Quality#Data Governance#CI/CD#IaC#Cloud Computing#Observability#FinOps#Python#SQL#Spark#Big Data
HCLTech Logo

Company

HCLTech

Job Posted

1 month ago

Employment Type

Full Time

WorkMode

On Site

Experience Level

Senior

Locations

Bengaluru, India

Applicants

Be an early applicant