Senior Technical Lead
Technology, Data & Digital · Data, AI & Analytics · Data Engineering
In short
Responsibilities
- Design, develop, test, and maintain ETL/ELT data pipelines in production.
- Implement high-throughput ingestion of large multi-modal vehicle data.
- Ensure reliable, resumable, and restart-safe data transfers.
- Work with relational and non-relational databases using Python and SQL.
- Implement and maintain object storage and lakehouse solutions.
- Support immutable raw data storage and curated datasets.
- Implement data quality checks, including checksum validation, schema verification, and time-synchronization checks.
- Maintain data contracts, manifests, metadata, and searchable catalogs.
- Develop and maintain automated data workflows using orchestration tools such as Apache Airflow.
- Implement pipeline monitoring, observability, and alerting.
- Develop data-access APIs using FastAPI or Flask.
- Enable efficient access to multi-modal data for internal teams.
- Create and maintain technical documentation for data pipelines and related solutions.
- Support team collaboration and onboarding of new data sources.
- Work closely with engineering teams to support data ingestion, processing, and accessibility.
- Troubleshoot data pipeline issues and contribute to reliable data engineering solutions.
Requirements
- Design, develop, test, and maintain ETL/ELT data pipelines in production, covering vehicle-edge ingestion through Bronze, Silver, and Gold lakehouse layers.
- Implement high-throughput ingestion of large multi-modal vehicle data, including camera, LiDAR, and MCAP recordings.
- Ensure reliable, resumable, and restart-safe data transfers from vehicle-edge systems to centralized storage without unnecessary re-uploading.
- Work with relational and non-relational databases using Python and SQL.
- Implement and maintain object storage and lakehouse solutions using S3-compatible storage, MinIO, Azure ADLS, Apache Iceberg, or Delta Lake.
- Support immutable raw data storage and curated datasets.
- Implement data quality checks, including checksum validation, schema verification, and time-synchronization checks.
- Maintain data contracts, manifests, metadata, and searchable catalogs to ensure data integrity and discoverability.
- Develop and maintain automated data workflows using orchestration tools such as Apache Airflow.
- Implement pipeline monitoring, observability, and alerting to ensure pipeline health.
- Develop data-access APIs using FastAPI or Flask.
- Enable efficient access to multi-modal data for internal teams.
- Create and maintain technical documentation for data pipelines and related solutions.
- Support team collaboration and onboarding of new data sources.
- Work closely with engineering teams to support data ingestion, processing, and accessibility.
- Troubleshoot data pipeline issues and contribute to reliable data engineering solutions.
Benefits
- Consolidated revenues as of 12 months ending September 2026 totaled $14.8 billion.
Skills
HCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered around digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending September 2022 totaled $12.1 billion. To learn how we can supercharge progress for…
Company
HCLTechJob Posted
4 days ago
Employment Type
Full Time
Work mode
On Site
Experience Level
Senior
Locations
Bengaluru, India
Applicants
Be an early applicant
