Senior Administrator - Extract Transform Load (ETL), Data Lakes
Technology, Data & Digital · Data, AI & Analytics · Data Engineering · DevOps
In short
We are seeking a Senior Administrator with expertise in Extract Transform Load (ETL) and Data Lakes. The role involves designing, developing, and maintaining data integration workflows using Informatica IDMC and building ETL/ELT pipelines for Databricks Lakehouse, requiring proficiency in Linux and Python.
Responsibilities
- Translate business requirements and functional specifications into software solutions.
- Develop, configure, or modify integrated business and/or enterprise application solutions.
- Design, develop, and maintain data integration workflows using Informatica IDMC (Cloud Data Integration (Azure), Cloud Application Integration, etc.).
- Build and optimize ETL/ELT pipelines that ingest, transform, and load data into Databricks Lakehouse.
- Leverage DevOps, Waterfall, and Agile practices to design, develop, and deliver resilient, secure, multichannel, high-volume, high-transaction, on/off-premise, cloud-based solutions.
- Provide insight into recommendations for technical solutions that meet design and functional needs.
- Provide systems' incident support and troubleshooting for basic to moderately complex issues.
- Assist in identification of specific interfaces, methods, parameters, procedures, and functions required to support technical solutions.
- Support collaboration between team members, architects, and/or software consultants to ensure functional specifications are converted into flexible, scalable, and maintainable solution designs.
- Assist in translating business requirements and functional specifications into code modules and software solutions.
- Assist in the implementation and post-implementation triage and support of business software solutions.
- Develop and execute unit testing to identify application errors and ensure software solutions meet functional specifications.
- Support component integration testing (CIT) and user acceptance testing (UAT) for application initiatives.
- Assist in the development, configuration, or modification of integrated business and/or enterprise application solutions.
- Write technical specifications and documentation.
- Assist with efforts to ensure new and existing software solutions are developed with insight into industry best practices, strategies, and architectures.
- Assist in building partnerships with IT teams and vendors to ensure written code adheres to company architectural standards, design patterns, and technical specifications.
- Collaborate with data architects, analysts, and business stakeholders to understand data requirements and deliver robust solutions.
- Implement best practices for data governance, metadata management, and data quality using Informatica tools.
- Configure and manage connections between Informatica IICS/IDMC and Databricks, including secure authentication and data movement strategies.
- Monitor and troubleshoot data pipelines for performance, reliability, and scalability.
- Automate data workflows and orchestrate jobs using Informatica’s scheduling and event-driven capabilities.
- Document technical specifications, data flow diagrams, and operational procedures.
Requirements
- 1 year of Experience in Design, develop, and maintain data integration workflows using Informatica IDMC (Cloud Data Integration (Azure), Cloud Application Integration, etc.).
- Build and optimize ETL/ELT pipelines that ingest, transform, and load data into Databricks Lakehouse.
- Proficiency in Linux/Python scripting.
- Experience with Peta byte scale data ingestion, data migration.
- Ability to adopt various type of data formats and data sizes.
#ETL#Data Lakes#Informatica#Databricks#Python#Linux#DevOps#Agile#Data Integration#Data Engineering
Company
HCLTech
Job Posted
1 month ago
Employment Type
Full Time
WorkMode
On Site
Experience Level
Senior
Locations
India
Applicants
Be an early applicant