Senior Technical Architect

Technology, Data & Digital · Data Science · Data Engineering · Cloud Engineering

Smart sammanfattning

AI-genererad översikt av denna tjänst

HCLTech seeks a Senior Technical Architect in India to design and build robust data infrastructure for AI/ML systems, focusing on batch and streaming pipelines, data quality, and multi-cloud platforms like Databricks and Snowflake. The role involves developing ETL pipelines using Spark, Airflow, and Dagster, implementing real-time streaming architectures, and managing enterprise data architecture for AI initiatives.

Job Summary

Designing and building data infrastructure for AI/ML systems: batch and streaming pipelines, data quality, metadata management, feature stores, and cloud data platforms. ETL pipeline development (Spark, Airflow, Dagster). Real time streaming architectures and feature store implementation. Data quality frameworks, schema evolution, and data contracts. Multi-cloud data platform design (Databricks, Snowflake, BigQuery). Pipeline orchestration at scale and performance tuning. Enterprise data architecture for AI: data mesh/fabric strategies, cross-platform data lineage and governance, FinOps. Junior: ETL pipelines, basic cloud data services (ADF, Glue, Dataflow), SQL/NoSQL. Senior: streaming architectures, feature stores, multi-cloud platforms, schema evolution. Expert: enterprise data architecture for AI, data mesh/fabric, FinOps, lineage and governance.

Key Responsibilities

Designing and building data infrastructure for AI/ML systems: batch and streaming pipelines, data quality, metadata management, feature stores, and cloud data platforms. ETL pipeline development (Spark, Airflow, Dagster). Real time streaming architectures and feature store implementation. Data quality frameworks, schema evolution, and data contracts. Multi-cloud data platform design (Databricks, Snowflake, BigQuery). Pipeline orchestration at scale and performance tuning. Enterprise data architecture for AI: data mesh/fabric strategies, cross-platform data lineage and governance, FinOps. Junior: ETL pipelines, basic cloud data services (ADF, Glue, Dataflow), SQL/NoSQL. Senior: streaming architectures, feature stores, multi-cloud platforms, schema evolution. Expert: enterprise data architecture for AI, data mesh/fabric, FinOps, lineage and governance.

Skill Requirements

null

Other Requirements

null

#Technical Architect#Data Infrastructure#AI/ML#Pipelines#Cloud#Databricks#Snowflake#BigQuery#Spark#Airflow#Dagster#Data Quality#Feature Stores#Streaming