Senior Apache Spark Technical Lead - Scala, Python

Technology, Data & Digital · Technology · Data Science · Data Engineering

Smart Summary

AI-generated overview of this position

Quantexa is seeking a Senior Apache Spark Technical Lead with hands-on experience in Scala/Spark, Hadoop, and data integration tools. The role involves leading technical teams in designing and implementing big data solutions, providing technical guidance, and ensuring best practices are followed.

Job Summary

Quantexa

Hands on development work on Scala/Spark, Hadoop, platform operations with Quantexa Certified Experience.

Data integration – ETL tools like IBM Datastage and Informatica. Ingestion mechanism like Flume & Kafka.
Data modelling – Dimensional modelling using RDBMS, NO-SQL and Big Data technologies.
Big data – Hadoop eco-system, Distributions like Cloudera / Hortonworks, Pig and HIVE.
Data processing frameworks – Scala/python/Spark.
Hands-on experience with databases like Oracle, MS SQL Server, NOSQL (HBase / Cassandra, MongoDB)
Data Platform administration for Hadoop, Elastic, Oracle.
Exposure to containers, kubernetes, application setup (load balancer), logging, Prometheus

Key Responsibilities

1. Lead technical teams in the design and implementation of solutions using apache spark, scala, and python
2. Provide technical expertise and guidance to team members in resolving complex technical issues
3. Collaborate with stakeholders to gather requirements and define project scope
4. Ensure adherence to best practices in coding, testing, and deployment processes
5. Conduct code reviews and performance optimization activities
6. Troubleshoot and debug technical issues to ensure seamless project delivery
7. Stay updated with the latest trends and advancements in apache spark, scala, and python technologies
8. Mentor team members and facilitate knowledge sharing within the team

Skill Requirements

1. Strong proficiency in apache spark, scala, and python
2. Experience in leading technical teams and projects
3. Excellent problem-solving skills and ability to think critically
4. Solid understanding of software development life cycle
5. Good communication and interpersonal skills
6. Ability to work effectively in a collaborative team environment
7. Strong organizational and leadership abilities
8. Experience in performance tuning and optimization techniques
9. Knowledge of big data concepts and tools is a plus

Other Requirements

1.Relevant certifications in apache spark, scala, or python are a plus

#Apache Spark#Scala#Python#Technical Lead#Big Data#Hadoop#ETL#Data Integration#Data Modelling#Cloud#Kubernetes