Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines for large-scale data processing
• Build and optimize big data solutions using Apache Spark and the Hadoop ecosystem
• Develop and optimize Hive tables, queries, and data models
• Process structured and unstructured data from multiple sources
• Optimize Spark jobs and Hive queries for high performance and scalability
• Ensure data quality, integrity, and consistency across data platforms
• Collaborate with Data Analysts, Data Scientists, and business stakeholders to deliver robust data solutions
• Troubleshoot and resolve data pipeline issues while implementing best practices for performance tuning and automation
• Document technical designs, workflows, and ETL processes

Requirements

• 6+ years of experience in data engineering or related field
• Strong expertise in Apache Spark, Hadoop, Hive, ETL, and SQL
• Experience designing and maintaining large-scale data pipelines
• Ability to optimize data processing for performance and scalability
• Good understanding of data quality, governance, and data modeling
• Excellent collaboration with cross-functional teams and stakeholders

Skills Required