Key Responsibilities 1. Data Pipeline Development Design, build, and optimize scalable data pipelines using Scala and Hadoop frameworks. Implement ETL processes for ingesting, transforming, and storing data from various sources. 2. Data Analysis and Query Optimization Write and optimize complex SQL queries for efficient data retrieval and transformation. Troubleshoot and resolve performance issues in queries and data workflows. 3. Big Data Ecosystem Management Manage Hadoop-based data infrastructure, including HDFS, Hive, and related components. Monitor system performance and optimize resource utilization in a distributed environment. 4. Collaboration and Problem Solving Work closely with data analysts, data scientists, and business stakeholders to understand requirements and translate them into technical solutions. Provide technical guidance and best practices for data engineering within the team. 5. Automation and Monitoring Develop scripts and tools to automate repetitive tasks and ...