Job Description

Must Have Technical/Functional Skills

Associates with 7-10 years' experience in data/ETL testing with hands-on PySpark and exposure to big data tools.



Ability to perform below responsibilities:



1. Design scalable PySpark-based test architectures for ETL/data pipelines, including modular frameworks for batch processing.

2. Architect end-to-end data validation systems in Hadoop environment for lineage, schema evolution

3. Lead system design for Hadoop/Hive test environments, including YARN resource management, dynamic partitioning.

4. Exposure to Zephyr-Jira-ServiceNow integrated test management systems with experience on API-driven automation.

5. Design CI/CD test pipelines for PySpark/Hadoop jobs, incorporating artifact management, parallel execution, and blue-green deployments.

6. Create data quality system designs using PySpark integrated with Hive metadata servi...