What you will do
Maintain an evergrowing Datalake with a diverse set of data sources and data consumers;
Help implement data governance initiative;
Maintenance, deprecation, and eventual migration of legacy ETL jobs into a LakeHouse solution.
Must haves
Airflow experience;
Worked with Spark/Glue;
IaC experience; CDK or Terraform;
Familiar with Datalake/Datawarehouse/Lakehouse;
Worked with Iceberg/Delta;
Experience with AWS tools;
Experience with data visualization tools;
Experience with creating software for retrieving, parsing, and processing structured and unstructured data;
Experience with building scalable ETL and ELT workflows for reporting and analytics;
Experience with creating solutions in a collaborative, cross-functional team environment;
Ability to develop scripts and programs for converting various data types into usable formats, support project team to scale, and monitor and operate data platforms;
Ability ...