Mandatory skills: GCP Composer (Apache Airflow)



Job Description:

• Fix and enhance Dataflow and/or Dataproc Data ingestion pipelines (batch & Streaming) .

• Have Good Understanding of GCP Composer (Apache Airflow) environments.

• Manage Airflow DAGs and troubleshoot Airflow-related issues.

• Monitor and optimize Composer performance and scalability.

• Maintain GCP infrastructure including services like:

o Google Cloud Storage (GCS)

o BigQuery (BQ)

o Cloud Dataflow ( Apache Beam )

o Cloud Dataproc

o Cloud Functions

o Cloud Run

• Have Good Understanding of:

o IAM and service account permissions

o Secret Manager

• Monitor GCP usage and quotas, and proactively prevent quota-related issues.

• Set up and manage aler...