Airflow Engineer
quest-global
Job Description
- Deploy and maintain Apache Airflow applications in AWS EKS clusters.
- Provide 24/7 support for Airflow consumers, ensuring SLAs are met.
- Monitor system performance and troubleshoot issues related to DAG execution, scheduler latency, or pod lifecycle.
- Implement and manage CI/CD pipelines for Airflow deployment.
- Collaborate with data engineering teams to onboard and support DAGs and orchestration logic.
- Maintain and manage integrations with AWS services like S3, RDS, Load Balancer, IAM, and CloudWatch.
- Work with Kubernetes manifests, Helm charts, and ServiceAccount configurations to ensure secure and scalable deployments.
- Create and maintain dashboards, alerts, and operational documentation.