Senior DevOps Engineer

· Design, provision, configure, and maintain AWS Databricks environments across DEV, QA/UAT, and PROD using reusable Terraform modules and infrastructure-as-code standards.

· Configure AWS services required for Databricks, including VPCs, subnets, routing, security groups, IAM roles and policies, S3, KMS, CloudWatch, Secrets Manager, and VPC endpoints.

· Assess Azure Databricks platform configurations and map workspaces, compute, cluster policies, identities, permissions, storage, secrets, and networking to suitable AWS equivalents.

· Support migration of Databricks workspaces, jobs/workflows, repositories, libraries, Unity Catalog objects, storage credentials, external locations, volumes, grants, and service principals.

· Retrofit platform references from ADLS Gen2/abfss paths to Amazon S3 and migrate the approved secret-management and authentication patterns from Azure to AWS.

· Build and maintain basic CI/CD pipelines for Terraform and Databricks assets using Azure DevOps, GitHub Actions, Jenkins, GitLab CI, or equivalent tools.

· Implement SSO, SCIM, Unity Catalog access controls, logging, monitoring, auditability, and standard platform security controls.

· Support Databricks Runtime and library compatibility, Azure-to-AWS compute mapping, deployment promotion, cutover, rollback planning, production validation, and hypercare.

· Troubleshoot networking, IAM, authentication, authorization, cluster, workflow, Terraform, and deployment issues across Azure, AWS, and Databricks.

· Create deployment guides, architecture/configuration documents, operational runbooks, and knowledge-transfer material.

Role Description Project Delivery

· Collaborate with solution architects, data engineers, QA, BI, security, networking, and customer platform teams to complete environment readiness and migration activities.

· Participate in discovery, sprint planning, technical reviews, release planning, cutover activities, and Agile ceremonies; proactively track risks, dependencies, and blockers.

· Provide estimates, review Terraform and CI/CD changes, guide junior engineers, and communicate technical decisions clearly to customer and internal stakeholders.

· Coordinate required access and approvals and identify requirements outside standard platform setup that may need design review or change control.