Senior Data Engineer

- Design and implement enterprise-grade Databricks Lakehouse architectures using Delta Lake and Unity Catalog.

- Build scalable batch and real-time data ingestion pipelines using Lakeflow Connect, SDP, Auto Loader, Spark, and Kafka.

- Design and implement CDC architectures using Debezium, Kafka/Kafka Connect, and relational databases such as PostgreSQL, MySQL, SQL Server, and Oracle.

- Implement streaming and event-driven data pipelines using Kafka, Spark Structured Streaming, and related technologies.

- Design and manage schema evolution and data contracts using Karapace / Schema Registry.

- Implement centralized governance using Unity Catalog, including catalogs, schemas, RBAC, row/column-level security, lineage, and data access policies.

- Develop metadata-driven ingestion frameworks, data quality, reconciliation, profiling, and observability solutions.

- Design Bronze, Silver, and Gold data layers and appropriate data modeling strategies for analytical workloads.

- Establish engineering best practices covering CI/CD, testing, deployment, monitoring, logging, and operational support.

- Use Databricks Asset Bundles (DAB) and CI/CD tools such as Jenkins/GitHub Actions for automated deployment.

- Work with cloud services such as AWS S3, IAM, networking, monitoring, and security services.

- Lead technical discussions with clients, translate business requirements into technical solutions, and drive architecture decisions.

- Troubleshoot complex data engineering, CDC, streaming, performance, and production issues.

- Mentor engineers and provide technical direction across data engineering initiatives.