Lead Data Engineer

Set and enforce design standards, patterns, and best practices for pipelines built on Databricks, Spark, Python, PySpark, and dbt.

Provide technical leadership and direction to a team of engineers, owning architecture and design reviews.

Lead the design of CI/CD (GitLab, Jenkins), DataOps, and data observability strategy for the team.

Own Databricks platform administration standards and deployment governance using Databricks Asset Bundles.

Drive resolution of complex technical challenges across pipelines and the platform.

Mentor and grow engineers, establishing a strong code review and quality culture.

Partner with product and stakeholders on roadmap, prioritization, and technical trade-offs.

Own the deployment, migration, and upgrade strategy for the team's data platforms.

Required Qualifications
Bachelor's or Master's degree in computer science or a related technical field.

8+ years of data engineering experience, including demonstrated technical leadership.

Expert hands-on experience with Databricks (including administration), Spark, Python, and PySpark.

Deep dbt/TDD and CI/CD (GitLab, Jenkins) expertise.

Proven ability to define engineering standards and lead a team's practices.

Strong architecture and platform ownership across observability, cost, and performance.

Excellent communication and stakeholder management skills.

Ideally you have experience in banking, e-commerce, credit cards or payment processing and exposure to both SaaS and premises-based architectures. In addition, you have a post-secondary degree in computer science, mathematics, or quantitative science.