Senior Data Engineer
dsv
Job Description
Design and operate the real-time replication flow using Qlik Replicate, with multiple RDBMS as sources and Databricks as a main target.
• Build and maintain CI/CD and DevOps pipelines for data workloads using Azure DevOps, with automation and repeatable deployments as a priority.
• Integrate core Azure services — Data Factory, Synapse, Azure Functions, Key Vault, and ADLS Gen2.
• Implement data governance, security, and access control, including encryption and PII protection across ingestion and storage.
• Apply strong Databricks best practices — Delta Lake, Unity Catalog, cluster and job optimization, medallion architecture, and performance and cost tuning.
• Build a Data Quality framework across the data ingestion process.
• Monitor, troubleshoot, and tune replication latency, pipeline performance, and platform health.
• Provide technical leadership, mentor junior and mid-level engineers, and maintain clear documentation of architecture and processes.