Sr. Software Engineer

Design, develop, and maintain scalable data pipelines using Azure Databricks, PySpark, and Azure Data Factory (ADF).
Build and optimize ETL/ELT workflows for processing large-scale structured and unstructured datasets.
Develop data solutions using Delta Lake and Medallion Architecture (Bronze, Silver, Gold layers).
Integrate data from multiple sources including databases, APIs, cloud storage, and enterprise applications.
Implement data transformation, cleansing, validation, and quality frameworks.
Optimize Spark jobs, cluster configurations, and SQL queries to improve performance and reduce costs.
Develop and maintain data models supporting business intelligence, reporting, and analytics requirements.
Utilize Azure services such as ADLS Gen2, Azure Synapse Analytics, Azure SQL Database, Key Vault, and Azure Monitor.
Implement CI/CD pipelines using Azure DevOps and Git-based version control practices.
Ensure data security, governance, compliance, and access management using Unity Catalog and Azure security best practices.
Monitor production workloads, troubleshoot data issues, and perform root cause analysis.
Collaborate with business stakeholders, solution architects, data scientists, and development teams to deliver end-to-end data solutions.
Participate in Agile ceremonies, code reviews, design discussions, and technical mentoring of junior team members.
Support cloud migration and modernization initiatives from on-premises platforms to Azure.