Data Engineer
avigna
Job Description
- Work with an AI-first mindset, actively identifying opportunities to integrate AI into processes, workflows, and solution design.
- Design and develop robust data pipelines for large-scale data ingestion, transformation, and analytics.
- Implement scalable Lakehouse architectures using tools like Microsoft Fabric for structured and semi-structured data.
- Develop custom KQL/SQL for both Structured and Unstructured queries and manage data using Power BI, Azure Cosmos DB, or similar tools.
- Work with Python, PySpark, and Azure services to support data modelling, automation, and predictive insights.
- Collaborate with cross-functional teams to integrate data-driven components with application backends and frontends.
- Ensure secure, efficient, and reliable CI/CD pipelines for automated deployments and data updates.