Expert Data Engineer

Architecture & Strategy

• Design and implement end-to-end Microsoft Fabric architecture using the Medallion data model (Bronze, Silver, Gold).
• Define standards and best practices for data ingestion, curation, and consumption within Microsoft Fabric.
• Establish a governed, metadata-driven approach for lineage, observability, and data quality.
• Partner with business stakeholders to align architecture with analytics, BI, and AI/ML use cases.

Data Ingestion & Transformation

• Architect data pipelines for batch and streaming ingestion into Fabric (from ERP, CRM, APIs, flat files, and external data).
• Implement Bronze layer for raw/landing data, Silver layer for cleansed and conformed data, and Gold layer for business-ready curated data.
• Optimize dataflows, pipelines, and notebooks within Microsoft Fabric for performance and scalability.
• Ensure reusability of transformation logic and conformed dimensions for enterprise-wide reporting

Governance & Security

• Define frameworks for data governance, security, and compliance across Fabric workspaces.
• Implement role-based access control (RBAC), data masking, and sensitivity labels for regulatory compliance (GDPR, HIPAA, etc.).
• Set up data cataloging, lineage, and quality monitoring using Fabric’s built-in capabilities and Microsoft Purview.

Technology & Tools

• Leverage Fabric’s components:
  • Data Factory – for orchestration & pipelines
  • Data Engineering – for Spark notebooks & transformations
  • Data Science – for ML pipeline integration
  • Real-Time Analytics – for streaming data ingestion & Synapse Data Warehouse (Lakehouse & SQL Endpoint) – for modeling curated data
  • Power BI – for dashboards, semantic models, and self-service analytics
  • Integrate Fabric with Microsoft Purview, Azure Active Directory, and external data platforms.

Collaboration & Leadership

• Work closely with data engineers, BI developers, and business analysts to ensure data products meet business requirements.
• Provide technical leadership and mentorship on Microsoft Fabric adoption.
  • Act as the subject matter expert (SME) for Medallion architecture implementation in Fabric.