EY - GDS Consulting - AI And DATA -MS Fabric -Senior

Data engineering and platform delivery

• Build governed analytics solutions using OneLake, Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Real-Time Intelligence, notebooks, and Power BI semantic models.
• Develop reliable batch and streaming pipelines using Python, PySpark, SQL, Spark notebooks, eventstreams, KQL where relevant, and deployable engineering practices.
• Design and own scalable batch and streaming data products, selecting appropriate data models, processing patterns, quality controls, and service-level expectations.
• Troubleshoot complex performance, reliability, and cost issues; establish reusable engineering patterns and guide code reviews and delivery standards.
• Translate business requirements into reusable, tested data products with clear ownership, contracts, documentation, data-quality checks, and service-level expectations.

AI-ready data and native AI capabilities

• Use native Fabric AI capabilities—Copilot in Fabric, AI Skills, Fabric data agents, and Azure AI integrations—where they are suitable for governed analytics and AI experiences.
• Design AI-ready data products and implement production patterns for retrieval, semantic search, evaluation, safety, and reliable operation.
• Assess when a governed RAG or agent workflow is justified and implement the supporting ingestion, metadata, access-control, evaluation, and deployment foundations.
• Develop and test scoped system prompts, context-assembly patterns, agent skills, and approved MCP integrations with clear tool contracts, least-privilege access, input/output validation, and traceability.
• Partner with data scientists, analytics teams, security, and business stakeholders to select the right pattern: deterministic analytics, semantic layer, retrieval-augmented generation (RAG), agent workflow, or model-based solution.
• Ensure AI solutions have documented data sources, access controls, quality thresholds, evaluations, human oversight where needed, and clear release controls.

Governance, security, and engineering excellence

• Implement governance across Microsoft Purview, OneLake security, workspace/domain design, sensitivity labels, lineage, data quality, semantic models, and capacity optimization.
• Implement automated testing, source control, code reviews, CI/CD, release controls, monitoring, alerting, incident learning, and clear runbooks.
• Work in Agile teams and communicate progress, dependencies, risks, and design decisions clearly to technical and non-technical stakeholders.
• Lead technical delivery for a workstream, mentor engineers, and communicate design choices and delivery risks to client stakeholders.

Skills and attributes for success

• Hands-on Python, PySpark, SQL, Spark notebook engineering, Delta/Parquet data formats, Power BI semantic modelling, and DAX or Power Query knowledge.
• Microsoft Fabric capacity, workspace, deployment pipeline, Git integration, Azure DevOps, and CI/CD experience.
• Practical experience implementing or supporting RAG, semantic search, agent workflows, context engineering, system prompts, agent skills, MCP integrations, or LLM evaluation would be an added advantage.
• Strong analytical problem solving, written and verbal communication, and a consulting mindset grounded in measurable client outcomes.
• Experience with API-based integrations, data contracts, security-by-design, and cloud-native identity, networking, and secrets-management concepts.
• Ability to explain trade-offs between batch, streaming, SQL, Python, PySpark, platform-native AI, and external AI services.