AI And DATA
ey
Job Description
Data engineering and platform delivery
- Design and build secure, performant data products on Snowflake using warehouses, dynamic tables, Snowpark, streams, tasks, Snowpipe, and domain-aligned data modelling.
- Implement data ingestion, transformation, orchestration, testing, and performance optimization using SQL, Python, Snowpark, dbt or comparable engineering patterns.
- Design and own scalable batch and streaming data products, selecting appropriate data models, processing patterns, quality controls, and service-level expectations.
- Troubleshoot complex performance, reliability, and cost issues; establish reusable engineering patterns and guide code reviews and delivery standards.
- Translate business requirements into reusable, tested data products with clear ownership, contracts, documentation, data-quality checks, and service-level expectations.
AI-ready data and native AI capabilities
- Use Snowflake Cortex capabilities—such as Cortex AI functions, Cortex Analyst, Cortex Search, Cortex Agents, and Snowflake Intelligence—where they provide a governed native AI path.
- Design AI-ready data products and implement production patterns for retrieval, semantic search, evaluation, safety, and reliable operation.
- Assess when a governed RAG or agent workflow is justified and implement the supporting ingestion, metadata, access-control, evaluation, and deployment foundations.
- Develop and test scoped system prompts, context-assembly patterns, agent skills, and approved MCP integrations with clear tool contracts, least-privilege access, input/output validation, and traceability.
- Partner with data scientists, analytics teams, security, and business stakeholders to select the right pattern: deterministic analytics, semantic layer, retrieval-augmented generation (RAG), agent workflow, or model-based solution.
- Ensure AI solutions have documented data sources, access controls, quality thresholds, evaluations, human oversight where needed, and clear release controls.
Governance, security, and engineering excellence
- Apply Snowflake governance, Horizon Catalog / lineage capabilities, RBAC, masking policies, row access policies, data quality controls, and cost management.
- Implement automated testing, source control, code reviews, CI/CD, release controls, monitoring, alerting, incident learning, and clear runbooks.
- Work in Agile teams and communicate progress, dependencies, risks, and design decisions clearly to technical and non-technical stakeholders.
- Lead technical delivery for a workstream, mentor engineers, and communicate design choices and delivery risks to client stakeholders.
Skills and attributes for success
- Advanced SQL plus Python; Snowpark and PySpark experience are required, including the ability to compare appropriate execution engines and integration patterns.
- Data modelling, query profiling, warehouse sizing, resource monitors, CI/CD, Git, and deployment automation.
- Practical experience implementing or supporting RAG, semantic search, agent workflows, context engineering, system prompts, agent skills, MCP integrations, or LLM evaluation would be an added advantage.
- Strong analytical problem solving, written and verbal communication, and a consulting mindset grounded in measurable client outcomes.
- Experience with API-based integrations, data contracts, security-by-design, and cloud-native identity, networking, and secrets-management concepts.
- Ability to explain trade-offs between batch, streaming, SQL, Python, PySpark, platform-native AI, and external AI services.