Data Engineer

• Build, test, and maintain production data pipelines across Clinical and Non-Clinical domains — ingestion, transformation, and publication into governed lakehouse layers (Bronze/Silver/Gold).
• Implement pipeline patterns defined by senior engineers and the architect — incremental loads, CDC, schema handling, and idempotent reprocessing — and extend them to new sources.
• Monitor and troubleshoot pipeline runs, resolve data quality and performance issues, and improve reliability against agreed SLAs.
• Contribute to curated, analytics-ready data products with documented schemas and contracts that downstream consumers and agentic systems depend on.

Agentic AI & LLM Application Development (Hands-On)

• Build agentic AI components against established patterns — tool-calling agents, retrieval-augmented generation (RAG) chains, and steps within multi-agent workflows — for Clinical and Non-Clinical use cases.
• Implement prompt, tool, and state-handling logic with attention to token/cost and latency budgets, retries, and graceful failure handling.
• Write and run evaluations, guardrail checks, and regression tests for agentic components, and act on the results to improve accuracy and groundedness.
• Instrument agents with logging, tracing, and observability so behaviour is traceable and auditable in a regulated environment.

Right Model, Right Task — Contributing to Data Engineering

• Implement and tune routing configurations that direct each agentic task to the appropriate model on complexity, cost, latency, and accuracy grounds, rather than defaulting to one model for every job.
• Wire agentic components into Lilly's internal data platform (Data Hub catalog, lineage, and metadata services) rather than building parallel metadata stores.
• Contribute to lineage-aware knowledge graph pipelines that capture data provenance, sensitivity, and domain context for agent reasoning.
• Apply routing and access policy correctly in code — ensuring regulatory-sensitive Clinical data is only handled by approved/validated models.
• Run benchmarks and cost/accuracy comparisons across candidate models and share the findings with the squad.

Semantic, Retrieval & Knowledge Graph Engineering

• Build and tune retrieval components — embeddings, chunking strategies, vector indexes, and hybrid search — that ground LLMs in governed Clinical and Non-Clinical data.
• Contribute to semantic models, taxonomies, and knowledge graph structures under the guidance of senior engineers and the architect.
• Measure retrieval quality against defined benchmarks such as groundedness, recall, and citation accuracy, and iterate on the results.

Automation, Reusability & AI-Native Ways of Working

• Use AI-assisted and agentic tooling by default across the data lifecycle — pipeline generation, schema alignment, data-quality scoring, test generation, and documentation.
• Adopt and extend the team's reusable accelerators — agent templates, orchestration patterns, evaluation harnesses, and pipeline frameworks — and contribute improvements back.
• Automate repetitive checks such as lineage capture, schema-drift detection, and data quality validation instead of handling them manually.

Quality, Governance & Responsible AI

• Apply access control, encryption, and prompt/data-leakage safeguards correctly when building on regulated data.
• Build data quality, lineage, and compliance checks into pipelines and agentic components as part of the delivery, not afterwards.
• Follow engineering discipline consistently: version control, code review, automated testing, and CI/CD practices appropriate to a GxP environment.

Collaboration & Knowledge Sharing

• Work with senior engineers, the data architect, business SMEs, and platform teams to turn Clinical and Non-Clinical requirements into working solutions.
• Ask questions early, raise blockers and risks promptly, and seek review on design decisions before building.
  • Share learnings, document what is built, and support teammates as the squad scales its AI-native delivery.