Senior AI Engineer LLM and Agentic Workflows
workforcenow
Job Description
- Build RAG and tool-calling workflows for implementation assistance, connector capability discovery, error explanation, and guided migration operations.
- Create context assembly from approved documentation, run status, mapping metadata, and evidence. Return citations and structured outputs.
- Implement MCP clients/tool integrations and thin adapters over platform contracts. Handle async request IDs, retries, timeouts, and duplicate action prevention with the orchestration team.
- Implement permission-aware retrieval, approval checkpoints, prompt-injection defenses, bounded tool use, and explicit failure or abstention behavior.
- Capture agent traces and user feedback, and add regression tests for prompts, tool selection, retrieval quality, and task completion.
- Improve retrieval and context assembly using measured failures. Evaluate rerankers or model adaptation when simpler prompting and retrieval changes do not meet quality needs.
- Partner with Data Platform and product engineers, clinical/data SMEs, DME/QA, and security teams. Build reusable services for ELLKAY products and preserve source evidence, approved data-use rights, and tenant isolation. Model or agent proposals follow established human approval and platform release workflows.
Qualifications
- 5+ years in software/AI engineering, including production LLM applications and backend services. The total experience requirement does not imply five years in generative AI alone.
- Strong Python, REST APIs, asynchronous programming, data handling, software testing, and production debugging.
- Hands-on RAG, hybrid/vector search, embeddings, reranking, structured outputs, context management, and tool calling.
- Experience implementing agent state, timeouts, bounded retries, approval checkpoints, fallback, and duplicate-action prevention.
- Ability to evaluate retrieval relevance, groundedness, tool selection, task completion, latency, and unit cost using representative tests.
- Experience deploying services to a cloud environment and implementing authorization-aware retrieval and PHI-safe telemetry. A relevant degree or equivalent practical experience is acceptable.