Staff Data Engineer

The Staff Data Engineer designs, builds, and operates the data foundation and the evaluation infrastructure behind ServiceNow CRM Agentic AI. The role has two halves that reinforce each other. The first is conventional but demanding data engineering: architecture, pipelines, and transformation across structured and unstructured sources, held to production standards of reliability and quality. The second is newer and rarer: building the measurement layer that tells product and AI teams whether an AI agent actually did its job.

That second half changes the nature of the work. Agent behavior is probabilistic, so quality cannot be asserted, only measured—against metrics that have to be defined before they can be tracked, and against ground truth that someone has to establish and defend. This engineer defines those metrics with product and AI teams, sets the labeling and validation standards, builds the datasets that reflect how agents behave in the real world, and automates the pipelines and dashboards that turn agent execution logs into a signal the organization can act on.

the Staff Data Engineer owns these decisions across a domain rather than within a single project. They set design standards and quality gates others adopt, mentor junior engineers, and generalize evaluation patterns so each Agentic AI team is not rebuilding measurement from scratch.

What you do Design and architect data infrastructure

Design and oversee deployment of the data architecture and pipelines that capture, manage, and store structured and unstructured data from internal and external sources. Establish the processes and data flows across cloud services, local databases, and other applicable storage forms, and own the contracts between data producers and consumers.

Build and automate data transformation

Develop technical tools using machine learning and data-engineering techniques to cleanse, organize, and transform data. Implement automated processes that maintain the integrity of data structures and hold quality standards on an ongoing basis rather than at a point in time.

Define agentic evaluation metrics and ground truth

Partner with product and AI teams to define evaluation metrics for agentic workflows, including task and mission completeness, instruction adherence, tool use, and end-to-end workflow success. Establish ground truth labeling standards, annotation guidelines, and validation criteria, and design evaluation datasets that reflect real-world agent execution rather than idealized paths.

Build agentic evaluation pipelines

Design and implement automated evaluation infrastructure that measures AI agent performance using LLMs and agent execution logs. Create the dashboards, reporting, versioning, and reproducibility that make evaluation datasets and results trustworthy over time and comparable across releases.

Establish standards and continuous improvement

Create design standards and quality assurance processes for data systems. Define quality gates and validation frameworks, analyze workflow performance, and recommend optimizations that keep the platform aligned with evolving CRM AI requirements.

Lead cross-functional collaboration

Collaborate with product, engineering, and data science teams. Mentor junior engineers on data engineering and evaluation design, and generalize evaluation patterns and metrics so they can be reused across Agentic AI products rather than rebuilt per team.