Lead Automation Engineer
lilly
Job Description
1) Automation build — from validated fix to production automation
- Build production-grade automation that executes validated remediation steps end to end — detection through action — with safe rollback built in.
- Take automation opportunities and known-fix candidates surfaced through discovery, telemetry, and alert correlation, and turn them into shipped, tested automations.
- Maintain and extend the existing automation portfolio (scripted orchestration, platform tooling, and, where appropriate, RPA) as new opportunities are approved.
- Working proficiency with Ansible, Terraform, or equivalent IaC/configuration management tooling for building safe, repeatable remediation and provisioning automation.
2) Runbook codification
- Translate SRE-authored remediation procedures into codified, executable automation steps: safe execution order, rollback steps, and exception handling.
- Partner with SRE to keep the runbook library and its automated counterparts in sync as remediation procedures evolve.
- Document each codified runbook clearly enough that another engineer, or an agent, can execute or extend it without relying on tribal knowledge.
3) Observability & instrumentation — first-class, not an afterthought
- Design and implement the metrics, logs, traces, and events that make every automation's behavior explainable in production — inputs, decisions, actions taken, rollbacks, outcomes.
- Define and maintain SLIs/SLOs for the automations you own, and wire them into dashboards and alerts that Operations and SRE actually use.
- Partner with the platform and alerting teams on signal correlation and alert tuning so automations trigger on clean, high-confidence signal rather than noise.
- Build the feedback loops that push automation outcome data into the knowledge base and confidence model driving what gets automated next.
4) SRE pattern implementation
- Apply SRE patterns in what you build: error-budget awareness, blameless-postmortem-driven fixes, and graduation-criteria-aware rollout.
- Implement confidence thresholds and staged autonomy — an automation earns broader scope only as it demonstrates accuracy over volume, per agreed graduation criteria.
- Build in the checks and evidence trail that let an automation prove it has caused zero P1/P2 incidents before it's trusted with more scope.
5) Cross-team partnership & autonomy graduation
- Partner with SRE to validate that an automation meets graduation criteria (accuracy over volume, zero P1/P2 caused) before it moves to a higher autonomy tier.
- Work with Operations to review outcome validation and incident feedback, closing the loop between what ran and what should run differently next time.
- Support ticket volume and shift flexibility as operational demand requires, in line with the team's protected build-time model.