Sr Software Engineer Syseng
lplfinancial
Job Description
Observability & Monitoring
- Design, implement, and operate enterprise observability solutions using Dynatrace, including logs, metrics, traces, and RUM.
- Define golden signals, SLOs, SLIs, and error budgets for critical applications and infrastructure.
- Create dynamic and adaptive alerting strategies to reduce noise and improve signal quality.
- Partner with application teams to embed reliability and observability standards into services from design through production.
AI Agents & Intelligent Automation
- Build and deploy AI/ML-powered agents that analyze:
- Dynatrace logs and metrics
- Historical incident data
- Behavioral and anomaly patterns
- Implement predictive monitoring and anomaly detection using statistical models or ML techniques.
- Automate alert triage, correlation, and root cause analysis (RCA) using AI-driven approaches.
- Continuously improve models based on operational feedback and outcomes.
Self-Healing & Recovery
- Design and implement self-healing systems that automatically:
- Restart or rescale services
- Redirect traffic
- Roll back failed deployments
- Trigger remediation workflows
- Integrate automated recovery actions with CI/CD pipelines, runbooks, and orchestration tools.
- Ensure recovery mechanisms are safe, auditable, and compliant with enterprise governance.
Reliability Engineering & Operations
- Lead incident response, post-incident reviews, and root cause analysis.
- Establish runbooks, automated remediation playbooks, and operational standards.
- Drive reductions in MTTR, incident frequency, and operational toil.
- Participate in on-call rotations and help build a sustainable on-call culture.
Platform, Cloud & Infrastructure
- Work with cloud and onprem infrastructure teams to improve reliability at scale.
- Build automation using Infrastructure as Code (IaC) practices.
- Ensure systems meet security, compliance, and resiliency requirements expected in financial or regulated environments.