AI Platform

Build the AI platform: APIs, SDKs, and self-service workflows so AI/ML engineers deploy, version, evaluate, and monitor workloads without touching raw infrastructure.

Own GPU infrastructure: Cluster design, provisioning, scheduling, isolation, and utilisation optimisation across shared multi-team demand.

Run inference in production: vLLM, Triton, KServe, or Ray — continuous batching, autoscaling, model routing, and multi-model serving against real latency and throughput SLOs.

Optimize relentlessly: Drive tokens/sec, p99 latency, GPU utilization, and cost per token through quantization, batching strategy, and capacity planning.

Engineer GPU-aware Kubernetes: GPU Operator, device plugins, GPU-aware scheduling, MIG partitioning, and distributed workloads over NCCL.

Debug the hard layer: GPU OOMs, driver and CUDA runtime mismatches, interconnect bottlenecks, throughput regressions.

Build the MLOps backbone: Model registry and versioning, CI/CD for AI workloads, and safe rollout/rollback.

Own the evaluation layer: Offline and online eval harnesses, golden datasets, LLM-as-a-judge scoring, accuracy and hallucination metrics, and automated regression gates so no model, prompt, or quantisation change ships without a measured quality verdict.

Make it observable: Response quality and accuracy drift, latency, throughput, GPU utilisation, and per-tenant cost — with SLOs, alerting, and incident response.

Secure and multi-tenant by default: AuthN/AuthZ, secrets, data protection, tenant isolation, and governed access to models and GPUs.

Codify and lead: Terraform for everything, HA and DR designed in, plus architecture direction and mentoring across teams.

Contribute beyond AI: Design and build secure, multi-tenant microservices and APIs on AWS, run thorough code reviews, and own feature delivery end-to-end with Product and Design.