Sr. Test Engineer
yash
Job Description
- Designs, builds, and executes rigorous automated test strategies for foundational services, including vector databases, knowledge graphs, APIs, and LLM-based applications.
- Validates data integrity, retrieval accuracy, semantic relevance, and latency performance of vector search and knowledge graph query layers (e.g., Cypher, SPARQL, semantic matching).
- Performs comprehensive functional, integration, regression, and performance/load testing on AI and data platforms, including Dataiku workflows and multi-agent orchestration frameworks.
- Tests multi-tenant security boundaries, Role-Based Access Control (RBAC), and authentication mechanisms across cloud-managed and self-managed backends.
- Collaborates closely with software engineers and platform developers to identify edge cases, trace root causes of failures, and establish automated CI/CD quality gates.
- Maintains test environments, generates synthetic test data for complex graph and vector structures, and establishes automated monitoring for test suites.
- Develops and documents QA standards, test scenarios, and validation scripts aligned with enterprise governance and data compliance requirements.
- Engages in performance profiling, stress testing under concurrent user loads, and validating system recovery and failover behaviour.
Technology:
- Graph Databases: Neo4j (Enterprise, Cypher, Causal Clustering) and exploration of alternative graph options.
- Testing Frameworks & Tools: PyTest, Selenium/Playwright, RestAssured, Postman, JMeter / k6 for load testing.
- Data & AI Stack: Vector Databases (e.g., Milvus, Qdrant, Pinecone), Graph Databases (e.g., Neo4j), LLM Orchestration (LangChain, LlamaIndex), and Dataiku. · Cloud & Infrastructure: Microsoft Azure, Docker, Kubernetes, CI/CD pipelines (GitHub Actions, Azure DevOps).
- Programming & Scripting: Python or Java for test automation script development.
- Cloud Infrastructure: Microsoft Azure (Azure Marketplace, AKS, Azure VMs, Managed Disks, Key Vault, VNet peering).
Previous Experience and Competencies:
- Bachelor’s or Master’s degree in Computer Science, IT, or a related discipline. · 6 – 10 years of strong QA engineering experience with a heavy focus on backend services, APIs, and data platforms.
- Proven experience testing modern AI/ML workloads, vector search systems, or graph-based data architectures.
Preferred Qualifications:
- Deep hands-on experience writing robust automation scripts in Python.
- Familiarity with testing non-deterministic outputs of Large Language Models (LLMs) and evaluating Retrieval-Augmented Generation (RAG) pipeline quality.
- Systematic problem-solving approach, strong analytical mindset, and acute attention to detail.
- Experience working within platform engineering or foundational infrastructure teams.