We are looking for someone who has validated AI systems in production - where outputs are non-deterministic, failure modes are ambiguous, and correctness is probabilistic. You will be responsible for evaluating, hardening, and governing GenAI and agentic systems before and after they go live. This role sits at the intersection of engineering, QA, and AI safety, ensuring that what we build works consistently, reliably, and safely - at scale. You will partner closely with AI engineers, architects, and clients to define what “correct” means in an AI world — and prove it. We are looking for someone who has validated AI systems in production - where outputs are non-deterministic, failure modes are ambiguous, and correctness is probabilistic. You will be responsible for evaluating, hardening, and governing GenAI and agentic systems before and after they go live. This role sits at the intersection of engineering, QA, and AI safety, ensuring that what we build works consistently, reliably, and safely - at scale. You will partner closely with AI engineers, architects, and clients to define what “correct” means in an AI world — and prove it. ### Responsibilities AI Quality Engineering & Evaluation - Define evaluation frameworks for LLMs, RAG pipelines, and multi-agent systems - Design test strategies for non-deterministic systems (semantic correctness, hallucination detection, consistency checks) - Build testing pipelines for LLM evaluation (offline + online evals) - Establish ground truth datasets, benchmarks, and scoring metrics (precision, recall, relevance, factuality) Agent & Workflow Validation - Validate multi-agent orchestration flows. - Test failure modes: hallucination, incomplete reasoning. - Simulate edge cases and adversarial inputs. - Ensure robustness across multi-step workflows and chained reasoning tasks RAG & Data Validation - Validate end-to-end RAG pipelines: - Chunking quality - Embedding correctness - Retrieval relevance - Re-ranking effectiveness - Detect and quantify RAG failure points (retrieval gaps, stale data, hallucinations) - Ensure data lineage and traceability in AI responses Guardrails, Safety & Governance - Ensure compliance with enterprise AI governance and auditability requirements - Validate explainability and traceability of AI outputs Automation & Tooling - Automate prompt testing, regression testing, and response comparison - Integrate AI validation into CI/CD pipelines ### Qualifications - Built or contributed to evaluation frameworks for LLM-based systems - Tested RAG pipelines end-to-end and identified failure points - Defined and executed non-deterministic test strategies - Automated LLM evaluation or prompt regression pipelines - Worked on agent-based or multi-step AI workflows - Debugged incorrect or hallucinated model outputs using structured methods - Established quality metrics where ground truth was unclear or evolving ### About the Company At Zensar, we’re “experience-led everything”. We are committed to conceptualizing, designing, engineering, marketing, and managing digital solutions and experiences for over 130 leading enterprises. We are a company driven by a bold purpose: Together, we shape experiences for better futures. Whether for our clients, our people, or the world around us, this belief powers everything we do. At the heart of our culture is ONE with Client - a set of four core values that reflect who we are and how we work: One Zensar, Nurturing, Empowering, and Client Focus. Part of the $4.8 billion RPG Group, we’re a community of 10,000+ innovators across 30+ global locations, including Milpitas, Seattle, Princeton, Cape Town, London, Zurich, Singapore, and Mexico City. Explore [Life at Zensar](https://www.zensar.com/careers/) and join us to [Grow. Own. Achieve. Learn.](https://www.youtube.com/embed/i2NZsiQqVnU?autoplay=1&fs=1) to be the best version of yourself. We believe the best work happens when individuality is celebrated, growth is encouraged, and well-being is prioritized. We are an equal employment opportunity (EEO) and affirmative action employer, committed to creating an inclusive workplace. All qualified applicants will be considered without regard to race, creed, color, ancestry, religion, sex, national origin, citizenship, age, sexual orientation, gender identity, disability, marital status, family medical leave status, or protected veteran status.