Key Responsibilities

• Design and implement comprehensive test architectures for AI agent systems
• Define evaluation frameworks for LLM outputs, agentic workflows, and multi-agent pipelines
• Build automated testing suites for AI agents including unit, integration, and regression tests
• Develop adversarial testing and red-teaming strategies for AI safety validation
• Collaborate with AI architects and engineers to establish quality gates and deployment criteria
• Establish observability and monitoring standards for AI agents in production

Requirements

• 6+ years of software testing/QA architecture experience
• 2+ years working with LLM-based or AI agent systems
• Experience with evaluation frameworks such as RAGAS, TruLens, or custom LLM eval pipelines
• Proficiency in Python and test automation tools (pytest, Selenium, etc.)
• Familiarity with agent frameworks like LangGraph, AutoGen, CrewAI, or similar
• Understanding of AI safety, hallucination detection, and responsible AI principles
• Strong analytical and problem-solving skills

Skills Required