Key Responsibilities

• Design and execute quality assurance strategies for LLM-based applications
• Test and validate RAG (Retrieval-Augmented Generation) pipelines built in Python
• Evaluate outputs and performance of OpenAI and Azure AI/ML models
• Build and maintain test automation frameworks for Agentic AI systems
• Identify, document, and track defects across AI/ML pipelines
• Collaborate with AI/ML engineering teams to define quality benchmarks

Requirements

• 12+ years of overall QA/engineering experience
• Hands-on experience testing LLM-based and Generative AI systems
• Strong Python skills for test automation and pipeline validation
• Experience with OpenAI and Azure AI/ML services
• Experience with RAG (Retrieval-Augmented Generation) pipelines
• Experience with Agentic AI systems and frameworks
• Must be local to Eden Prairie, MN and available to start ASAP

Skills Required