Key Responsibilities

• Create active quality monitoring for data pipelines and processes
• Oversee quality engineering analysts in the execution of test scenarios, cases and coverage; define test plan and contribute to the strategy and resource estimates
• Responsible for adherence to quality metrics across the project lifecycle
• Work closely with data engineers, data scientists and analysts to inform solution designs in terms of testability
• Build and maintain testing frameworks for big data environments that are highly secure, scalable, flexible, and performant using appropriate SQL technologies
• Support data governance and data profiling efforts to ensure data quality and proper meta-data documentation for data lineage
• Create quality metrics to evaluate data pipelines, visualizations, and dashboards
• Design and develop integration, regression, and stress tests using industry-standard tools
• Collaborate with Product Management and Engineers to translate requirements into test cases and determine product quality goals and measurements
• Develops and enhances the test infrastructure and continuous integration framework used across teams
• Validate data pipelines and data processing jobs that collect data from disparate systems and store it in internal databases
• Identify edge cases that can potentially break the data pipelines or compromise data quality or integrity
• Manage and oversee a team of data quality engineers
• Proactively analyze data to answer key questions from stakeholders with an eye for what drives business performance

Requirements

• Experience with data pipelines, data quality monitoring, and testing strategies
• Knowledge of data governance and metadata documentation
• Strong SQL skills and experience with data ingestion pipelines
• Experience with big data environments and secure, scalable testing frameworks
• Ability to reproduce, isolate, and debug issues with detailed bug reports

Skills Required