Lead Data Engineer
Back to Jobs
USPGet Smart Job AI Coach in the appFree on iOS and Android Google
Google
Google
Google
Google

Lead Data Engineer
Location
Hyderabad, TEL, 500081, IND
Experience
Senior
Posted
Jul 18, 2026
Apply by
August 17, 2026
Applicants
0
Early applicantEasy applyFull-timeWork from Office
Job Description
Who is USP?
The U.S. Pharmacopeial Convention (USP) USP is an independent scientific organization that collaborates with the world's top experts in health and science to develop quality standards for medicines, dietary supplements, and food ingredients. USP's fundamental belief that Equity = Excellence manifests in our core value of Passion for Quality through our more than 1,100 talented professionals across five global locations to deliver the mission to strengthen the supply of safe, quality medicines and supplements worldwide.
Brief Job Overview
The Digital product engineering team at USP is seeking a Data Engineer with expertise in building robust data pipelines, managing large-scale data infrastructure, and enabling advanced analytics capabilities. This role is critical to supporting projects that align with our mission to protect patient safety and improve the health of people around the world. We are looking for a professional who understands the value of clean, accessible, and well-organized data, and who enjoys designing systems that empower data scientists, analysts, and business stakeholders to derive actionable insights. The ideal candidate will be passionate about data architecture, cloud technologies, and scalable engineering solutions that drive innovation and impact.
How will YOU create impact here at USP?
- Design and implement scalable data collection, storage, and processing pipelines to support enterprise-wide data needs.
- Implement and maintain data governance frameworks and data quality checks within data pipelines to ensure compliance and reliability.
- Build and optimize data models and data marts to support self-service analytics and reporting tools such as Tableau, Looker, and Power BI
- Partner with data scientists to operationalize models by integrating them into production-grade pipelines, ensuring scalability, performance, and maintainability.
- Collaborate with cross-functional stakeholders (business, product, analytics, and engineering teams) to translate business requirements into scalable data solutions and prioritize data initiatives.
- Provide technical leadership and architectural guidance for data platform design, ensuring alignment with enterprise standards and long-term data strategy.
- Lead design reviews, code reviews, and data architecture discussions to ensure best practices, reusability, and high-quality deliverables.
- Collaborate with platform, DevOps, and security teams to ensure secure, cost-effective, and scalable data infrastructure.
- Influence data roadmap and strategy by identifying opportunities for data platform enhancement, automation, and cost optimization.
Who USP is Looking For?
The successful candidate will have a demonstrated understanding of our mission, commitment to excellence through inclusive and equitable behaviors and practices, ability to quickly build credibility with stakeholders, along with the following competencies and experience:
Education
Bachelor’s degree in relevant field (e.g. Engineering, Analytics or Data Science, Computer Science, Statistics) or equivalent experience.
Experience
- 7+ years of experience in big data technologies such as Python, PySpark, and SQL for processing structured, semi-structured, and unstructured data.
- Strong experience with AWS data services including Redshift, S3, Glue, Lambda, EventBridge, Postgres, Neo4j (Azure/GCP equivalents such as ADLS, Synapse, ADF acceptable).
- Experience in building batch, micro-batch, and streaming pipelines (real-time / near real-time) using Lambda/Kappa architectures.
- Hands-on expertise in designing and delivering enterprise-scale data platforms, including data lakehouse, data warehouse, data lake, and data marts.
- Strong understanding and hands-on implementation of data modeling techniques including - Data Vault 2.0, Dimensional Modeling, Knowledge Graphs, One Big Table (OBT) approaches (Certification in at least one area preferred)
- Experience with medallion architecture and metadata-driven data pipeline frameworks.
- Strong expertise in data governance frameworks, including- Data discovery, Data quality, Data security, Hands-on experience with DQ tools such as Great Expectations, Pydantic, etc.
- Strong SQL and programming skills for data transformation, modeling, and analysis.
- Hands-on experience in building and maintaining complex ETL/ELT pipelines and managing day-to-day data operations.
- Experience with workflow orchestration tools such as Airflow (2+ years or equivalent).
- Knowledge of streaming/event-driven architectures and modern data processing patterns.
- Good understanding of dashboarding and visualization techniques with tools like Tableau, Power BI, or equivalent.
- Experience with Agile (SAFe) methodologies, CI/CD pipelines, and modern deployment practices for data platforms.
- Exposure to AI/ML concepts, with familiarity in Generative AI patterns (e.g., RAG, chunking techniques) as an added advantage.
Additional Desired Preferences
- Experience with scientific chemistry nomenclature or prior work experience in life sciences, chemistry, or hard sciences or degree in sciences
- Experience with pharmaceutical datasets and nomenclature
- Experience in developing Machine learning & Deep learning models; Familiarity with MLOps and deploying ML models in production environments.
- Ability to explain complex technical issues to a non-technical audience
Supervisory Responsibilities:
No
Benefits
USP provides you with the benefits you need to protect yourself and your family today and tomorrow. From company-paid time off, comprehensive healthcare options to retirement savings, you can have peace of mind that your personal and financial wellbeing is protected.
Note: USP does not accept unsolicited resumes from 3rd party recruitment agencies and is not responsible for fees from recruiters or other agencies except under specific written agreement with USP.
#
Key Responsibilities
- Design and implement scalable data collection, storage, and processing pipelines.
- Implement and maintain data governance frameworks and data quality checks.
- Build and optimize data models and data marts for self-service analytics.
- Partner with data scientists to operationalize models in production-grade pipelines.
- Collaborate with cross-functional stakeholders to translate business requirements into data solutions.
- Provide technical leadership and architectural guidance for data platform design.
- Lead design reviews, code reviews, and data architecture discussions.
- Collaborate with platform, DevOps, and security teams for secure infrastructure.
- Influence data roadmap and strategy by identifying opportunities for enhancement and cost optimization.
Requirements
- Bachelor’s degree in Engineering
- Analytics
- Data Science
- Computer Science
- Statistics
- or equivalent experience
Skills Required
PythonPySparkSQLAWSRedshiftS3GlueLambdaEventBridgePostgresNeo4jData Vault 2.0Dimensional ModelingKnowledge GraphsOne Big TableMedallion architectureData governanceGreat ExpectationsPydanticETL/ELTAirflowTableauPower BIAgileSAFeCI/CDTechnical leadershipCollaborationCommunicationProblem solvingGenerative AIRAGChunking techniquesMachine LearningDeep LearningMLOpsAzureGCPADLSSynapseADFScientific chemistry nomenclatureLife sciencesPharmaceutical datasetsAbility to explain complex technical issues to non-technical audiences
Benefits
- Company-paid time off
- Comprehensive healthcare options
- Retirement savings
App exclusive · Free
Smart Job AI Coach
Your personal interview coach on every job — readiness tips, profile improvements, and role-specific prep. Available only in the Pulse Job app.
Interview readiness
See how prepared you are and what to improve for each role.
Personalized tips
Actionable suggestions based on your profile and the job.
After you apply
Keep coaching momentum from job detail through application success.
Similar roles for you
Matched using this role's title and skills. Open the job search anytime to see every listing.
Software Engineer III, AI/ML, gUP Customer Support
147,000–211,000 / Year
Full-timeEasy applyWork from Office
SunnyvaleMid
Staff Software Engineer, Health Data Foundations and Analytics
480 000–492 000 / Year
Full-timeEasy applyWork from Office
WarsawSenior
Python Fullstack Lead
Ehub Global Inc
$70–$90 / Hour
ContractEasy applyHybrid
Old Bridge
Senior Software Engineer, AI/ML
Full-timeEasy applyWork from Office
HyderabadSenior1 applicant
Senior Software Engineer, AI/ML GenAI, Google Cloud
174,000–253,000 / Year
Full-timeEasy applyWork from Office
SunnyvaleSenior
Senior Software Engineer, Infrastructure, Google Cloud Storage
174,000–253,000 / Year
Full-timeEasy applyWork from Office
New YorkSenior