DevOps Engineer | SDE - 1 | SDE - 2
Role: QA Automation Engineer
Function: Quality Assurance
Location: Mumbai, India
Type: Full-time
Compensation: Competitive compensation and equity
Industry: AI/ML, Consumer Technology
About Company
A venture-backed AI company building the intelligence layer for India at massive scale. Backed by partnerships with global tech leaders like Meta and Google.
The team creates AI that serves the entire Indian user base across languages, contexts, and daily needs. Engineered from day one for 100M+ users early and 1B-ready constraints on latency, cost, reliability, and safety.
Position Overview
You'll build comprehensive testing frameworks for a Core Intelligence platform that powers AI workflows for millions of users across India. You'll design automation strategies that ensure quality and performance at massive scale, validating everything from product features to LLM outputs to multi-agent workflows. This role offers direct ownership of QA frameworks that will shape how AI systems operate reliably for 100M+ users.
Role & Responsibilities
- Design and implement comprehensive QA automation frameworks for AI/ML systems and large-scale distributed platforms
- Build automated testing suites using Playwright and PyTest for end-to-end validation of AI workflows
- Develop specialized testing frameworks for LLM validation, hallucination detection, and AI safety guardrails using LangSmith
- Create automated test scenarios for multi-agent coordination, task delegation, and partner integrations
- Implement CI/CD pipelines with automated quality gates for continuous deployment of AI features
- Build performance and load testing frameworks to validate 100M+ user scale constraints
- Collaborate with AI engineers to establish testing standards for model routing and orchestration systems
Must Have Criteria
- 5-10 years of hands-on QA automation experience with large-scale distributed systems
- Expert-level proficiency in Python automation frameworks, specifically PyTest
- Strong hands-on experience with Playwright for end-to-end web and mobile application testing
- Proven experience building and maintaining automated testing integration of CI/CD pipelines
- Experience with AI/ML testing frameworks and validation of machine learning model outputs
- Strong knowledge of JavaScript/TypeScript for React Native mobile app automation
- Experience testing APIs, microservices, and distributed system architectures
Nice to Have
- Experience with LangSmith, LangChain evaluation tools, or similar LLM testing frameworks
- Background in testing conversational AI, chatbots, or agent-based systems
- Experience with performance testing tools for high-scale consumer applications
- Knowledge of AI model evaluation metrics and automated quality assessment
- Previous experience in fintech, consumer tech, or AI-first product companies
What We Offer
- Opportunity to build QA frameworks for India's largest AI platform serving 100M+ users
- Work directly with cutting-edge AI technologies and multi-agent systems
- High-ownership environment with direct impact on product quality and user experience
- Competitive compensation and equity in a high-growth AI company
- Collaborative culture with top-tier AI engineers and product teams
FAQs
Are there any additional costs for payroll processing in multiple countries?
Throughout history, These artists have inspired countless others to explore the instrument and its diverse musical possibilities.
Mastering the Accordian rying ability.
The history of t, the Accordian has evolved, with various types emerging, including the piano accordian and the button accordian, each offering unique playing styles and sounds.
The Accordian is a versatile musical instrument that has been used in various genres, from folk to classical music. Its uniqu.
