SDET 3 - Model Evaluation

Overview

Own end-to-end quality for the ML models behind AiDASH's products, from training data preparation through inference pipelines to production-scale performance, as a senior QA engineer on the Platform ML team.

What you'll do

• Own model quality before production by building validation frameworks for computer vision models (preferably satellite or geospatial).
• Catch regressions in accuracy, precision/recall, and IoU before they reach customers.
• Set up version-aware model regression pipelines that compare inference outputs across model versions on curated golden datasets.
• Deliver a clear, data-backed go/no-go signal for every release.
• Validate training and evaluation datasets for label quality, class balance, geographic and seasonal coverage, leakage, and drift.
• Design load and benchmark tests for inference services and batch pipelines to find throughput, latency, and cost limits on AWS and Kubernetes.
• Automate API, contract, and data-pipeline tests across distributed services.
• Use logs, metrics, and traces to speed up root-cause analysis.
• Strengthen shared automation frameworks and set standards for maintainable, non-flaky tests.
• Show how tools like Cursor and Claude can speed up test authoring, triage, and reporting while staying under human review.
• Influence release decisions across AiDASH's product lines.

What you'll need

• 8–10 years in software testing and test automation, with strong ownership of backend system quality.
• Hands-on backend automation experience using Java + RestAssured or Python + pytest.
• Experience testing ML applications, including model/service behavior, data quality, inference pipelines, and evaluation/regression of model-backed features.
• Strong coding ability in Java or Python (proficient in at least one), with willingness to learn the other or additional languages as needed.
• Solid grasp of testing fundamentals and best practices: test design, risk-based coverage, isolation, maintainability, flaky-test control, and clear defect/RCA discipline.
• Good understanding of backend system design and how services interact in distributed systems.
• Practical exposure to databases, SQL and/or NoSQL (at least one is mandatory).
• Working knowledge of application deployment and orchestration (e.g., containers, Kubernetes, service rollout/health concepts).
• CI/CD pipeline experience (Jenkins/GitHub Actions/Bitbucket Pipelines or equivalent): triggering suites, reading failures, improving pipeline reliability.
• Performance testing experience (JMeter/k6/Locust or equivalent): load/benchmark basics, interpreting results, flagging SLA/risk.
• Experience improving shared automation frameworks and raising team standards for maintainable tests.

Nice to have

• Broader stack familiarity: observability (logs/metrics/traces), cloud (AWS), contract testing, data-pipeline validation.
• Exposure to agentic development tools such as Cursor and Claude (or similar) for coding, test authoring, log/triage assistance, and documentation, used with human review and quality gates.

Details

• Locations: Bengaluru, Karnataka, India; Gurugram, Haryana, India.