Overview
Staff Data Scientist on SurveyMonkey's Audience team, defining where synthetic data can responsibly extend audience capabilities and establishing the validation standards and roadmap for that work.
What you'll do
- Analyze existing panel and response data to identify where real supply is constrained, including low-incidence cohorts, B2B/professional audiences, and specific geographies.
- Assess where synthetic augmentation could provide value and where it may not be appropriate.
- Lead the development and evaluation of LLM-based synthetic respondents, fine-tuning and calibrating models against real response distributions.
- Evaluate alternative approaches, including tabular generative models (GANs, VAEs, and diffusion models) and statistical methods (Bayesian networks and copulas), based on their suitability for different use cases.
- Develop a framework to evaluate synthetic data quality, including fidelity to real-world distributions, preservation of utility for downstream analysis, and privacy and re-identification risk.
- Define the metrics and validation thresholds that will guide future synthetic data development.
- Translate technical findings into a phased product roadmap, identifying priorities, dependencies, and infrastructure requirements.
- Develop a bottoms-up business case evaluating cost-to-serve, margin impact, and potential reach expansion, and present recommendations to stakeholders and executive leadership.
- Work with Engineering to assess build-versus-buy decisions and infrastructure needs.
- Partner with Legal, Compliance, and Trust & Safety to evaluate disclosure, consent, and client-trust implications of introducing synthetic data into a business built on verified human panels.
- Partner with the response-quality team to identify signals that distinguish synthetic responses from authentic panel responses, and use synthetic respondents to stress-test quality scoring and inform detection approaches.
What you'll need
- 12+ years of experience combining applied statistics, machine learning, and product ownership.
- Hands-on experience building and validating generative or synthetic data models for structured/tabular data.
- Experience fine-tuning or calibrating LLMs to simulate human responses.
- Understanding of survey, panel, or market research methodologies, including sampling bias, weighting, non-response error, and incidence/feasibility.
- Ability to assess synthetic data against real panel data and identify limitations for research use cases.
- Experience evaluating model fidelity, downstream utility, and privacy or disclosure risk, including re-identification risk assessment, differential privacy, or comparable techniques.
- Sound judgment in determining when synthetic data does not meet required quality or privacy standards.
- Track record of taking a data product from concept to launch, including defining the roadmap, prioritizing and sequencing work, and developing business cases to secure investment.
- Strong SQL and Python skills, with experience using a modern analytics stack to independently query, analyze, and validate data.
Details
- Location: Bengaluru, India.
- Hybrid: work from the SurveyMonkey office in Bengaluru 3 days per week.
- Reports to the Senior Director of Product Management.
Read the full description and apply on the company’s own careers page.