Overview
Build speech models from the ground up for Blue Machines' enterprise voice AI agents: implement architectures from scratch, pre-train them on large audio corpora, then adapt and ship them to live production calls.
What you'll do
- Pre-train STT models from scratch (Conformer/Zipformer encoders, CTC/RNN-T/TDT decoders, self-supervised pre-training such as wav2vec 2.0/HuBERT/BEST-RQ), then fine-tune them for Indian languages, Hinglish and BFSI vocabulary
- Train TTS models from scratch (flow-matching, diffusion, neural codec LMs, vocoders) and extend them to new voices, languages and speaking styles
- Build speech-to-speech components from scratch: audio tokenizers and codecs, speech encoders connected to LLMs, streaming decoders
- Turn papers and new ideas into working PyTorch code, writing the model, tokenizer and training loop yourself and training from random initialisation
- Build data pipelines: cleaning, segmentation, forced alignment, pseudo-labelling, augmentation (noise, codecs, 8 kHz telephony simulation)
- Maintain evaluation suites: WER/CER, entity error rate, code-switch accuracy, MOS and speaker similarity, latency (time to first token / byte)
- Optimise models for serving: quantisation, streaming chunking, ONNX/TensorRT export, batching on GPUs
- Analyse production errors, turn them into test sets, and close the loop with the platform team
- Write clear experiment reports and share findings in weekly research reviews
What you'll need
- 1–3 years in ML, with at least one real project in speech or audio (industry, research lab, or a strong thesis)
- B.Tech/M.Tech/MS in CS, EE or a related field; a PhD is not required
- Strong Python and PyTorch; able to implement a model architecture and training loop from scratch, not just call APIs or fine-tune checkpoints
- Working knowledge of speech basics: spectrograms, MFCC/mel features, CTC, attention-based seq2seq, vocoders
- Hands-on with at least one of: NeMo, ESPnet, Hugging Face Transformers/Audio, fairseq, Coqui, k2/icefall
- Has trained at least one model from random initialisation (speech, audio or language), including multi-GPU jobs, mixed precision and experiment tracking (W&B, MLflow)
- Careful with evaluation: knows why a WER number can mislead, and checks your data
Nice to have
- Speak or understand Hindi or another Indian language
- Publications, Kaggle/benchmark results, or open-source contributions in speech
- Exposure to real-time audio (WebRTC, SIP, streaming inference) or telephony audio
- Familiarity with LLM fine-tuning (LoRA, SFT) or audio-language models
Details
- Location: Bengaluru (on-site / hybrid)
- Team: Speech Research, reporting into the CTO org
Read the full description and apply on the company’s own careers page.