Data Scientist, Evaluations (Chanakya) at Sarvam AI (Bengaluru)

Job Description

Sarvam is hiring a Data Scientist to anchor evaluations for its Chanakya vertical in Bengaluru: designing domain-specific evaluation frameworks for high-stakes AI use cases where a wrong answer has real consequences, and deciding whether a system is good enough to deploy and stays good afterwards.

At a glance

• Role: Data Scientist, Evaluations (Chanakya vertical)
• Where: Bengaluru, India
• Experience: 3–6 years in data science, ML research or applied AI, including 2+ years with LLMs in production
• Focus: evaluation frameworks for document comprehension, summarisation, geospatial reasoning and workflow automation
• Apply: open until filled (re-checked 12 October 2026)

About the evaluations role

• Design and build evaluation frameworks for Sarvam’s AI outputs across domain-specific tasks such as document comprehension, command summarisation, geospatial reasoning and enterprise workflow automation
• Define quality metrics with domain experts and clients, turning operational requirements into measurable, defensible signals
• Run structured evaluation cycles before and after deployment, and build dashboards that show model quality in production
• Find failure modes, edge cases and distribution shifts, looking for what is wrong rather than confirming what is right
• Work with the MLOps engineer to automate evaluation pipelines, and with the product manager and deployment team

Indian-language AI is a distinct research problem: many languages have little training data, scripts vary widely, and users mix languages in a single sentence. Building models, evaluations and serving systems that work well for these users is what makes roles at an Indian foundation-model company different from similar roles elsewhere.

What Sarvam is looking for

• 3–6 years in data science, ML research or applied AI, with at least 2 years on LLMs in production
• Strong statistics and probability, and an understanding of what makes an evaluation valid or misleading
• Experience designing evaluation frameworks from scratch: custom metrics, inter-rater reliability and red-teaming
• Python with tools such as pandas, NumPy, Hugging Face datasets, RAGAS, the EleutherAI evaluation harness or LangSmith
• Experience with prompt engineering, fine-tuning or RLHF in applied settings, and with unstructured domain data such as PDFs, transcripts and field reports

Nice to have

• Prior work in high-stakes domains such as healthcare, legal, defence or finance
• Red-teaming or adversarial evaluation experience

Pay and location

Sarvam’s posting does not state a salary. The role is based in Bengaluru.

How to apply

Apply through the official Sarvam job posting. Sarvam’s posting does not give a closing date, so the role is open until filled; ResearchJobs.in will re-check this listing on 12 October 2026. Sarvam’s posting says it wants people who can own the outcomes described, not people who match every line of the specification. Check the posting for location and work-authorisation details before you apply.

See all our artificial intelligence jobs, or browse more research jobs on ResearchJobs.in.