Staff Research Engineer, Model Efficiency (LLM Inference) at Cohere (New York)

Job Description

Cohere’s Model Efficiency team is hiring a Staff Research Engineer to push the limits of LLM inference efficiency, which the ad calls the main bottleneck. The team works across the model execution stack: model architecture and mixture-of-experts routing, decoding algorithms, software–hardware co-design for GPUs and performance tuning without hurting quality. The team is concentrated in the Eastern and Pacific time zones.

At a glance

• Team: Model Efficiency
• Location: New York preferred (EST/PST), other Cohere offices possible
• Focus: LLM inference efficiency
• Apply: open until filled (re-checked 14 October 2026)

What you would do

• Develop and prototype efficiency improvements across model architecture and MoE routing
• Improve decoding and inference-time algorithms
• Co-design software and hardware for GPU acceleration
• Ship performance gains without reducing model quality

Why this role matters

Serving large language models is expensive, and every gain in inference efficiency lowers cost and latency for customers. Cohere’s efficiency team works on the whole stack, from model design to GPU kernels, so a staff engineer here sees the full picture of how modern LLMs run in production. The team sits mainly in the Eastern and Pacific time zones, which the ad names as preferred locations.

What Cohere is looking for

• Deep experience in ML systems or model efficiency research
• Strong GPU and performance engineering skills
• A record of shipping improvements in production models

Nice to have

• Experience with speculative decoding, quantisation or MoE systems

Pay and location

Cohere lists pay ranges on the posting that vary by location; check the posting for the New York range.

How to apply

Apply through the official Cohere job posting. Cohere’s posting does not give a closing date, so the role is open until filled; ResearchJobs.in will re-check this listing on 14 October 2026. Check the posting for location and work-authorisation details before you apply.

See all our artificial intelligence jobs, or browse more research jobs on ResearchJobs.in.