Staff Software Engineer, Inference

CoreWeave — London, England, United Kingdom · In-Person · Staff+ · £116k - £155k/yearly

Apply for this role

Role description

Role Overview

As a Staff Software Engineer on the Inference team, you will operate as a technical leader across multiple teams and services, driving architecture, performance, and reliability for CoreWeave’s Kubernetes-native inference platform. You will define and lead complex, cross-cutting design initiatives spanning request routing, adaptive scheduling, GPU resource management, and cost-per-token optimization under strict P99 SLAs.

What You Will Do

You will implement advanced inference optimizations—such as speculative decoding and KV-cache reuse—while establishing performance benchmarking frameworks, guiding cross-functional alignment across infrastructure boundaries, and raising the bar for engineering rigor and observability practices.

Why It Might Be a Fit

We believe in investing in our people, and value candidates who can bring their own diversified experiences to our teams even if you aren't a 100% skill or experience match. You will be surrounded by some of the best talent in the industry, who will want to learn from you, too.

Requirements

  • ~8–12+ years of experience building large-scale distributed systems or cloud platforms.
  • Proven track record of leading cross-team or organization-level technical initiatives at scale.
  • Strong coding skills in Go, Python, or C++.
  • Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design.
  • Strong understanding of networked systems, performance optimization, and distributed system design.
  • Hands-on engineering experience with inference systems, including batching/micro-batching strategies, caching, memory optimization, mixed precision (BF16/FP8), and streaming token delivery.
  • Demonstrated ability to systematically improve tail latency (P95/P99) and platform reliability through metrics-driven engineering.
  • Experienced in owning system-wide SLIs/SLOs, capacity planning, autoscaling strategies, and mentoring senior and mid-level engineers.
  • Bachelor’s degree in Computer Science, Engineering, or a related technical field.

Benefits

  • Family-level Medical Insurance
  • Family-level Dental Insurance
  • Generous Pension Contribution
  • Life Assurance at 4x Salary
  • Critical Illness Cover
  • Employee Assistance Programme
  • Tuition Reimbursement

More jobs at CoreWeave

Canonical job page

More roles at CoreWeave