Staff Software Engineer, Inference
CoreWeave — London, England, United Kingdom · In-Person · Staff+ · £116k - £155k/yearly
Role description
Role Overview
As a Staff Software Engineer on the Inference team, you will operate as a technical leader across multiple teams and services, driving architecture, performance, and reliability for CoreWeave’s Kubernetes-native inference platform. You will define and lead complex, cross-cutting design initiatives spanning request routing, adaptive scheduling, GPU resource management, and cost-per-token optimization under strict P99 SLAs.
What You Will Do
You will implement advanced inference optimizations—such as speculative decoding and KV-cache reuse—while establishing performance benchmarking frameworks, guiding cross-functional alignment across infrastructure boundaries, and raising the bar for engineering rigor and observability practices.
Why It Might Be a Fit
We believe in investing in our people, and value candidates who can bring their own diversified experiences to our teams even if you aren't a 100% skill or experience match. You will be surrounded by some of the best talent in the industry, who will want to learn from you, too.
Requirements
- ~8–12+ years of experience building large-scale distributed systems or cloud platforms.
- Proven track record of leading cross-team or organization-level technical initiatives at scale.
- Strong coding skills in Go, Python, or C++.
- Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design.
- Strong understanding of networked systems, performance optimization, and distributed system design.
- Hands-on engineering experience with inference systems, including batching/micro-batching strategies, caching, memory optimization, mixed precision (BF16/FP8), and streaming token delivery.
- Demonstrated ability to systematically improve tail latency (P95/P99) and platform reliability through metrics-driven engineering.
- Experienced in owning system-wide SLIs/SLOs, capacity planning, autoscaling strategies, and mentoring senior and mid-level engineers.
- Bachelor’s degree in Computer Science, Engineering, or a related technical field.
Benefits
- Family-level Medical Insurance
- Family-level Dental Insurance
- Generous Pension Contribution
- Life Assurance at 4x Salary
- Critical Illness Cover
- Employee Assistance Programme
- Tuition Reimbursement