Principal Observability Platform Engineer

Role description

Role Overview

As a Principal/Staff Observability Platform Engineer, you'll own the technical direction of Nscale's observability platform, setting the architectural roadmap and ensuring the platform scales ahead of the business. You'll treat observability as a product and a discipline, not a tooling exercise.

What You Will Do

Own the technical strategy and architecture for observability across metrics, logs, traces, and alerting at scale. Drive platform decisions that have multi-year impact, identify systemic gaps, and design platforms that make failure visible and fast to diagnose.

Why It Might Be a Fit

You'll influence without authority, mentor and technically grow the observability team, and lead incident postmortems to drive durable platform improvements. You'll have the opportunity to work on a high-performance infrastructure for AI start-ups and large enterprise customers.

Requirements

• 8+ years in SRE, infrastructure engineering, platform engineering, or observability-focused roles
• Deep hands-on experience with a significant subset of: Prometheus, Thanos, VictoriaMetrics, Grafana, Loki, Tempo, OpenTelemetry, ClickHouse, Elastic
• Strong engineering fundamentals, proficient in Python, Go, or similar; comfortable owning complex systems end to end
• Experience with Kubernetes at scale; familiarity with GPU infrastructure or HPC environments (Slurm) is a strong plus
• Infrastructure-as-Code is default, not optional (Terraform, Ansible, or equivalent)

Benefits

• medical
• dental
• vision
• flexible paid time off
• parental leave
• retirement plan participation