Port 9090: Prometheus — The Internet's Pulse Monitor

Port 9090 is the default home of Prometheus, the monitoring system that became the heartbeat tracker for the cloud-native era. When you see a Grafana dashboard showing request latency, error rates, or CPU usage in a Kubernetes cluster, the data flowing into those graphs almost certainly passed through port 9090.

What Prometheus Does

Prometheus is a time-series database and monitoring system designed for dynamic, ephemeral infrastructure. It collects metrics, stores them with timestamps and labels, provides a powerful query language to analyze them, and fires alerts when things go wrong.¹

The key insight is the pull model. Most monitoring systems from the previous era used push: your application would shout its metrics at a central server whenever something happened. Prometheus inverts this. Every 15 seconds (by default), the Prometheus server reaches out to each target, hits its /metrics endpoint over HTTP, and asks: "What's your current state?"²

This sounds backwards, but it's genius for three reasons:

1. Health detection is automatic. If Prometheus can't reach you, that itself is a signal. The up metric tells you immediately which targets are unreachable.³
2. Targets stay simple. Your application just needs to expose an HTTP endpoint with plain text. It doesn't need to know where the monitoring server lives or have credentials to push anywhere.⁴
3. The server controls the pace. No risk of getting overwhelmed by a thundering herd of metrics during an incident. Prometheus scrapes at its own rhythm.⁵

How the Protocol Works

The Prometheus exposition format is elegantly minimal. When Prometheus hits your /metrics endpoint, it expects plain text, one metric per line:⁶

# HELP http_requests_total Total number of HTTP requests
# TYPE http_requests_total counter
http_requests_total{method="GET",path="/api/users",status="200"} 1847293
http_requests_total{method="POST",path="/api/users",status="201"} 42891
http_requests_total{method="GET",path="/api/users",status="500"} 127

The format is designed to be human-readable and machine-parseable. Lines starting with # provide metadata about what the metric means and what type it is. The metric name comes first, followed by labels in curly braces, then the value. That's it.

The labels are the revolutionary part. Instead of having separate metrics for each HTTP method, each path, each status code, you have one metric with dimensions. This is what Prometheus calls its "multi-dimensional data model," and it's what lets you slice and dice your data in ways that weren't practical before.⁷

Want to know your total request rate? Sum across all labels. Want to know just the error rate for POST requests to /api/users? Filter by those labels. The query language, PromQL, makes this slicing natural:

rate(http_requests_total{status=~"5.."}[5m])

This gives you the per-second rate of 5xx errors over the last 5 minutes, automatically broken down by whatever labels exist on that metric.

The Story Behind Port 9090

In 2012, SoundCloud was drowning. They had migrated to microservices, which meant hundreds of services, thousands of instances, containers spinning up and dying constantly. Their existing monitoring, built on StatsD and Graphite, couldn't keep up. You can't hardcode dashboard targets when your infrastructure is a moving target.⁸

Two engineers at SoundCloud, Matt Proud and Julius Volz, had both come from Google. At Google, they had used an internal monitoring system called Borgmon, which was built for exactly this kind of dynamic environment. Borgmon monitored the Borg cluster scheduler (the ancestor of Kubernetes), and it used the same pull-based, label-rich approach that would become Prometheus.⁹

But Borgmon wasn't open source. It didn't exist outside Google's walls.

So Proud and Volz built Prometheus. The first commit landed on November 24, 2012.¹⁰ Research had started earlier that year, exploring different languages (C, C++, Java, Go) and storage backends (Cassandra, LevelDB) before settling on Go with a custom time-series database.¹¹

By the end of 2012, they had an end-to-end prototype: scrape metrics, store them, query them with an early version of PromQL, and graph the results. That proof of concept gave them the confidence to make it SoundCloud's production monitoring system in 2013.¹²

They stayed quiet about it publicly. No announcement, no marketing, just code and word of mouth. Docker and a few other companies found it and started contributing. Then in January 2015, they made the official public announcement.¹³

In May 2016, the Cloud Native Computing Foundation was forming, and they needed projects beyond just Kubernetes. Prometheus joined as the second CNCF project, right alongside the container orchestrator it was born to monitor.¹⁴ It graduated from incubation in August 2018, cementing its place as critical infrastructure for the cloud-native ecosystem.¹⁵

The name? Prometheus stole fire from the gods and gave it to humanity. This Prometheus takes the observability practices that Google developed internally and gives them to everyone.

The Technical Foundation

Time-Series Storage

Prometheus stores data in a custom time-series database (TSDB) inspired by Facebook's Gorilla paper.¹⁶ It uses a block-based architecture:

• Head Block: Recent samples live in memory, served from RAM for fast queries
• Write-Ahead Log (WAL): All incoming data is written to the WAL first for durability
• Blocks: Every two hours, the head block is compacted into an immutable block on disk
• Compaction: Older blocks are merged together to save space and improve query performance¹⁷

The compression is aggressive. Prometheus exploits the fact that metrics are scraped at regular intervals and values tend to change gradually. A sample that might take 16 bytes uncompressed can be stored in about 1-2 bytes on average.¹⁸

Service Discovery

Static configuration doesn't work when your infrastructure is dynamic. Prometheus integrates with service discovery systems (Kubernetes, Consul, EC2, etc.) to automatically find targets to scrape.¹⁹ When a new pod spins up in Kubernetes, Prometheus discovers it within seconds and starts collecting its metrics.

Alerting

Prometheus doesn't send alerts directly. Instead, you define alerting rules in PromQL:

alert: HighErrorRate
expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.1
for: 5m
labels:
  severity: critical
annotations:
  summary: "High error rate detected"

When this rule fires, Prometheus pushes the alert to the Alertmanager (port 9093), which handles deduplication, grouping, silencing, and routing to notification channels like Slack, PagerDuty, or email.²⁰

Port 9090 in the Ecosystem

Prometheus chose 9090 as its default port and established a convention for the entire ecosystem:²¹

Port 9092 is deliberately skipped to avoid collision with Apache Kafka.

Security Considerations

Prometheus was not designed with security as a primary concern. The official security model documentation is blunt: "It is presumed that untrusted users have access to the Prometheus HTTP endpoint and logs."²²

This has led to serious real-world problems:

Exposed Endpoints

Research in 2024 found over 336,000 Prometheus servers and exporters exposed directly to the Internet.²³ The /metrics endpoint can reveal:

• Internal API endpoints and architecture
• Subdomains and infrastructure topology
• Environment variables and configuration details
• Credentials embedded in metric labels²⁴

Debug Endpoints as Weapons

The /debug/pprof/heap endpoint, if exposed, can be weaponized for denial-of-service attacks. Repeatedly requesting heap profiles can overwhelm the server with CPU and memory load, potentially crashing it.²⁵

Information Disclosure

• /api/v1/status/config can leak usernames and passwords from configuration files
• /api/v1/targets can expose metadata labels including environment variables
• /api/v1/status/flags can reveal if administrative interfaces are enabled²⁶

Mitigations

Prometheus added support for TLS and basic authentication in version 2.24.0 (January 2021), but it's not enabled by default.²⁷ Operators are responsible for:

• Never exposing Prometheus directly to the Internet
• Running it behind a reverse proxy with authentication
• Using network policies to restrict access
• Monitoring debug endpoints for abuse

The Philosophical Shift

Prometheus represents a fundamental change in how we think about monitoring.

The old model was checks: Is this service up? Is this disk more than 90% full? Is this process running? You defined thresholds, and the monitoring system told you pass or fail.

Prometheus's model is metrics: Collect everything, store the time series, and let humans (and alerting rules) ask questions later. Instead of "is the disk full?" you collect disk usage continuously and ask "how fast is disk usage growing?" or "given current growth rate, when will the disk fill?"²⁸

This is the philosophy Google's SRE teams developed and documented in their Site Reliability Engineering book: focus on the Four Golden Signals (latency, traffic, errors, saturation), collect them as metrics, set Service Level Objectives (SLOs), and alert when you're at risk of missing them.²⁹

Prometheus made this philosophy accessible to everyone. You don't need Google's resources to monitor like Google.

Related Ports

Current Relevance

Prometheus is not just relevant; it's foundational. According to the 2025 Grafana Observability Survey, 67% of organizations use Prometheus in production.³⁰ The CNCF's 2024 survey shows that among organizations using Kubernetes (93% of respondents), Prometheus is the most common monitoring solution.³¹

The OpenMetrics format, an evolution of Prometheus's exposition format, is now an IETF standard, meaning the way Prometheus expects metrics to be formatted has become the universal language for cloud-native observability.³²

Every major cloud provider offers managed Prometheus services. AWS has Amazon Managed Service for Prometheus. Google Cloud Monitoring supports PromQL natively. Azure Monitor integrates with Prometheus. The format and query language have become industry standards even outside the original project.