Push vs. Pull Monitoring

Every monitoring system faces the same question: how do metrics get from the thing being monitored to the thing doing the monitoring?

There are only two answers. Either the monitored system pushes metrics to the collector, or the collector pulls metrics from the monitored system. That's it. Every monitoring architecture is one of these, or a hybrid of both.

Pull asks: "Are you there?" Push announces: "I'm here." That difference cascades through every design decision.

The Pull Model

In pull-based monitoring, collectors reach out to monitored systems and fetch metrics. Prometheus is the canonical example.

The monitored system exposes an HTTP endpoint—typically /metrics—that returns current metric values. The collector connects to this endpoint at regular intervals, parses the response, and stores the data.

This creates a specific relationship: the collector is in control. It decides when to sample, how often to scrape, and which targets to monitor. The monitored system just sits there, exposing its current state to anyone who asks.

What pull gets right:

Service discovery becomes natural. When a new container spins up and registers itself, the collector automatically starts scraping it. When the container dies, scraping stops. No configuration changes needed on either side.

Troubleshooting is direct. You can curl the metrics endpoint yourself. The data is right there, exposed, queryable. If the collector is broken, you can still see what the monitored system would report.

Collector scaling is simple. Want more redundancy? Point another collector at the same targets. Each collector independently scrapes everything, giving you automatic failover.

What pull gets wrong:

Ephemeral processes are nearly impossible to monitor. If a batch job runs for 30 seconds and your scrape interval is 60 seconds, you might never successfully pull metrics from it. The job exists, does its work, and vanishes—never once answering when you called.

Firewalls become adversaries. The collector must be able to reach every monitored system. In distributed environments, cloud scenarios, or anything behind NAT, this means firewall rules, VPN configurations, or accepting that some systems simply can't be pulled from.

Every metric endpoint needs authentication. Otherwise anyone who can reach your services can read your metrics—potentially sensitive operational data about your infrastructure.

The Push Model

In push-based monitoring, monitored systems send metrics to collectors. StatsD, Datadog agents, and CloudWatch follow this pattern.

The monitored system runs an agent or instrumentation library that periodically gathers metrics and pushes them to a collector endpoint. The monitored system initiates the connection and controls what gets sent.

This inverts the relationship: the monitored system is in control. It decides when to send data, what to include, and handles its own scheduling.

What push gets right:

Firewalls stop being obstacles. The monitored system makes outbound connections to the collector. No inbound rules needed. A server behind NAT, a Lambda function, a container in a private subnet—all can push metrics to a central collector without any network gymnastics.

Ephemeral processes work naturally. A 30-second batch job pushes its metrics during execution. When it terminates, the metrics are already safely at the collector. The process doesn't need to exist long enough to be discovered and scraped.

Immediate delivery is possible. When something important happens, metrics can be pushed immediately rather than waiting for the next scrape interval.

What push gets wrong:

Collector failures cause data loss. If the collector is unreachable, where do pushed metrics go? Agents can buffer locally, but buffers fill. Eventually, data drops. You need retry logic, persistent queues, or acceptance that network blips mean metric gaps.

Load balancing gets complicated. Push to one collector and you have a single point of failure. Push to a pool of collectors and now every agent needs to know about load balancing, or you need infrastructure to distribute incoming pushes.

Configuration sprawl happens. Every monitored system needs to know where to push metrics. Changing collector endpoints means updating configuration everywhere. Service discovery helps, but the fundamental problem remains: the monitored systems need to know about the collectors.

The Decision Framework

The choice isn't about which architecture is "better." It's about which problems you'd rather have.

Choose pull when:

• Your services are long-lived with stable endpoints
• You have service discovery (Kubernetes, Consul, etc.)
• Collectors can reach all monitored systems
• You want collector-controlled sampling intervals
• You value being able to query metrics endpoints directly during incidents

Choose push when:

• You're monitoring ephemeral processes (batch jobs, Lambda functions)
• Monitored systems are behind firewalls or NAT
• You can't establish inbound network connectivity to all sources
• You're already using push-based instrumentation libraries
• Immediate metric delivery matters more than collector-controlled sampling

Hybrid Reality

Most production environments use both.

Prometheus, the poster child for pull-based monitoring, includes Pushgateway specifically for cases where pull doesn't work. Batch jobs push to Pushgateway; Prometheus pulls from Pushgateway. Push-to-pull bridge.

Cloud monitoring services typically accept both. Push metrics via API for your applications, pull via integrations for infrastructure that exposes endpoints.

The pragmatic approach: use pull for long-lived services where it works elegantly, use push for ephemeral processes and network-restricted sources. Let the architecture fit the problem rather than forcing the problem to fit the architecture.

Failure Modes

Understanding how each approach fails helps you plan for it.

When pull fails:

If your collector dies, metrics remain exposed at the source. Spin up a new collector, point it at the same targets, and you're scraping again. You lose the data between the last successful scrape and now, but nothing is permanently broken.

If a monitored system dies, scraping fails. This is actually useful—failed scrapes are a signal that something is wrong. The absence of data is itself data.

When push fails:

If your collector dies, agents buffer locally until buffers fill, then start dropping metrics. When the collector recovers, agents resume pushing, but there's a gap—and no way to recover data that was never sent.

If a monitored system dies, pushes stop. But silence is ambiguous: is the system down, or is the network broken? You need separate health checking because the absence of pushes doesn't tell you why there are no pushes.

The Underlying Truth

Push and pull aren't just implementation details. They're different answers to the question: who's responsible for making sure metrics arrive?

With pull, the collector is responsible. It reaches out, it schedules, it handles failures by trying again. The monitored system's only job is to be there when asked.

With push, the monitored system is responsible. It initiates, it buffers, it retries. The collector's only job is to be available to receive.

That division of responsibility shapes everything else—scalability, reliability, configuration, troubleshooting. When you understand which side owns the responsibility, the rest of the tradeoffs become obvious.