The Prometheus Certified Associate Exam (PCA)

The metric user_last_login_timestamp_seconds{email="john.doe@example.com"} is unsuitable for Prometheus because it includes a high-cardinality label (email). Each unique email address would generate a separate time series, potentially numbering in the millions, which severely impacts Prometheus performance and memory usage.

Prometheus is optimized for low- to medium-cardinality metrics that represent system-wide behavior rather than per-user data. High-cardinality metrics cause data explosion , complicating queries and overwhelming the storage engine.

By contrast, the other metrics—prometheus_engine_query_log_enabled, promhttp_metric_handler_requests_total{code="500"}, and http_response_total{handler="static/*filepath"}—adhere to Prometheus best practices. They represent operational or service-level metrics with limited, manageable label value sets.

The most widely used open-source visualization and dashboarding platform for Prometheus data is Grafana . Grafana provides native integration with Prometheus as a data source, allowing users to create real-time, interactive dashboards using PromQL queries.

Grafana supports advanced visualization panels (graphs, heatmaps, gauges, tables, etc.) and enables users to design custom dashboards to monitor infrastructure, application performance, and service-level objectives (SLOs). It also provides alerting capabilities that can complement or extend Prometheus’s own alerting system.

While Kibana is part of the Elastic Stack and focuses on log analytics, Thanos extends Prometheus for long-term storage and high availability, and Loki is a log aggregation system. None of these tools serve as the primary dashboarding solution for Prometheus metrics the way Grafana does.

Grafana’s seamless Prometheus integration and templating support make it the de facto standard visualization tool in the Prometheus ecosystem.

The max operator in PromQL is an aggregation operator , not a binary vector matching operator . Therefore, the valid syntax for aggregation uses by() or without(), not on().

✅ max by (instance) up → Valid; aggregates maximum values per instance.

✅ max without (instance) up and max without (instance, job) up → Valid; aggregates over all labels except those listed.

❌ max on (instance) (up) → Invalid; the keyword on() is only valid in binary operations (e.g., +, -, and, or, unless), where two vectors are being matched on specific labels.

Hence, max on (instance) (up) is a syntax error in PromQL because on() cannot be used directly with aggregation operators.

In Prometheus, histogram metrics use cumulative buckets to record the count of observations that fall within specific duration thresholds. Each bucket has a label le (“less than or equal to”), representing the upper bound of that bucket.

In the given metric, the bucket labeled le="0.1" has a value of 1 , meaning exactly one request took less than or equal to 0.1 seconds. Buckets are cumulative, so:

le="0.05" → 0 requests ≤ 0.05 seconds

le="0.1" → 1 request ≤ 0.1 seconds

le="1" → 3 requests ≤ 1 second

le="+Inf" → all 3 requests total

The _sum and _count values represent total duration and request count respectively, but the number of requests below a given threshold is read directly from the bucket’s le value.

In Prometheus, the scrape_interval parameter specifies how frequently the Prometheus server should scrape metrics from its configured targets . Each target exposes an HTTP endpoint (usually /metrics) that Prometheus collects data from at a fixed cadence. By default, the scrape_interval is set to 1 minute , but it can be overridden globally or per job configuration in the Prometheus YAML configuration file.

This setting directly affects the resolution of collected time series data —a shorter interval increases data granularity but also adds network and storage overhead, while a longer interval reduces load but might miss short-lived metric variations.

It is important to distinguish scrape_interval from evaluation_interval, which defines how often Prometheus evaluates recording and alerting rules. Thus, scrape_interval pertains only to data collection frequency , not to alerting or rule evaluation.

Prometheus collects metrics by scraping an HTTP endpoint exposed by the target application. Therefore, the only essential requirement for an application to expose metrics to Prometheus is that it serves metrics in the Prometheus text exposition format over HTTP .

This endpoint is conventionally available at /metrics and provides metrics in plain text format (e.g., Content-Type: text/plain; version=0.0.4). The application can run on any operating system, architecture, or network — as long as Prometheus can reach its endpoint.

It does not need to be Internet-accessible (it can be internal) and is not limited to Linux or any specific bitness.

In Prometheus alerting rules, the for field specifies how long a condition must remain true continuously before the alert transitions from the pending to the firing state. This feature prevents transient spikes or brief metric fluctuations from triggering false alerts.

Example:

alert: HighRequestLatency

expr: http_request_duration_seconds_avg > 1

for: 5m

labels:

severity: warning

annotations:

description: "Request latency is above 1s for more than 5 minutes."

In this configuration, Prometheus evaluates the expression every rule evaluation cycle. The alert only fires if the condition (http_request_duration_seconds_avg > 1) remains true for 5 consecutive minutes . If it returns to normal before that duration, the alert resets and never fires.

This mechanism adds stability and noise reduction to alerting systems by ensuring only sustained issues generate notifications.

In Prometheus, a rule group is a logical collection of recording and alerting rules that are evaluated sequentially at a specified interval. Rule groups are defined in YAML files under the groups: key, with each group containing a name, an interval, and a list of rules.

For example:

groups:

- name: example

interval: 1m

rules:

- record: job:http_inprogress_requests:sum

expr: sum(http_inprogress_requests) by (job)

All rules in a group share the same evaluation schedule and are executed one after another. This ensures deterministic order, especially when one rule depends on another’s result.

Metamonitoring refers to monitoring the monitoring system itself —ensuring that Prometheus, Alertmanager, exporters, and dashboards are functioning properly. In other words, it’s the observability of your observability stack.

This practice helps detect issues such as:

Prometheus not scraping targets,

Alertmanager being unreachable,

Exporters not exposing data, or

Storage being full or corrupted.

Without metamonitoring, an outage in the monitoring system could go unnoticed, leaving operators blind to actual infrastructure problems. A common approach is to use a secondary Prometheus instance (or external monitoring service) to monitor the health metrics of the primary Prometheus and related components.