defmodule FeedPug.PromEx.Plugins.Library do
@moduledoc """
What feedpug is holding: how many people use it, how many feeds they follow,
how many entries have accumulated, and how much of that gets read.
None of this is derived from a request or a job — they are inventory
questions only the database can answer, so every metric here is polled
rather than driven by a `:telemetry` event the app already emits. That is
the same shape as `RoomObservatory.Plugins.Sanctum` in apollos-crib, and it
carries the same consequence for the dashboard: **these gauges describe the
database, not the pod.** Every replica polls and reports the same numbers,
so panels aggregate them with `max` and never `sum` — the inversion of the
rule that applies to every other panel on the feedpug dashboard.
## Two poll rates
The fast group (60s) only touches tables that scale with *users*: `users`,
`feeds`, `groups`, `group_feeds`, `group_follows`. Those stay in the
thousands however long the instance runs, so a `GROUP BY` over them costs
nothing. It also carries `rows_estimate`, which reads `pg_class.reltuples`
for `items` and `item_reads` and touches neither table — the planner's
number, refreshed by autovacuum's ANALYZE, so it lags a big fetch by
minutes and is off by a percent or two. That is the right trade for a line
whose job is showing a trend.
The slow group (15m) is the one that scans. `items` grows without bound —
every entry of every feed anyone has ever added — and the read ratio joins
`item_reads` against it. It sits in its own polling group so its slow query
runs in its own poller process and cannot delay the cheap metrics above.
## What "followed" means, three ways
A feed is global and fetched once; the following happens in
`FeedPug.Groups.GroupFeed`, which places a feed in a user's group. So there
are three different true answers and all three are here:
* `feed_placements_count` — `group_feeds` rows. One user filing the same
feed under two groups counts twice.
* `feed_subscriptions_count` — distinct `{user, feed}` pairs. This is the
"followed rss feeds" number in the ordinary sense.
* `subscribed_feeds_count` — distinct feeds with at least one placement.
Subtract it from `feeds_count` to get feeds nobody follows any more:
still fetched on their timer, still filling up `items`.
`group_follows_count` is a fourth and unrelated kind of following — a user
following *another user's group*, which pulls that group's subtree into
their newsfeed.
## The read ratio, and what it leaves out
`read_ratio` is `subscribed_item_reads / readable_items`, where a readable
item is one `{user, item}` pair the user's own subscriptions put in front of
them. Both halves are exported separately, because a ratio nobody can
decompose is a number nobody trusts.
Two things are deliberately not in the denominator:
* **Feeds reached through a group follow.** `FeedPug.Groups.effective_feed_ids/1`
resolves those per user in Elixir — subtree by materialized path, minus
each `FollowExclusion` subtree — which is a query per user, not an
aggregate. Doing it here would make the cost scale with the user count
on every poll to move a ratio by a few percent.
* **Reads of items that are no longer subscribed.** `item_reads` keeps the
marker after a feed leaves a group, so counting all of them against a
subscribed-only denominator would let the ratio exceed 1. The numerator
is restricted to the same `{user, feed}` set as the denominator, which
keeps it in `0..1`. `item_reads_count` is the unrestricted total, and
the gap between the two is the abandoned-subscription tail.
## Where the numbers go stale
A gauge keyed by label keeps its last value until the process restarts, and
nothing here can say "this series is gone". Both label sets are closed and
zero-filled on every poll — the four `feeds.status` values from
`FeedPug.Feeds.Feed.statuses/0`, both `users.state` values — so a status
that empties out reads as zero instead of freezing at its last count.
"""
use PromEx.Plugin
require Logger
alias FeedPug.Feeds.Feed
alias FeedPug.Repo
@users_event [:feed_pug, :library, :users]
@feeds_event [:feed_pug, :library, :feeds]
@following_event [:feed_pug, :library, :following]
@estimate_event [:feed_pug, :library, :rows, :estimate]
@corpus_event [:feed_pug, :library, :corpus]
@library_poll_rate :timer.seconds(60)
@corpus_poll_rate :timer.minutes(15)
# The corpus query walks every row of `items` and `item_reads`. Long enough
# that doing so on a cold cache does not trip the 15s default and leave a
# permanent hole in the metric.
@corpus_timeout :timer.minutes(2)
# Estimate-only, because these are the two tables an exact count is too
# expensive to run at the fast rate.
@estimated_tables ~w(items item_reads)
@impl true
def polling_metrics(opts) do
prefix = Keyword.get(opts, :metric_prefix, [:phx, :feedpug])
[
library_metrics(prefix, Keyword.get(opts, :poll_rate, @library_poll_rate)),
corpus_metrics(prefix, Keyword.get(opts, :corpus_poll_rate, @corpus_poll_rate))
]
end
defp library_metrics(prefix, poll_rate) do
Polling.build(
:feedpug_library_polling_events,
poll_rate,
{__MODULE__, :execute_library_metrics, []},
[
last_value(
prefix ++ [:users, :count],
event_name: @users_event,
measurement: :users,
description:
"Registered users, split by whether they have confirmed their email address.",
tags: [:state]
),
last_value(
prefix ++ [:feeds, :count],
event_name: @feeds_event,
measurement: :feeds,
description:
"Canonical feeds known to the instance, by fetch status. A feed is stored once no matter how many people follow it.",
tags: [:status]
),
last_value(
prefix ++ [:feed, :subscriptions, :count],
event_name: @following_event,
measurement: :subscriptions,
description:
"Distinct {user, feed} pairs across every user's own groups — followed feeds in the ordinary sense."
),
last_value(
prefix ++ [:feed, :placements, :count],
event_name: @following_event,
measurement: :placements,
description:
"Rows in group_feeds. Higher than the subscription count by however often a user files one feed under several groups."
),
last_value(
prefix ++ [:subscribed, :feeds, :count],
event_name: @following_event,
measurement: :subscribed_feeds,
description:
"Distinct feeds with at least one placement. The shortfall against feeds_count is feeds nobody follows and the fetcher still polls."
),
last_value(
prefix ++ [:group, :follows, :count],
event_name: @following_event,
measurement: :group_follows,
description:
"Users following another user's group, which pulls that group's subtree into their newsfeed."
),
last_value(
prefix ++ [:rows, :estimate],
event_name: @estimate_event,
measurement: :rows,
description:
"Planner row estimate (pg_class.reltuples) for the two unbounded tables. Refreshed by ANALYZE, so it lags a large fetch.",
tags: [:table]
)
],
detach_on_error: false
)
end
defp corpus_metrics(prefix, poll_rate) do
Polling.build(
:feedpug_corpus_polling_events,
poll_rate,
{__MODULE__, :execute_corpus_metrics, []},
[
last_value(
prefix ++ [:items, :count],
event_name: @corpus_event,
measurement: :items,
description: "Exact count of stored feed entries across every feed."
),
last_value(
prefix ++ [:item, :reads, :count],
event_name: @corpus_event,
measurement: :item_reads,
description:
"Exact count of read markers, including markers left on items the user no longer subscribes to."
),
last_value(
prefix ++ [:readable, :items, :count],
event_name: @corpus_event,
measurement: :readable_items,
description:
"{user, item} pairs a user's own subscriptions put in front of them — the read ratio's denominator. Excludes feeds reached through a group follow."
),
last_value(
prefix ++ [:subscribed, :item, :reads, :count],
event_name: @corpus_event,
measurement: :subscribed_item_reads,
description:
"Read markers on items the user still subscribes to — the read ratio's numerator, on the same {user, feed} set as the denominator."
),
last_value(
prefix ++ [:read, :ratio],
event_name: @corpus_event,
measurement: :read_ratio,
description:
"subscribed_item_reads / readable_items, in 0..1. 0 when nothing is subscribed at all."
)
],
detach_on_error: false
)
end
@doc false
def execute_library_metrics do
emit_user_counts()
emit_feed_counts()
emit_following_counts()
emit_row_estimates()
rescue
error -> log_skipped(:library, error)
catch
:exit, reason -> log_skipped(:library, reason)
end
@doc false
def execute_corpus_metrics do
emit_corpus_counts()
rescue
error -> log_skipped(:corpus, error)
catch
:exit, reason -> log_skipped(:corpus, reason)
end
# -- users ---------------------------------------------------------------
defp emit_user_counts do
%{rows: rows} =
Repo.query!("SELECT confirmed_at IS NOT NULL, count(*) FROM users GROUP BY 1")
counts = Map.new(rows, fn [confirmed, count] -> {confirmed, count} end)
# Both states every poll: an instance where everybody has confirmed should
# read `pending 0`, not go quiet.
for {confirmed, state} <- [{true, "confirmed"}, {false, "pending"}] do
:telemetry.execute(
@users_event,
%{users: Map.get(counts, confirmed, 0)},
%{state: state}
)
end
end
# -- feeds ---------------------------------------------------------------
defp emit_feed_counts do
%{rows: rows} = Repo.query!("SELECT status, count(*) FROM feeds GROUP BY 1")
counts = Map.new(rows, fn [status, count] -> {status_label(status), count} end)
# Zero-filled across the schema's full status list rather than only the
# statuses present, so `failed` dropping back to zero is visible as a line
# returning to zero rather than as a series that stops.
statuses = Enum.uniq(Feed.statuses() ++ Map.keys(counts))
for status <- statuses do
:telemetry.execute(@feeds_event, %{feeds: Map.get(counts, status, 0)}, %{status: status})
end
end
defp status_label(nil), do: "unknown"
defp status_label(status), do: to_string(status)
# -- following -----------------------------------------------------------
@following_query """
SELECT
(SELECT count(*) FROM group_feeds),
(SELECT count(*) FROM (
SELECT DISTINCT g.user_id, gf.feed_id
FROM group_feeds gf
JOIN groups g ON g.id = gf.group_id
) s),
(SELECT count(DISTINCT feed_id) FROM group_feeds),
(SELECT count(*) FROM group_follows)
"""
defp emit_following_counts do
%{rows: [[placements, subscriptions, subscribed_feeds, group_follows]]} =
Repo.query!(@following_query)
:telemetry.execute(@following_event, %{
placements: placements,
subscriptions: subscriptions,
subscribed_feeds: subscribed_feeds,
group_follows: group_follows
})
end
# -- estimates -----------------------------------------------------------
@estimate_query """
SELECT c.relname::text, GREATEST(c.reltuples, 0)::bigint
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = current_schema()
AND c.relkind = 'r'
AND c.relname::text = ANY($1)
"""
defp emit_row_estimates do
%{rows: rows} = Repo.query!(@estimate_query, [@estimated_tables])
estimates = Map.new(rows, fn [table, estimate] -> {table, estimate} end)
# A table missing from pg_class has not been migrated yet; report 0 rather
# than leaving a gap, so the series exists from the first scrape.
for table <- @estimated_tables do
:telemetry.execute(@estimate_event, %{rows: Map.get(estimates, table, 0)}, %{table: table})
end
end
# -- corpus and read ratio -----------------------------------------------
# `subs` is the {user, feed} set a user's own groups subscribe them to.
# `per_feed` collapses items to one row per feed *before* the join, so the
# denominator costs one aggregate over items rather than a DISTINCT over
# every {user, item} pair it implies.
@corpus_query """
WITH subs AS (
SELECT DISTINCT g.user_id, gf.feed_id
FROM group_feeds gf
JOIN groups g ON g.id = gf.group_id
),
per_feed AS (
SELECT feed_id, count(*) AS items FROM items GROUP BY 1
)
SELECT
(SELECT count(*) FROM items),
(SELECT count(*) FROM item_reads),
-- ::bigint because sum() over a count is numeric, which Postgrex hands
-- back as a Decimal — and a Decimal denominator makes `ratio/2` raise
-- rather than divide.
(SELECT COALESCE(sum(pf.items), 0)::bigint
FROM subs s JOIN per_feed pf ON pf.feed_id = s.feed_id),
(SELECT count(*)
FROM item_reads r
JOIN items i ON i.id = r.item_id
JOIN subs s ON s.user_id = r.user_id AND s.feed_id = i.feed_id)
"""
defp emit_corpus_counts do
%{rows: [[items, item_reads, readable_items, subscribed_item_reads]]} =
Repo.query!(@corpus_query, [], timeout: @corpus_timeout)
:telemetry.execute(@corpus_event, %{
items: items,
item_reads: item_reads,
readable_items: readable_items,
subscribed_item_reads: subscribed_item_reads,
read_ratio: ratio(subscribed_item_reads, readable_items)
})
end
# A fresh instance with no subscriptions has read nothing rather than an
# undefined amount, and 0/0 is the state it spends its first day in.
defp ratio(_reads, 0), do: 0.0
defp ratio(reads, readable), do: reads / readable
defp log_skipped(group, reason) do
Logger.warning("FeedPug.PromEx.Plugins.Library skipped #{group} metrics: #{inspect(reason)}")
:ok
end
end
neiam
/feedpug
feedpub
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
Log in to mark this repository.
14.3 KiB · text
History
d7c0ce0