neiam /apollos_crib

It's Apollo's Crib

public · Issues · Pulls 1 · Labels · Forks · Compare · Actions failure · Badges · Packages

Let the database find the stops near a point

8685bee · Gabriel Morell · 2026-09-03 16:47

13 files +108 -10

Message

{commit_body(@commit)}

Files changed

modified PLANI.md

+21 −10

@@ -67,11 +67,13 @@ and the work is a spatial query.

67	67	\| `gbfs` stations \| `gbfs_station_information.place`, PostGIS \| yes — `ST_DWithin` + `<->` ordering already used \|
68	68	\| `gbfs` free bikes \| `gbfs_free_bike_status`, PostGIS \| yes \|
69	69	\| `aqi` monitors \| `airnow_hourly_observations.point`, PostGIS \| yes — `nearby_aqi_stations/3` \|
70		−\| `gtfs` stops \| `stops.stop_lat` / `stop_lon`, plain floats \| no — no geometry column, no index, no nearest-stop query anywhere \|
	70	+\| `gtfs` stops \| `gtfs_stops.place`, generated from `stop_lat` / `stop_lon` \| yes — `nearby_stops/3`, GiST indexed \|
71	71
72		−GTFS is the one anybody would want first and the only one needing real work: a
73		−geometry column on `stops`, a GiST index, and backfilling it on import. It is
74		−also the largest table in the system, so the index is not optional.
	72	+GTFS was the gap and is now closed. The column is generated rather than
	73	+written, because the importer builds its bulk inserts from the schema's fields
	74	+and would otherwise have to maintain it; a generated column cannot drift from
	75	+the coordinates it comes from and stays out of the importer's way by staying
	76	+out of the Ecto schema.
75	77
76	78	*Sources answered at* a point rather than near one.** Weather, sunrise,
77	79	pollen, the tide at the nearest station: there is one answer and it is

@@ -97,18 +99,27 @@ rather than discovering at runtime.

97	99	changes.
98	100	3. Nearest-N for what is already indexed. GBFS stations and bikes, AQI
99	101	monitors. The queries exist; they need a limit and a caller.
100		−4. GTFS stops. Geometry column, GiST index, backfill on import, then
101		− nearest-stop. The biggest piece and the one worth having.
	102	+4. ~~GTFS stops.~~ Done: `gtfs_stops.place` is generated and GiST indexed,
	103	+ and `nearby_stops/3` uses it. What is still missing is a query that asks
	104	+ for them — GBFS has `mode: :area`, GTFS has only a named stop.
102	105
103	106	## Open questions
104	107
105	108	- N or a radius? "Five nearest" and "everything within 500m" answer
106	109	differently when you are somewhere empty. GBFS queries already take a
107	110	radius; nearest-N would be a second mode rather than a replacement.
108		−- Is a Plani its own thing, or a mode on a Pythiae? A Pythiae already has
109		− `curr_foci`. A Plani could be a foci whose place is resolved at read time,
110		− which would mean nothing else has to know about it — the smaller change, and
111		− probably the right one.
	111	+- Is a Plani its own thing, or a mode on a Pythiae? Settled: not a mode,
	112	+ and nothing XORs with visions. A Pythiae already carries `curr_foci`, which
	113	+ is settable in the UI and read by nothing. If a query resolved its anchor
	114	+ through the Pythiae's `curr_foci` rather than its own `foci_id`, the same
	115	+ vision would answer for wherever the Pythiae is pointed, and a Plani would
	116	+ be a foci whose place comes from a GenServer rather than a column. The
	117	+ vision machinery, the condensers and the clients would all be untouched.
	118	+
	119	+ What does not compose is asking for things there is no query for. GBFS
	120	+ already has `mode: :area` around a foci with a radius, so "bikes near me"
	121	+ works the moment the foci moves. GTFS has only a named stop, so "stops near
	122	+ me" needs a new query mode — a query type, not a Pythiae mode.
112	123	- What does a client see while it has never reported? The home foci, which
113	124	is correct but silent. A board that says which anchor it used would save
114	125	somebody wondering why the times are for the wrong town.

modified ?

+0 −0

Binary file — not rendered.

modified ?

+0 −0

Binary file — not rendered.

modified apps/room_sanctum/lib/room_sanctum/storage.ex

+38 −0

@@ -3257,6 +3257,44 @@ defmodule RoomSanctum.Storage do

3257	3257	One row per station: the feed replaces the whole set each hour, so the
3258	3258	current reading and the station itself are the same record.
3259	3259	"""
	3260	+ @doc """
	3261	+ Stops nearest a point, closest first.
	3262	+
	3263	+ The counterpart of `nearby_aqi_stations/3` for transit, and the query a
	3264	+ moving anchor needs: a GTFS query names one stop, so until now there was no
	3265	+ way to ask which stops are near anything.
	3266	+
	3267	+ `place` is a generated column and deliberately absent from the Stop schema --
	3268	+ the importer builds its inserts from the schema's fields and would try to
	3269	+ write it -- so it is named in a fragment rather than through the struct.
	3270	+
	3271	+ Ordered by the index operator rather than by distance, so this uses the GiST
	3272	+ index instead of measuring every stop and sorting the result.
	3273	+ """
	3274	+ def nearby_stops(source_id, point, limit \\ 5)
	3275	+
	3276	+ def nearby_stops(source_id, %Geo.Point{} = point, limit) do
	3277	+ from(s in Stop,
	3278	+ where: s.source_id == ^source_id and not is_nil(s.stop_lat) and not is_nil(s.stop_lon),
	3279	+ limit: ^limit,
	3280	+ order_by: fragment("place <-> ?", ^point)
	3281	+ )
	3282	+ \|> Repo.all()
	3283	+ end
	3284	+
	3285	+ def nearby_stops(_source_id, _point, _limit), do: []
	3286	+
	3287	+ @doc """
	3288	+ Stops nearest a foci, closest first.
	3289	+
	3290	+ Resolving the anchor separately from the search is what lets a Plani hand in
	3291	+ a position instead: see `nearby_stops/3`.
	3292	+ """
	3293	+ def nearest_stops(source_id, foci_id, limit \\ 5) do
	3294	+ foci = Cfg.get_foci!(foci_id)
	3295	+ nearby_stops(source_id, foci.place, limit)
	3296	+ end
	3297	+
3260	3298	def nearest_aqi_stations(source_id, foci_id, limit \\ 5) do
3261	3299	foci = Cfg.get_foci!(foci_id)
3262	3300	nearby_aqi_stations(source_id, foci.place, limit)

added apps/room_sanctum/priv/repo/migrations/20260903120000_index_stops_by_place.exs

+49 −0

@@ -0,0 +1,49 @@

	1	+defmodule RoomSanctum.Repo.Migrations.IndexStopsByPlace do
	2	+ use Ecto.Migration
	3	+
	4	+ # CREATE INDEX CONCURRENTLY cannot run inside a transaction, and Ecto wraps
	5	+ # migrations in one by default.
	6	+ @disable_ddl_transaction true
	7	+ @disable_migration_lock true
	8	+
	9	+ @moduledoc """
	10	+ Make "the stops near here" a question the database can answer.
	11	+
	12	+ GTFS stops carry `stop_lat` and `stop_lon` as plain floats, so finding the
	13	+ nearest ones meant reading every row of the largest table in the system.
	14	+ There was no such query anywhere as a result.
	15	+
	16	+ The column is generated rather than filled in: the GTFS importer builds its
	17	+ bulk inserts from `schema.__schema__(:fields)`, so a normal column would be
	18	+ named in every INSERT and would have to be maintained by the import. A
	19	+ generated one cannot drift from the coordinates it is derived from, needs no
	20	+ backfill, and is invisible to the importer because it stays out of the Ecto
	21	+ schema.
	22	+
	23	+ SRID 4326 to match foci, which are stored that way -- comparing geometries
	24	+ with different SRIDs is an error rather than a wrong answer, which is at
	25	+ least loud, but only at query time.
	26	+ """
	27	+
	28	+ def up do
	29	+ execute("""
	30	+ ALTER TABLE gtfs_stops
	31	+ ADD COLUMN place geometry(Point, 4326)
	32	+ GENERATED ALWAYS AS (
	33	+ ST_SetSRID(ST_MakePoint(stop_lon, stop_lat), 4326)
	34	+ ) STORED
	35	+ """)
	36	+
	37	+ # Concurrently: this table is large and is written by an import that runs
	38	+ # on a timer, so a plain CREATE INDEX would hold a lock against it.
	39	+ execute("""
	40	+ CREATE INDEX CONCURRENTLY IF NOT EXISTS gtfs_stops_place_index
	41	+ ON gtfs_stops USING GIST (place)
	42	+ """)
	43	+ end
	44	+
	45	+ def down do
	46	+ execute("DROP INDEX CONCURRENTLY IF EXISTS gtfs_stops_place_index")
	47	+ execute("ALTER TABLE gtfs_stops DROP COLUMN IF EXISTS place")
	48	+ end
	49	+end

Parents: 5856372