defmodule GitGud.Repo.Migrations.AddWorkflowJobMatrix do
use Ecto.Migration
@moduledoc """
Matrix builds, and the `runs-on` change they need underneath.
`runs_on` was stored as `%{"value" => <whatever the YAML said>}` and
matched with `runs_on->>'value' = ANY(runner labels)`. That only ever
worked for the scalar form: `runs-on: [self-hosted, arm64]` stored a
JSON array, `->>` rendered it as the literal text `["self-hosted",
"arm64"]`, and it matched no runner at all — silently, as a job that
simply never got claimed.
The list form is exactly how you select an architecture, so it has to
work. `runs_on` becomes `%{"labels" => [...]}` — always a list, even
for a scalar — and claiming asks whether the job's labels are
*contained by* the runner's, which is GitHub's rule: a runner must
carry every label the job asks for.
`matrix` holds the one combination a job row was expanded for
(`%{"arch" => "arm64"}`). Jobs from one matrix keep their YAML
`job_id`, so `needs:` still names the group rather than a synthetic
per-leg id; the combination is what tells the legs apart.
"""
def up do
alter table(:workflow_jobs) do
add :matrix, :map
end
execute """
UPDATE workflow_jobs
SET runs_on = jsonb_build_object('labels',
CASE jsonb_typeof(runs_on -> 'value')
WHEN 'array' THEN runs_on -> 'value'
WHEN 'string' THEN jsonb_build_array(runs_on -> 'value')
ELSE '[]'::jsonb
END)
WHERE runs_on ? 'value'
"""
# Containment (`<@`) is a GIN operator; without this every claim
# sequentially scans the queued jobs, and claiming happens on every
# runner poll.
create index(:workflow_jobs, ["(runs_on -> 'labels') jsonb_path_ops"],
using: :gin,
name: :workflow_jobs_runs_on_labels_index
)
# `(run, job_id)` was unique, which is exactly what a matrix breaks:
# its legs deliberately share a `job_id` so `needs:` can name the
# group. The combination is what separates them, and `COALESCE` is
# load-bearing — NULLs compare as distinct in a unique index, so
# without it two matrix-less jobs with the same id would slip
# through the very constraint being replaced.
drop unique_index(:workflow_jobs, [:workflow_run_id, :job_id])
create unique_index(
:workflow_jobs,
[:workflow_run_id, :job_id, "(COALESCE(matrix, '{}'::jsonb))"],
name: :workflow_jobs_run_job_matrix_index
)
end
def down do
# `IF EXISTS` / `IF NOT EXISTS` throughout: a teardown that only
# works from the exact state the matching `up` left behind is a
# teardown you can't run when you actually need it.
execute "DROP INDEX IF EXISTS workflow_jobs_run_job_matrix_index"
execute """
CREATE UNIQUE INDEX IF NOT EXISTS workflow_jobs_workflow_run_id_job_id_index
ON workflow_jobs (workflow_run_id, job_id)
"""
execute "DROP INDEX IF EXISTS workflow_jobs_runs_on_labels_index"
# Back to the scalar shape. A multi-label job can't be represented,
# so it keeps its first label — which is what the old code would
# have matched on had it been written as a scalar.
execute """
UPDATE workflow_jobs
SET runs_on = jsonb_build_object('value',
COALESCE(runs_on -> 'labels' -> 0, '"ubuntu-latest"'::jsonb))
WHERE runs_on ? 'labels'
"""
alter table(:workflow_jobs) do
remove_if_exists :matrix, :map
end
end
end
neiam
/gitgud
Git Gud
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
1
Log in to mark this repository.
3.5 KiB · text
History
b868907