AuroraMeter. Migration
(Aurora Meter v1.0.0)
View Source
Creates and drops the Aurora Meter core tables, in versions.
A host migration delegates here, so schema changes ship as new versions of
this module rather than as edits to the host's migration file. Every version
is idempotent (create_if_not_exists), so up/1 can safely run from version 1
each time.
Fresh install (generated by mix aurora_meter.gen.migration):
def up, do: AuroraMeter.Migration.up(from: 1, version: 10, concurrently: false)
def down, do: AuroraMeter.Migration.down(version: 10, to: 1, confirm_data_loss: true)Upgrading an existing install to a newer schema version, add a migration that runs only the new versions, pinned at both ends so that two databases built from the same migration files end up with the same schema whenever they are built:
def up, do: AuroraMeter.Migration.up(from: 3, version: 3)
def down, do: AuroraMeter.Migration.down(version: 3, to: 3, confirm_data_loss: true)Versions
- 1:
aurora_meter_subscriptions,aurora_meter_counters,aurora_meter_events - 2:
aurora_meter_history(UTC day buckets forAuroraMeter.history/3) - 3:
aurora_meter_credit_balances,aurora_meter_credit_transactions(the prepaid ledger behindAuroraMeter.Credits) - 4:
promotional_afteronaurora_meter_credit_transactions, so the promotional figure can be rebuilt from the log likebalanceandheld - 5: a partial index on open holds, so the sweep for reservations nothing will ever close does not scan the whole ledger
- 6:
aurora_meter_flush_receipts, committed atomically with counter and history deltas to deduplicate retries after uncertain writes - 7: durable event identity on
aurora_meter_events(event_id,payload_hash,occurred_at,period_start,kind,dimensions, plan attribution and aseqidentity column),quantitywidened tobigint, and the newaurora_meter_event_totalsandaurora_meter_checkpointstables. Additive: an 0.4.x node keeps writing events through it. Runmix aurora_meter.events.backfillafter it and before version 8 - 8: the unique index on
(tenant_key, event_id), builtCONCURRENTLY, andevent_id,payload_hashandoccurred_atpromoted toNOT NULLbehind it, which is what makes "one identity, one fact" a guarantee of the database rather than a convention of the application. It also validates the constraints version 7 addedNOT VALID. It must be generated into a host migration of its own, carrying@disable_ddl_transaction trueand@disable_migration_lock true, because Postgres refusesCREATE INDEX CONCURRENTLYinside a transaction block.up/1raisesAuroraMeter.Migration.ConcurrentVersionErrorrather than let it share a file. A fresh install passesconcurrently: falseand builds the index inside the transaction, which is right for a table with no rows in it - 9: the credit lot engine's tables (
aurora_meter_credit_lots,aurora_meter_credit_allocations,aurora_meter_credit_recurrences),debt,expired,lots_enabled_at,projection_checked_atandlow_balance_crossing_idonaurora_meter_credit_balanceswith theheld >= 0andpromotional >= 0checks the changeset has always claimed, andseq,updated_atandhold_transaction_idonaurora_meter_credit_transactions.seqis aGENERATED ALWAYS AS IDENTITYcolumn and it replacesinserted_atas the ledger's ordering key: a wall clock steps backwards and an identity cannot. Additive and writes no data; wallets stay on the legacy writer untilmix aurora_meter.credits.migrate_lotssetslots_enabled_at - 10: plan version identity.
aurora_meter_plan_versions(the snapshot of each plan version's commercial content, written byAuroraMeter.Plans.register!/0and read only to interpret a version whose block has been deleted from the host's plans module),aurora_meter_plan_transitions(created here, written by a later release), and ten columns onaurora_meter_subscriptions, of whichplan_versionis the one the version exists for: it pins a tenant to the contract they were sold, so redeploying an edited plan block is a refused boot rather than a silent repricing. Additive and writes no data; the legacy assignment ofplan_version = '1'is done byAuroraMeter.Plans.register!/0at the first boot after the upgrade, because the fingerprint it writes beside it is a digest of the compiled definition and cannot be computed in SQL. It also sets aclock_timestamp()default onaurora_meter_flush_receipts.inserted_at, so a receipt is stamped by the database that later compares it rather than by the node that wrote it
The schema marker
From version 7 on, a successful version writes the version it reached into
aurora_meter_checkpoints under the name "schema:core":
AuroraMeter.Checkpoints.get("schema:core").cursor["version"]Absence of the row means the database is below version 7, because the table itself arrives with version 7.
Options
:from/:version: the inclusive rangeup/1runs (:version/:tofordown/1). Always pin both ends in a generated file.:concurrently(up/1, defaulttrue): whether a concurrent version may useCREATE INDEX CONCURRENTLY.falsebuilds the index inside the migration's transaction, which is what a fresh install wants and what an install with a small events table can afford.:validate_checks(up/1, defaulttrue): whether to prove the constraints version 7 addedNOT VALIDagainst the existing rows.:lock_timeout(up/1, default"5s"): thelock_timeoutversion 7 sets around itsALTER TABLE, so a stuck lock fails fast rather than queueing every query onaurora_meter_eventsbehind it.:confirm_data_loss(down/1, defaultfalse): required to run thedownof a version that destroys a commercial fact.
Any other option raises ArgumentError, naming it. Until V1 an unknown
option was dropped, so up(prefix: "tenant_a") migrated the public schema
and returned :ok.
Schemas and prefixes
V1 creates and reads its tables in the repository's default schema, and
nothing else is supported. Three things are refused rather than half done:
a prefix: option here, a repository configured with
migration_default_prefix or a default_options prefix (refused at boot by
AuroraMeter.Config.validate!/0), and a migration run under
mix ecto.migrate --prefix. The reason is one fact about this package:
every runtime query it issues omits the prefix, so a migration that honoured
one would build the tables somewhere no query would ever read them.
Summary
Functions
The versions that must run in their own host migration, outside a DDL transaction.
The versions whose down destroys a commercial fact and so needs
confirm_data_loss: true.
Runs the down step of every version from :version (default the latest)
down to :to (default 1).
The newest schema version this release of Aurora Meter knows about.
Splits a version range into the host migration files it has to become.
Runs the up step of every version from :from (default 1) to :version
(default the latest). See the module documentation for the options.
Functions
@spec concurrent_versions() :: [pos_integer()]
The versions that must run in their own host migration, outside a DDL transaction.
Examples
iex> is_list(AuroraMeter.Migration.concurrent_versions())
true
@spec data_loss_versions() :: [pos_integer()]
The versions whose down destroys a commercial fact and so needs
confirm_data_loss: true.
Examples
iex> 7 in AuroraMeter.Migration.data_loss_versions()
true
@spec down(keyword()) :: :ok
Runs the down step of every version from :version (default the latest)
down to :to (default 1).
Raises AuroraMeter.Migration.DataLossError when the range holds a version
whose down destroys a commercial fact and confirm_data_loss: true was not
passed.
@spec latest_version() :: pos_integer()
The newest schema version this release of Aurora Meter knows about.
Examples
iex> AuroraMeter.Migration.latest_version()
10
@spec ranges(keyword()) :: [ %{from: pos_integer(), to: pos_integer(), concurrent: boolean()} ]
Splits a version range into the host migration files it has to become.
A contiguous run of ordinary versions travels in one file. A concurrent
version travels alone, because it creates an index CONCURRENTLY and its file
has to carry @disable_ddl_transaction true and @disable_migration_lock true, which are file-level attributes and would then apply to every other
version in the same file.
This is what makes a generated migration reproducible: each entry names both ends of its range, so a file committed today runs exactly the versions it ran the day it was written, whatever version the installed package has reached by the time the history is replayed (finding S1).
Options: :from (default 1) and :to (default latest_version/0).
Examples
iex> AuroraMeter.Migration.ranges(from: 7)
[
%{from: 7, to: 7, concurrent: false},
%{from: 8, to: 8, concurrent: true},
%{from: 9, to: 10, concurrent: false}
]
iex> AuroraMeter.Migration.ranges(from: 9)
[%{from: 9, to: 10, concurrent: false}]
@spec up(keyword()) :: :ok
Runs the up step of every version from :from (default 1) to :version
(default the latest). See the module documentation for the options.
Raises AuroraMeter.Migration.ConcurrentVersionError when the range holds a
concurrent version alongside another version and concurrently: false was
not passed.