AuroraMeter.Migration (Aurora Meter v1.0.0)

View Source

Creates and drops the Aurora Meter core tables, in versions.

A host migration delegates here, so schema changes ship as new versions of this module rather than as edits to the host's migration file. Every version is idempotent (create_if_not_exists), so up/1 can safely run from version 1 each time.

Fresh install (generated by mix aurora_meter.gen.migration):

def up, do: AuroraMeter.Migration.up(from: 1, version: 10, concurrently: false)
def down, do: AuroraMeter.Migration.down(version: 10, to: 1, confirm_data_loss: true)

Upgrading an existing install to a newer schema version, add a migration that runs only the new versions, pinned at both ends so that two databases built from the same migration files end up with the same schema whenever they are built:

def up, do: AuroraMeter.Migration.up(from: 3, version: 3)
def down, do: AuroraMeter.Migration.down(version: 3, to: 3, confirm_data_loss: true)

Versions

  • 1: aurora_meter_subscriptions, aurora_meter_counters, aurora_meter_events
  • 2: aurora_meter_history (UTC day buckets for AuroraMeter.history/3)
  • 3: aurora_meter_credit_balances, aurora_meter_credit_transactions (the prepaid ledger behind AuroraMeter.Credits)
  • 4: promotional_after on aurora_meter_credit_transactions, so the promotional figure can be rebuilt from the log like balance and held
  • 5: a partial index on open holds, so the sweep for reservations nothing will ever close does not scan the whole ledger
  • 6: aurora_meter_flush_receipts, committed atomically with counter and history deltas to deduplicate retries after uncertain writes
  • 7: durable event identity on aurora_meter_events (event_id, payload_hash, occurred_at, period_start, kind, dimensions, plan attribution and a seq identity column), quantity widened to bigint, and the new aurora_meter_event_totals and aurora_meter_checkpoints tables. Additive: an 0.4.x node keeps writing events through it. Run mix aurora_meter.events.backfill after it and before version 8
  • 8: the unique index on (tenant_key, event_id), built CONCURRENTLY, and event_id, payload_hash and occurred_at promoted to NOT NULL behind it, which is what makes "one identity, one fact" a guarantee of the database rather than a convention of the application. It also validates the constraints version 7 added NOT VALID. It must be generated into a host migration of its own, carrying @disable_ddl_transaction true and @disable_migration_lock true, because Postgres refuses CREATE INDEX CONCURRENTLY inside a transaction block. up/1 raises AuroraMeter.Migration.ConcurrentVersionError rather than let it share a file. A fresh install passes concurrently: false and builds the index inside the transaction, which is right for a table with no rows in it
  • 9: the credit lot engine's tables (aurora_meter_credit_lots, aurora_meter_credit_allocations, aurora_meter_credit_recurrences), debt, expired, lots_enabled_at, projection_checked_at and low_balance_crossing_id on aurora_meter_credit_balances with the held >= 0 and promotional >= 0 checks the changeset has always claimed, and seq, updated_at and hold_transaction_id on aurora_meter_credit_transactions. seq is a GENERATED ALWAYS AS IDENTITY column and it replaces inserted_at as the ledger's ordering key: a wall clock steps backwards and an identity cannot. Additive and writes no data; wallets stay on the legacy writer until mix aurora_meter.credits.migrate_lots sets lots_enabled_at
  • 10: plan version identity. aurora_meter_plan_versions (the snapshot of each plan version's commercial content, written by AuroraMeter.Plans.register!/0 and read only to interpret a version whose block has been deleted from the host's plans module), aurora_meter_plan_transitions (created here, written by a later release), and ten columns on aurora_meter_subscriptions, of which plan_version is the one the version exists for: it pins a tenant to the contract they were sold, so redeploying an edited plan block is a refused boot rather than a silent repricing. Additive and writes no data; the legacy assignment of plan_version = '1' is done by AuroraMeter.Plans.register!/0 at the first boot after the upgrade, because the fingerprint it writes beside it is a digest of the compiled definition and cannot be computed in SQL. It also sets a clock_timestamp() default on aurora_meter_flush_receipts.inserted_at, so a receipt is stamped by the database that later compares it rather than by the node that wrote it

The schema marker

From version 7 on, a successful version writes the version it reached into aurora_meter_checkpoints under the name "schema:core":

AuroraMeter.Checkpoints.get("schema:core").cursor["version"]

Absence of the row means the database is below version 7, because the table itself arrives with version 7.

Options

  • :from / :version: the inclusive range up/1 runs (:version / :to for down/1). Always pin both ends in a generated file.
  • :concurrently (up/1, default true): whether a concurrent version may use CREATE INDEX CONCURRENTLY. false builds the index inside the migration's transaction, which is what a fresh install wants and what an install with a small events table can afford.
  • :validate_checks (up/1, default true): whether to prove the constraints version 7 added NOT VALID against the existing rows.
  • :lock_timeout (up/1, default "5s"): the lock_timeout version 7 sets around its ALTER TABLE, so a stuck lock fails fast rather than queueing every query on aurora_meter_events behind it.
  • :confirm_data_loss (down/1, default false): required to run the down of a version that destroys a commercial fact.

Any other option raises ArgumentError, naming it. Until V1 an unknown option was dropped, so up(prefix: "tenant_a") migrated the public schema and returned :ok.

Schemas and prefixes

V1 creates and reads its tables in the repository's default schema, and nothing else is supported. Three things are refused rather than half done: a prefix: option here, a repository configured with migration_default_prefix or a default_options prefix (refused at boot by AuroraMeter.Config.validate!/0), and a migration run under mix ecto.migrate --prefix. The reason is one fact about this package: every runtime query it issues omits the prefix, so a migration that honoured one would build the tables somewhere no query would ever read them.

Summary

Functions

The versions that must run in their own host migration, outside a DDL transaction.

The versions whose down destroys a commercial fact and so needs confirm_data_loss: true.

Runs the down step of every version from :version (default the latest) down to :to (default 1).

The newest schema version this release of Aurora Meter knows about.

Splits a version range into the host migration files it has to become.

Runs the up step of every version from :from (default 1) to :version (default the latest). See the module documentation for the options.

Functions

concurrent_versions()

@spec concurrent_versions() :: [pos_integer()]

The versions that must run in their own host migration, outside a DDL transaction.

Examples

iex> is_list(AuroraMeter.Migration.concurrent_versions())
true

data_loss_versions()

@spec data_loss_versions() :: [pos_integer()]

The versions whose down destroys a commercial fact and so needs confirm_data_loss: true.

Examples

iex> 7 in AuroraMeter.Migration.data_loss_versions()
true

down(opts \\ [])

@spec down(keyword()) :: :ok

Runs the down step of every version from :version (default the latest) down to :to (default 1).

Raises AuroraMeter.Migration.DataLossError when the range holds a version whose down destroys a commercial fact and confirm_data_loss: true was not passed.

latest_version()

@spec latest_version() :: pos_integer()

The newest schema version this release of Aurora Meter knows about.

Examples

iex> AuroraMeter.Migration.latest_version()
10

ranges(opts \\ [])

@spec ranges(keyword()) :: [
  %{from: pos_integer(), to: pos_integer(), concurrent: boolean()}
]

Splits a version range into the host migration files it has to become.

A contiguous run of ordinary versions travels in one file. A concurrent version travels alone, because it creates an index CONCURRENTLY and its file has to carry @disable_ddl_transaction true and @disable_migration_lock true, which are file-level attributes and would then apply to every other version in the same file.

This is what makes a generated migration reproducible: each entry names both ends of its range, so a file committed today runs exactly the versions it ran the day it was written, whatever version the installed package has reached by the time the history is replayed (finding S1).

Options: :from (default 1) and :to (default latest_version/0).

Examples

iex> AuroraMeter.Migration.ranges(from: 7)
[
  %{from: 7, to: 7, concurrent: false},
  %{from: 8, to: 8, concurrent: true},
  %{from: 9, to: 10, concurrent: false}
]

iex> AuroraMeter.Migration.ranges(from: 9)
[%{from: 9, to: 10, concurrent: false}]

up(opts \\ [])

@spec up(keyword()) :: :ok

Runs the up step of every version from :from (default 1) to :version (default the latest). See the module documentation for the options.

Raises AuroraMeter.Migration.ConcurrentVersionError when the range holds a concurrent version alongside another version and concurrently: false was not passed.