AuroraMeter.Credits.LotMigration (Aurora Meter v1.0.0)

View Source

Replays a legacy wallet's ledger into lots and allocations, and cuts that wallet over to the allocator.

This is schema-migration step S5: a data step against the tables core schema version 9 created. It adds no DDL. Run it with mix aurora_meter.credits.migrate_lots, or call run/1 from a remote console on a release that has no Mix.

AuroraMeter.Credits.LotMigration.run(shadow: true)
AuroraMeter.Credits.LotMigration.status()

What it does, per wallet

  1. Snapshot, holding no lock: read the wallet's ledger rows and fold them into a book of lots.
  2. Lock: take the balance row FOR UPDATE, re-read lots_enabled_at, read any rows committed since the snapshot and fold them in.
  3. Verify: compare the folded book's projection with the locked balance row's balance, held and promotional. Any mismatch, and any ambiguity the fold raised, blocks the wallet.
  4. Write: insert the lots and allocations, backfill hold_transaction_id on the settle and release rows, set debt, expired and lots_enabled_at, and run the allocator's conservation check as the last statement before commit.
  5. Report: one aurora_meter_checkpoints row per wallet, plus the run's own cursor row. A shadow run and a real run keep separate cursor rows, so a rehearsal can never tell the real migration that it has already covered ground it has not touched (open-findings.md X427).

Steps 2 to 5 are one transaction, so a wallet is either entirely migrated or entirely untouched. Step 1 holds no lock, so a wallet with a long history costs the live path one short lock rather than a minutes-long stall.

The balance row is the oracle

The fold reproduces the arithmetic the legacy ledger performed, and then checks itself against numbers that ledger wrote: every row's balance_after, held_after and promotional_after, and finally the balance row itself. Those figures were computed by legacy arithmetic that knew nothing about lots, so they are an independent statement of the same facts rather than the fold's own arithmetic run twice. A wallet whose replay does not reproduce them exactly is never migrated, and is never adjusted to fit.

What it never does

It never writes balance, held, promotional, currency or low_balance_threshold; it never changes any historical column except hold_transaction_id, which was null; it deletes nothing; it contacts no provider and sends no mail. With shadow: true, which is the default, the only rows it writes at all are checkpoint rows, and the only cursor it writes is its own: a shadow run cannot change where the real migration starts.

Reading a run that migrated nothing

Two opposite situations used to print the same summary of zeros. The summary now separates them, and run/1 reports both figures:

  • resumed_from is the cursor the scan started after, nil if it started at the beginning of the table;
  • unexamined counts wallets still on the legacy writer that this run did not look at and that no run of this mode has recorded a verdict for. It is nil when the caller named tenants explicitly.

A run with wallets: 0 and unexamined: 0 had nothing to do. A run with wallets: 0 and unexamined above zero skipped wallets that still need migrating, reports state: "skipped_by_cursor", and the Mix task exits non-zero for it.

Cutting a wallet over

A real cutover was refused while there was no lot-aware refund path: a paid refund on a cut-over wallet would have gone through reverse/4, consumed promotional lots in spend order and written nothing into reversed (open-findings.md X250). AuroraMeter.Credits.reverse_lot/4 is that path and is the gate's own condition, so from the release that carries it shadow: false is permitted and cutover_blocked/0 answers nil.

A host that still wants the refusal keeps shadow: true, which remains the default: allow_cutover: true has to be asked for explicitly on every run that writes.

Both refund paths are lot aware from repair unit R1, and that is the correction this paragraph used to need. Opening the gate on reverse_lot/4 alone left reverse/4 planning a reversal as a debit, so a host that had no payment provenance, or an aurora_meter_pro fallback that could not match one, still took promotional credit first. reverse/4 now takes the wallet's non-promotional lots in spend order and writes reversed, so a cut-over wallet is safe on either call. reverse_lot/4 remains the right one to use where the payment is known, because only it is capped by that payment's own lots.

Ordering

A row written before schema version 9 has no sound order. inserted_at comes from a wall clock that steps backwards, and seq was assigned by version 9's table rewrite in physical order rather than insertion order. So the fold trusts neither: it folds in (inserted_at, id) order and checks the result against every row's own balance_after and held_after, which form an exact chain in true commit order. If that chain does not hold it folds again in seq order. If neither reproduces the chain, the wallet's history cannot be ordered from what it carries, and the wallet is blocked.

Summary

Types

The five figures this unit reconciles.

One ambiguity or observation the fold raised.

One wallet's outcome.

What one run did.

What one wallet's replay concluded.

Functions

The checkpoint name this unit uses for tenant_key.

Whether a real cutover is refused, and why. nil when it is permitted.

Folds rows into a book of lots, or says why it cannot.

Replays every wallet and, unless shadow: is false, reports without writing.

What the last run left behind: the aggregate cursors and every wallet report.

Types

figures()

@type figures() :: %{
  balance: integer(),
  held: non_neg_integer(),
  promotional: non_neg_integer(),
  debt: non_neg_integer(),
  expired: non_neg_integer()
}

The five figures this unit reconciles.

flag()

@type flag() :: %{flag: atom(), blocking: boolean(), detail: map()}

One ambiguity or observation the fold raised.

report()

@type report() :: %{
  tenant_key: String.t(),
  checkpoint: String.t(),
  state: verdict(),
  reason: atom() | nil,
  flags: [flag()],
  ordering: :inserted_at | :seq | nil,
  rows: non_neg_integer(),
  lots: non_neg_integer(),
  allocations: non_neg_integer(),
  hold_links: non_neg_integer(),
  lock_ms: non_neg_integer() | nil,
  before: figures(),
  after: figures() | nil
}

One wallet's outcome.

summary()

@type summary() :: %{
  shadow: boolean(),
  wallets: non_neg_integer(),
  migrated: non_neg_integer(),
  blocked: non_neg_integer(),
  deferred: non_neg_integer(),
  skipped: non_neg_integer(),
  rows: non_neg_integer(),
  lots: non_neg_integer(),
  allocations: non_neg_integer(),
  duration_ms: non_neg_integer(),
  lock_ms_max: non_neg_integer(),
  cursor: String.t() | nil,
  resumed_from: String.t() | nil,
  selection: :scan | :tenants,
  unexamined: non_neg_integer() | nil,
  state: String.t(),
  reports: [report()]
}

What one run did.

verdict()

@type verdict() :: :migrated | :blocked | :deferred | :shadow_ok | :skipped

What one wallet's replay concluded.

Functions

checkpoint_name(tenant_key)

@spec checkpoint_name(String.t()) :: String.t()

The checkpoint name this unit uses for tenant_key.

"lot_migration:<tenant_key>" when that is a legal AuroraMeter.Operations name, and "lot_migration:sha256-<digest>" when it is not. A tenant key is host supplied and may hold a space, an @ or a slash; AuroraMeter.Operations.pause/1 raises on anything outside [a-z_]+:[A-Za-z0-9_.:-]+, and a migration that raises halfway through a run because one customer's key has an @ in it is not a migration (open-findings.md X221). The row's counts always carries the verbatim tenant_key, so a digested name is still resolvable.

Examples

iex> AuroraMeter.Credits.LotMigration.checkpoint_name("org_42")
"lot_migration:org_42"

iex> digest = :sha256 |> :crypto.hash("ops@example.com") |> Base.encode16(case: :lower)
iex> AuroraMeter.Credits.LotMigration.checkpoint_name("ops@example.com") ==
...>   "lot_migration:sha256-" <> digest
true

cutover_blocked()

@spec cutover_blocked() :: %{finding: String.t(), reason: String.t()} | nil

Whether a real cutover is refused, and why. nil when it is permitted.

See the module documentation. The check is behavioural rather than a version number: it asks whether the lot-aware refund path (AuroraMeter.Credits.reverse_lot/4) exists, so it opened itself when that path shipped in build unit 06e and nothing else could have opened it. It answers nil from that release on, and the clause below is kept so a node running an older core still refuses rather than silently cutting wallets over.

What the probe cannot tell you, and it is worth saying rather than leaving for someone to discover: AuroraMeter.Credits and this module ship in one package, so within a release the probe can only report that release's own state. It was written to order two build units during development, not to version-check a deployment, and a core that carried reverse_lot/4 without repair unit R1's wallet-wide fix would have answered nil while AuroraMeter.Credits.reverse/4 was still planning a refund as a debit. No such core was ever published: 06e and R1 are both unreleased at the time of writing, so reverse_lot/4 and the lot-aware reverse/4 reach Hex together.

replay(rows, tenant_key)

@spec replay([AuroraMeter.Schema.CreditTransaction.t()], String.t()) ::
  {:ok, map()} | {:blocked, [flag()]}

Folds rows into a book of lots, or says why it cannot.

Pure: no repo, no clock, no configuration. rows are AuroraMeter.Schema.CreditTransaction structs for one wallet, in the order they are believed to have been committed. Exposed because it is the whole of the migration's judgement, and judgement is worth testing on its own.

run(opts \\ [])

@spec run(keyword()) :: {:ok, summary()} | {:error, term()}

Replays every wallet and, unless shadow: is false, reports without writing.

  • :repo (atom/0) - The repo to work through. Defaults to the configured one.

  • :shadow (boolean/0) - Compute and report, write nothing financial. The default value is true.

  • :tenant (term/0) - One tenant term, or a list of them. The default value is nil.

  • :batch (pos_integer/0) - Wallets per aggregate checkpoint write. The default value is 50.

  • :resume (boolean/0) - Continue from this mode's own cursor. Shadow and real keep separate ones. The default value is true.

  • :max_rows (pos_integer/0) - Defer a wallet larger than this. The default value is 50000.

  • :max_tail (non_neg_integer/0) - Rows folded in under the lock. The default value is 500.

  • :max_wallets (pos_integer/0) - Wallets one run examines. The default value is 100000.

  • :retry_blocked (boolean/0) - Reprocess wallets reported blocked. The default value is false.

  • :report_only (boolean/0) - Report an already migrated wallet too. The default value is false.

  • :allow_cutover (boolean/0) - Ask for a real cutover. The default value is false.

Returns {:ok, summary}. The run continues past a wallet it cannot migrate: a blocked wallet is recorded with its reasons and the next one is attempted, which is why the summary rather than the return tuple is what says whether the run was clean.

status(opts \\ [])

@spec status(keyword()) :: %{
  cursor: String.t() | nil,
  shadow_cursor: String.t() | nil,
  counts: map(),
  state: String.t() | nil,
  wallets: [map()]
}

What the last run left behind: the aggregate cursors and every wallet report.

:cursor is how far the real migration has got, which is the one an operator means by "where has this got to". :shadow_cursor is how far the rehearsal has got. They are separate rows and neither steers the other (open-findings.md X427).

Options: :repo, and :limit (default 200) on the wallet rows read back.