AuroraMeter.Oban.HoldReconciliation (Aurora Meter v1.0.0)

View Source

Asks the configured AuroraMeter.Credits.HoldReconciler about holds that are still open past a cutoff, and applies what it says.

One call to AuroraMeter.Credits.reconcile_holds/1 and nothing else.

config :my_app, Oban,
  plugins: [{Oban.Plugins.Cron, crontab: [
    {"*/15 * * * *", AuroraMeter.Oban.HoldReconciliation}
  ]}]

With no :credits_hold_reconciler configured this worker keeps every hold, which is the default and is why scheduling it changes nothing until a host writes a policy. Read AuroraMeter.Credits.HoldReconciler before configuring one: age is not evidence that work was abandoned.

Job arguments

KeyDefaultMeaning
"older_than_seconds"3600Only holds open longer than this are examined.
"limit"200The most holds one batch examines.
"max_batches"10Batches one job runs before returning.
"reference_prefix"noneNarrow to one kind of work.
"tenant"noneSweep one tenant.
%{"older_than_seconds" => 86_400, "reference_prefix" => "job:"}
|> AuroraMeter.Oban.HoldReconciliation.new()
|> Oban.insert()

The cursor is never a job argument (L05c-1). It lives in aurora_meter_checkpoints under "hold_reconciliation:global", so two jobs for this worker read the same position rather than each carrying its own stale copy.

Pausing

AuroraMeter.Operations.pause("hold_reconciliation:global")

The next batch boundary cancels the job with {:cancel, :paused}, leaving the cursor where the last batch left it.

The cutoff is pinned for the length of a scan

older_than_seconds selects the candidate set, and the set moves as time passes. A scan spanning several jobs stores the cutoff it started with in its checkpoint beside the cursor, so a resumed cursor points into the set it came from. When the scan completes, the cursor is cleared and the next run takes a fresh cutoff.

The cutoff is a duration, and it is an hours-scale one

older_than_seconds is the one duration in this worker. It selects candidates and decides nothing: having passed the cutoff makes a hold a hold the host is asked about, never a hold that is released. The default is an hour and the smallest value that makes sense is minutes, which is far outside the sub-second range in which a shared clock cannot be trusted to order two instants (AuroraMeter.Clock).

The cutoff is measured with AuroraMeter.Clock.now/0 rather than db_now/0, because the other side of the comparison is a hold row's inserted_at, and until the ledger's rows are stamped by the database that is a node's wall clock. Both sides of a time comparison come from the same clock, and which clock that is follows from where the other side came from.

Running it twice

Harmless, and the operation's doing rather than this worker's. Two nodes sweeping at the same instant both list a hold and both ask the host about it, and at most one terminal transition happens: the decision is applied through AuroraMeter.Credits.settle/3 or AuroraMeter.Credits.release/2, which re-read status = 'pending' under the hold row's own FOR UPDATE. The loser is told :already_closed and counted. There is no lease and no fence, so there is nothing a clock could invert.

A host callback that raises, hangs or answers nonsense keeps the hold, and so does a run this worker is killed in the middle of.

Without Oban, call AuroraMeter.Credits.reconcile_holds/1 from whatever scheduler you do have. See the scheduler map.

Summary

Functions

The checkpoint name this worker pauses and resumes under.

Reconciles pending holds in bounded batches and returns {:ok, report}.

Functions

operation()

@spec operation() :: String.t()

The checkpoint name this worker pauses and resumes under.

perform(job)

@spec perform(Oban.Job.t()) :: {:ok, term()} | {:error, term()} | {:cancel, term()}

Reconciles pending holds in bounded batches and returns {:ok, report}.

{:cancel, :paused} when the operation is paused. {:error, reason} when the listing itself fails, which fails the job and lets Oban retry it. It never raises on an operation error and never matches on the report's shape: a later release may add keys to it.

Examples

{:ok, report} =
  AuroraMeter.Oban.HoldReconciliation.perform(%Oban.Job{
    args: %{"older_than_seconds" => 3600}
  })