AuroraMeter.Exporter behaviour (Aurora Meter v1.0.0)

View Source

The seam between Aurora Meter and something that gets billed.

An exporter takes a list of items a caller has already decided to send, hands them to a provider, and says what happened to each one. It does not decide what to send, it does not persist anything, and it does not retry. Those are the caller's, which in Aurora Meter Pro is the outbox deliverer and in a host application is whatever the host wrote.

Aurora Meter ships exactly one implementation, AuroraMeter.Exporter.Journal, which records what it was given and returns scripted answers. The core contacts no network. A provider adapter is a host's code, or Aurora Meter Pro's.

defmodule MyApp.Exporter do
  @behaviour AuroraMeter.Exporter

  @impl true
  def describe do
    %{
      max_batch: 100,
      idempotency_horizon: 86_400,
      timestamp_window: %{past: 2_592_000, future: 300},
      supports: [:usage_window, :event]
    }
  end

  @impl true
  def deliver(items, _context) do
    Enum.map(items, fn item -> {item.id, send_one(item)} end)
  end
end

Write the adapter, then prove it with AuroraMeter.ExporterCase. See docs/exporters.md for the worked example.

The five outcomes

Every item gets exactly one of these, and the difference between them is money.

OutcomeMeansThe caller may
:acceptedthe provider took it, and gave nothing back to name itmark it accepted, with no provider reference
{:accepted, ref}the provider took it and named itmark it accepted and record ref
{:retry, seconds}it did not land, and sending the same item again is safe. nil means the provider named no delayschedule another attempt with the same payload
:uncertainit may have landed. Nobody knowsnot send a new identity for it. Reconcile
{:rejected, reason}it did not land and it never will in this formstop, and make the item visible to a human

Two rules hold this together, and both exist because the code this behaviour replaces got them wrong.

A rate limit is a retry, never a rejection. HTTP 429 means "not now", and an adapter that reports it as {:rejected, _} invites the caller to give up on an item, or worse, to re-derive a fresh idempotency key for it and double bill when the original request turns out to have landed.

An answer nobody understood is :uncertain, never :accepted and never {:rejected, _}. Both of those are terminal: one stops the money arriving and the other stops the retry. Uncertainty is the honest answer, and it is a state a caller can reconcile out of.

What an adapter must not do

  • Do not change the item. payload is the financial intent. It was decided when the item was created, and it must reach the provider byte for byte however many attempts it takes. An adapter that reads live application state to fill in a field has made the retry send something the original attempt did not.
  • Do not re-derive an identity. Key provider idempotency on item.payload["identifier"]. A fresh key on a retry is how one unit of usage gets billed twice.
  • Do not run inside the caller's transaction. deliver/2 performs network I/O and the caller guarantees it is called outside Repo.transaction/2 and outside a pinned connection. An adapter that opens its own database transaction around a provider call has put the provider inside a lock.
  • Do not raise for a provider problem. A timeout, a 500 and a rejection are all outcomes. A raise is a bug in the adapter, and although a careful caller rescues it (and must then treat every item in the call as :uncertain), an adapter that raises has thrown away the per-item answers it did have.

At-least-once, and what follows

deliver/2 may be called again for an item whose outcome the caller never learned: the caller died, the reply was lost, the lease expired. An adapter must answer a repeated item rather than refusing it, and the provider's idempotency on payload["identifier"] is what keeps the repeat from being a second charge. This is invariant I16's shape at the adapter level, and it is why :uncertain exists at all.

Time, and which clock

Nothing in this vocabulary is an absolute deadline, deliberately.

{:retry, seconds} is a duration, not an instant: the caller adds it to its own persisted clock reading, so the adapter never has to agree with the caller about what time it is. idempotency_horizon and timestamp_window are durations too.

first_attempt_at is the one instant that crosses the boundary, and it is there so an adapter can refuse an item older than its own idempotency horizon rather than send a second copy the provider will no longer recognise as a repeat. It is stamped by the caller's database, because it is persisted and later compared against, and in Aurora Meter that means AuroraMeter.Clock.db_now/0 or a clock_timestamp() default. The horizon is measured in hours, which is the only reason a clock may decide it at all: the shared database clock is shared but not monotonic, and it has been measured stepping backwards by hundreds of milliseconds on this hardware.

The consequence for anything short is blunt: a lease, a fence or a timeout is not decided with these types. Use SELECT ... FOR UPDATE SKIP LOCKED, an advisory lock, or a fencing token. context.attempt_started_at is for logging and context.timeout_ms is an in-process duration measured with AuroraMeter.Clock.monotonic_ms/0; neither is a correctness input.

Interpreting the answers

Do not read an adapter's return value directly. normalize/2 is the one interpretation, and it is shared so that Aurora Meter Pro, a host and the sample cannot disagree about what a missing entry means.

Summary

Types

Caller-scoped information that is not part of the financial payload. An adapter may ignore all of it, and nothing in it may change what is sent.

What an adapter promises about itself. Every number is a duration in seconds except max_batch, which is a count.

The answer for one item. Five shapes, and anything else is a bug the caller reads as :uncertain.

What an adapter returns: one entry per item it was given.

What an item is: a buffered usage window, one durable event, or a correction to one.

Callbacks

Delivers items and returns one {item_id, outcome} per item.

What this adapter can do. Pure, and constant for the life of the node.

Functions

How conservative an outcome is: higher wins when one id is answered twice.

Builds an AuroraMeter.Exporter.Item from a map or keyword list, raising ArgumentError on anything an adapter should not have to defend against.

Turns whatever an adapter returned into one outcome per item.

Whether term is one of the five documented outcome shapes.

The subject_kind values Aurora Meter knows about.

Types

context()

@type context() :: %{
  optional(:attempt_started_at) => DateTime.t(),
  optional(:lease_owner) => String.t(),
  optional(:timeout_ms) => pos_integer(),
  optional(atom()) => term()
}

Caller-scoped information that is not part of the financial payload. An adapter may ignore all of it, and nothing in it may change what is sent.

description()

@type description() :: %{
  max_batch: pos_integer(),
  idempotency_horizon: non_neg_integer(),
  timestamp_window: %{past: non_neg_integer(), future: non_neg_integer()},
  supports: [subject_kind()]
}

What an adapter promises about itself. Every number is a duration in seconds except max_batch, which is a count.

outcome()

@type outcome() ::
  :accepted
  | {:accepted, provider_ref :: String.t()}
  | {:retry, delay_seconds :: non_neg_integer() | nil}
  | :uncertain
  | {:rejected, reason :: atom() | String.t()}

The answer for one item. Five shapes, and anything else is a bug the caller reads as :uncertain.

result()

@type result() :: {item_id :: String.t(), outcome()}

What an adapter returns: one entry per item it was given.

subject_kind()

@type subject_kind() :: :usage_window | :event | :correction

What an item is: a buffered usage window, one durable event, or a correction to one.

Callbacks

deliver(items, context)

@callback deliver(items :: [AuroraMeter.Exporter.Item.t()], context :: context()) :: [
  result()
]

Delivers items and returns one {item_id, outcome} per item.

At most one provider round trip per item, although an adapter may batch them into one request. The order of the returned list does not matter; the id does. Returning fewer entries than items is not an error the adapter can express, but it is not free either: AuroraMeter.Exporter.normalize/2 reads a missing entry as :uncertain, which is the most expensive state a caller can be left in.

This is at-least-once. The same item may arrive again, with attempts incremented and payload unchanged.

describe()

@callback describe() :: description()

What this adapter can do. Pure, and constant for the life of the node.

max_batch is the largest list a caller may pass to deliver/2; the caller is at fault for exceeding it, and an adapter handed more may refuse the whole call. idempotency_horizon is how long, in seconds, the provider promises to recognise a repeated payload["identifier"]; past it the caller must reconcile rather than resend. timestamp_window bounds how far into the past and the future an occurrence time may be and still be accepted. supports lists the subject_kind values this adapter accepts.

Every number must come from the provider's current official documentation, with the URL and the date recorded by whoever wrote the adapter. A guess here is a guess about when a duplicate becomes a second charge.

Functions

conservatism(arg1)

@spec conservatism(outcome()) :: non_neg_integer()

How conservative an outcome is: higher wins when one id is answered twice.

Exposed because a caller that merges answers from more than one source (a delivery and a later reconciliation, say) needs the same order this module uses, rather than a second one that disagrees with it.

item!(fields)

@spec item!(map() | keyword()) :: AuroraMeter.Exporter.Item.t()

Builds an AuroraMeter.Exporter.Item from a map or keyword list, raising ArgumentError on anything an adapter should not have to defend against.

Checked: id, subject_ref and tenant_key are non-empty binaries; subject_kind is one of [:usage_window, :event, :correction]; payload is a map whose keys are all binaries; attempts is a non-negative integer; and first_attempt_at is nil or a DateTime in Etc/UTC.

The UTC check is not fussiness. first_attempt_at is compared against the idempotency horizon, and a local-zone instant in that comparison is a silent offset on the one decision that stops a second charge.

iex> item = AuroraMeter.Exporter.item!(%{
...>   id: "01HZ",
...>   subject_kind: :event,
...>   subject_ref: "evt_1",
...>   tenant_key: "acme",
...>   payload: %{"identifier" => "evt_1", "quantity" => 3}
...> })
iex> {item.attempts, item.first_attempt_at}
{0, nil}

normalize(items, raw_results)

@spec normalize([AuroraMeter.Exporter.Item.t()], term()) ::
  {:ok, %{optional(String.t()) => outcome()}}
  | {:error, {:unknown_ids, [String.t()]}}

Turns whatever an adapter returned into one outcome per item.

Four rules, and every one of them errs towards :uncertain because :uncertain is the only non-terminal state that cannot lose money:

  1. An id in the results that was not in items is an adapter bug and returns {:error, {:unknown_ids, ids}}. The caller treats the whole call as uncertain, because an adapter that answered for something it was not given may equally have sent something it was not given.
  2. An item with no entry becomes :uncertain. Not {:retry, nil}: a missing answer may follow a request that was actually made.
  3. An entry whose outcome is not one of the five documented shapes becomes :uncertain.
  4. Two entries for one id is an adapter bug, and the most conservative answer wins, ordered {:rejected, _}, then :accepted, then {:accepted, ref}, then {:retry, _}, then :uncertain.

An adapter that returned something that is not a list at all (:ok, or {:error, :batch_too_large}) leaves every item :uncertain.

Rule 4's order is a semantic promise, not an implementation detail: changing it would silently change what a caller does with an ambiguous answer, so a future change to it is a deliberate break.

iex> items = [
...>   AuroraMeter.Exporter.item!(%{id: "a", subject_kind: :event, subject_ref: "r1",
...>     tenant_key: "acme", payload: %{"identifier" => "r1"}}),
...>   AuroraMeter.Exporter.item!(%{id: "b", subject_kind: :event, subject_ref: "r2",
...>     tenant_key: "acme", payload: %{"identifier" => "r2"}})
...> ]
iex> AuroraMeter.Exporter.normalize(items, [{"a", :accepted}])
{:ok, %{"a" => :accepted, "b" => :uncertain}}

iex> items = [AuroraMeter.Exporter.item!(%{id: "a", subject_kind: :event,
...>   subject_ref: "r1", tenant_key: "acme", payload: %{"identifier" => "r1"}})]
iex> AuroraMeter.Exporter.normalize(items, [{"a", :accepted}, {"ghost", :accepted}])
{:error, {:unknown_ids, ["ghost"]}}

outcome?(arg1)

@spec outcome?(term()) :: boolean()

Whether term is one of the five documented outcome shapes.

An adapter does not need this; a caller that wants to log the difference between "the provider said something odd" and "the item went uncertain for another reason" does.

subject_kinds()

@spec subject_kinds() :: [subject_kind()]

The subject_kind values Aurora Meter knows about.