AuroraMeter.
View Source
The conformance suite every AuroraMeter.Exporter should pass.
An exporter decides how usage becomes money, and the ways it can be subtly wrong are the expensive ones: a rate limit read as a rejection, a lost acknowledgement read as an acceptance, a batch that stops at its first problem. This suite is how an adapter author finds out before a customer does.
defmodule MyApp.ExporterConformanceTest do
use AuroraMeter.ExporterCase, exporter: MyApp.Exporter, async: true
def script(subject_ref, scenario), do: MyApp.Exporter.Fake.script(subject_ref, scenario)
def sent(subject_ref), do: MyApp.Exporter.Fake.sent(subject_ref)
endThe three hooks
All optional, all defined in the test module itself.
setup_exporter/1 runs in setup and returns a map merged into the test
context: start a fake, check out a connection, whatever the adapter needs.
script/2 takes a subject reference and a scenario, and makes the adapter
answer that reference that way on its next delivery. Return :ok when the
adapter can be made to do it and :unsupported when it cannot. The scenarios:
| Scenario | Make the adapter behave as if |
|---|---|
:accepted | the provider took it |
{:accepted, ref} | the provider took it and named it |
{:retry, seconds} | the provider said "not now" and named a delay |
:uncertain | the answer was lost after the request was sent |
{:rejected, reason} | the provider refused it permanently |
:rate_limited | the provider returned a rate limit (HTTP 429) |
:malformed | the provider returned something the adapter could not parse |
:unknown_status | the provider returned a status the adapter does not know |
:raise | the provider call blew up |
sent/1 takes a subject reference and returns the payloads the adapter
actually sent for it, oldest first. Without it the payload-identity area
cannot be checked, because nothing else can see what left the process.
Areas that cannot be scripted are named, not hidden
Forcing every adapter author to fake a 429 would make this suite unusable, so
an area whose scenario script/2 reports :unsupported does not fail: the
test passes and the area is recorded. One test then always runs and names
every area that was not covered, through IO.warn/1, so "the suite passed"
never quietly means "the suite did nothing".
Pass require_scriptable: true to turn that warning into a failure. An
adapter that talks to a real provider should: the whole point of a fake is
that it can be made to misbehave.
Reading a failure
Every assertion is a public function of this module and every generated test
is one line, so a failure names a function you can open rather than a line
inside a macro expansion. That also makes the suite testable: a deliberately
wrong adapter can be handed straight to assert_identity!/1 and the
assertion proved to reject it, which is the only way to know the suite has
ever rejected anything.
What it does not do
It does not start your adapter, it does not know your provider, and it makes
no network call of its own. It asserts the contract in
AuroraMeter.Exporter and nothing else.
Summary
Functions
The contract areas this suite checks.
Names every area this suite could not exercise, and fails instead of warning
when the case was used with require_scriptable: true.
E4. describe/0 returns the documented map, with numbers that make sense.
E4. describe/0 is constant.
E3. A provider call that blows up leaves every item in that call :uncertain.
E1. Three items in, three outcomes out, keyed by the ids that went in.
E2. An answer the adapter could not parse is :uncertain.
E1. One batch, two members, two different answers.
E5, and core's half of invariant I15. Delivering the same item twice sends the same bytes.
E2. A rate limit is a retry.
E2. A scripted retry arrives as {:retry, delay} or {:retry, nil}, and as
nothing else.
E2. A provider status nobody wrote a clause for is :uncertain.
Calls the adapter the way a correct caller must, and returns
{raw_answer, normalized}.
Builds count items for this suite, with a subject reference nothing else
will collide with and a subject kind the adapter says it supports.
The scenarios script/2 may be asked for.
Asks the suite's script/2 for one scenario, returning :ok or
:unsupported.
Builds the test context, calling the suite's setup_exporter/1 when it has
one. Returns {:ok, context} the way an ExUnit setup expects.
Functions
@spec areas() :: [atom()]
The contract areas this suite checks.
@spec assert_coverage!(map()) :: :ok
Names every area this suite could not exercise, and fails instead of warning
when the case was used with require_scriptable: true.
It probes the hooks rather than depending on the other tests having run: ExUnit orders tests by seed, so a report that counted what earlier tests had recorded would report differently on different seeds.
@spec assert_description!(map()) :: :ok
E4. describe/0 returns the documented map, with numbers that make sense.
@spec assert_description_constant!(map()) :: :ok
E4. describe/0 is constant.
The caller reads it once and clamps its batch size to it. An adapter whose description moves has made that clamp a race.
E3. A provider call that blows up leaves every item in that call :uncertain.
An adapter should not raise for a provider problem, and if it does the caller
must survive it: call/2 here catches a raise, an exit and a throw alike,
because an adapter that exits is exactly as fatal to a caller that only
rescues as one that raises is to a caller that does neither.
The failure this catches is the opposite temptation: an adapter that catches
its own blow-up and reports :accepted so the batch looks clean.
E1. Three items in, three outcomes out, keyed by the ids that went in.
This is the assertion an adapter breaks by dropping a member of a batch, by answering for something it was not given, and by abandoning the return type altogether. All three are the same defect from the caller's side: it cannot tell what happened to an item it handed over.
E2. An answer the adapter could not parse is :uncertain.
Not :accepted, which would tell the caller the money arrived, and not
{:rejected, _}, which would tell it to stop. An unparseable answer is
exactly the case where the request may have been executed.
E1. One batch, two members, two different answers.
The defect this catches is an adapter that reduces over the batch and halts on the first member that is not accepted. The later members are then not merely unknown, they were never attempted, and the caller has no way to tell the difference.
E5, and core's half of invariant I15. Delivering the same item twice sends the same bytes.
The payload is the financial intent. An adapter that re-derives a field from live application state has made the retry send something the first attempt did not, under the same idempotency key, which is how one unit of usage becomes two charges.
Needs the suite's sent/1; without it nothing can see what left the process,
and the area is recorded as uncovered rather than quietly skipped.
E2. A rate limit is a retry.
This is the direct regression for the defect the outcome vocabulary exists to fix: one provider answer classified two ways by two functions in one package, with HTTP 429 terminal on the path that reports usage. A terminal rate limit either abandons revenue or, if the caller re-derives an identity for the next attempt, bills it twice.
E2. A scripted retry arrives as {:retry, delay} or {:retry, nil}, and as
nothing else.
A bare :retry, an {:error, :rate_limited} or any other invention is read
by AuroraMeter.Exporter.normalize/2 as :uncertain, which costs the caller
a reconciliation for something that was only ever a delay.
E2. A provider status nobody wrote a clause for is :uncertain.
An adapter that falls through to :accepted has turned "I do not know" into
"the money arrived", and an adapter that falls through to {:rejected, _} has
turned it into "give up".
@spec call(map(), [AuroraMeter.Exporter.Item.t()]) :: {term(), {:ok, %{optional(String.t()) => AuroraMeter.Exporter.outcome()}} | {:error, term()}}
Calls the adapter the way a correct caller must, and returns
{raw_answer, normalized}.
The guard is the point. A caller that lets an adapter's raise, exit or throw
escape has lost the outcome of every item in the call rather than recording
them as uncertain, and the difference between the three is only which
construct catches them. Build unit 04b's deliverer owes the same guard, and
the telemetry event [:aurora_meter, :exporter, :exception] beside it.
@spec items(map(), pos_integer()) :: [AuroraMeter.Exporter.Item.t()]
Builds count items for this suite, with a subject reference nothing else
will collide with and a subject kind the adapter says it supports.
@spec scenarios() :: [term()]
The scenarios script/2 may be asked for.
Asks the suite's script/2 for one scenario, returning :ok or
:unsupported.
Builds the test context, calling the suite's setup_exporter/1 when it has
one. Returns {:ok, context} the way an ExUnit setup expects.