AuroraMeter.
View Source
Internal. The canonical encoding behind payload_hash (core ADR 0009,
decision 2), and the validation every durable write passes before it is
allowed near a connection. Not part of the supported surface.
Build unit 03a landed the part the legacy backfill needs:
canonical_json/1, encode/1 and legacy_payload_hash/1. Build unit 03b
owns this module and added the facade's validation, the caller-id rules and
payload_hash/1 for a freshly recorded event. There is one canonical form
and it lives here; two encodings of "the same payload" is exactly the defect
the hash exists to detect.
The tuple is fixed by ADR 0009:
{feature_string, quantity, occurred_at_iso8601_usec, kind,
original_event_id, canonical_json(dimensions), canonical_json(metadata)}It is encoded as a canonical JSON array, so the bytes are the same whatever
language computes them and a hash can be reproduced by hand from a row. JSON
is self-delimiting: every element is quoted and escaped, so no field value
can impersonate the separator between two elements. encode/1 is fixed by
the hashes 03a's backfill has already written; it cannot be changed without
invalidating every payload_hash in every database that has run the V7
backfill.
Canonical JSON sorts object keys by their encoded bytes, recursively, and
refuses any term JSON cannot carry: an atom other than true, false and
nil, a tuple, a pid, a non-string object key. Floats need no check: the
BEAM has no NaN and no infinity.
Validation
validate/1 runs before the gate and before a connection is checked out, so
a malformed request costs nothing but CPU. It accumulates every failure it
finds rather than stopping at the first, because a caller fixing a payload
wants the whole list. Sizes are measured on the JSON text the caller is
sending, with byte_size/1, never on what Postgres would store: a
compressible payload of any size passes a stored-size test
(open-findings.md X104), and the limit is a promise about the request.
Summary
Types
What validate_correction/1 accepts.
What validate/1 accepts.
One validation failure: the field that failed and why.
One validation failure inside a batch, with the element's index.
Functions
The largest total encoded payload one batch may carry, in bytes.
The most elements one AuroraMeter.record_batch/2 call may carry.
The sha256 of a correction's canonical payload.
The largest caller event_id, in bytes.
The largest metadata a caller may send, measured as JSON text in bytes.
The sha256 of the canonical tuple for one validated (or read back) event.
The largest quantity a single event may carry.
Caller-id prefixes the library reserves for rows it writes itself.
canonical_json/1 without the raise: {:error, reason} for a term JSON
cannot carry.
Whether every object key in term, at every depth, is a binary.
The canonical tuple for one event, as ADR 0009 fixes it.
Validates one draft event and returns the canonical form of it.
Validates a list of drafts, then collapses repeated ids.
Validates one correction draft and returns the part of it a correction states.
Types
@type correction_draft() :: %{ :tenant_key => term(), :id => term(), :original_event_id => term(), :quantity => term(), optional(:metadata) => term(), optional(:dimensions) => term(), optional(:occurred_at) => term() }
What validate_correction/1 accepts.
A correction states three things and inherits the rest: its own caller id,
the id of the event it reduces, and the magnitude of the reduction. :metadata
is the reason, a ticket reference or an operator id. :quantity is
:remaining for AuroraMeter.replace/4, whose magnitude is not known until
the original is read under lock.
:dimensions and :occurred_at appear here only so that a caller who sends
one is told to use AuroraMeter.replace/4 rather than silently ignored.
@type draft() :: %{ :tenant_key => term(), :feature => atom(), :quantity => term(), :id => term(), :occurred_at => term(), optional(:dimensions) => term(), optional(:metadata) => term(), optional(:kind) => :usage | :correction, optional(:original_event_id) => String.t() | nil, optional(:future_tolerance) => non_neg_integer(), optional(:now) => DateTime.t() }
What validate/1 accepts.
One validation failure: the field that failed and why.
@type indexed_error() :: {non_neg_integer(), atom(), atom()}
One validation failure inside a batch, with the element's index.
Functions
@spec batch_bytes_limit() :: pos_integer()
The largest total encoded payload one batch may carry, in bytes.
@spec batch_limit() :: pos_integer()
The most elements one AuroraMeter.record_batch/2 call may carry.
@spec correction_hash(map(), pos_integer(), map()) :: binary()
The sha256 of a correction's canonical payload.
It is the same tuple payload_hash/1 computes for a usage event, with
the two fields a correction fills: kind is "correction" and
original_event_id names the fact being reduced. feature, occurred_at
and dimensions come from the original, which is why this takes the original
rather than a draft (architecture-map.md 4.2, and open-findings.md X110:
the encoding a shipped migration has already written is immutable, so a
correction reuses it and never re-encodes it).
original is any map carrying :feature, :event_id, :occurred_at and
:dimensions, which both a AuroraMeter.Schema.Event.t/0 row and an
AuroraMeter.Event.t/0 struct are.
@spec id_limit() :: pos_integer()
The largest caller event_id, in bytes.
@spec metadata_limit() :: pos_integer()
The largest metadata a caller may send, measured as JSON text in bytes.
The sha256 of the canonical tuple for one validated (or read back) event.
Accepts a map carrying feature, quantity, occurred_at, kind,
original_event_id, dimensions and metadata. feature may be an atom or
a string and kind an atom or a string, so the same function hashes a
request and a row read back out of Postgres.
@spec quantity_max() :: pos_integer()
The largest quantity a single event may carry.
@spec reserved_id_prefixes() :: [String.t()]
Caller-id prefixes the library reserves for rows it writes itself.
canonical_json/1 without the raise: {:error, reason} for a term JSON
cannot carry.
Whether every object key in term, at every depth, is a binary.
Atom keys are refused rather than stringified: %{"a" => 1} and %{a: 1}
would otherwise be the same payload, and a caller who changed one into the
other would never learn that the identity they retried under now means
something else.
The canonical tuple for one event, as ADR 0009 fixes it.
Validates one draft event and returns the canonical form of it.
The returned map carries the exact values that will be hashed and stored:
occurred_at truncated to the microsecond the column holds, dimensions and
metadata as given, their canonical JSON, the byte size of that JSON, and
the payload_hash over the whole tuple.
Every failure is {field, reason}, in validation order.
@spec validate_batch([draft()]) :: {:ok, [map()], [non_neg_integer()]} | {:error, [indexed_error()]}
Validates a list of drafts, then collapses repeated ids.
Returns {:ok, entries, plan} where entries are the distinct events to
insert, in input order of first appearance, and plan maps every input index
to the index in entries whose outcome it takes. Repeated ids with equal
payload hashes collapse; repeated ids with different hashes are an error, and
so is a batch that is too long or too large. Nothing here touches a
connection.
@spec validate_correction(correction_draft()) :: {:ok, map()} | {:error, [error()]}
Validates one correction draft and returns the part of it a correction states.
A correction is not a payload in its own right: its feature, its
occurrence instant and its dimensions are the original's, and they are not
known until the original row has been read. So this returns only what the
caller supplied, and correction_hash/3 computes the hash afterwards, inside
the transaction that holds the original under lock.
Passing :dimensions or :occurred_at is refused rather than ignored:
changing either is what AuroraMeter.replace/4 exists for, and accepting them
here would be a second way to do it with weaker guarantees.
Examples
iex> AuroraMeter.Events.Canonical.validate_correction(%{
...> tenant_key: "org_1", id: "c1", original_event_id: "e1", quantity: 0
...> })
{:error, [quantity: :not_a_positive_integer]}