GxPlex

GXP-OBS

What an observation, a signal and an evidence state ARE in this record. Every field carries three statements: what it records, what a reader may not conclude from it, and what its absence is.

The record publishes what official sources published, when, how it relates, and what is forming — and never what should be done about it.

The third statement is the one this document exists for. A schema says which keys exist and what type each holds. It cannot say that a digest covers a normalisation rather than a document, that a null digest is an absent measurement rather than an empty document, or that a run of unchanged digests is a fact about one address and not about a document's history. Those readings are all available to a consumer whose records validate perfectly.

Versions

A later version takes a later address and every earlier one keeps resolving: a published URL is treated as a citation here. The spec version is independent of the API contract version and of a record's own schema_version — three numbers that move separately.

GXP-OBS — the published versions
Version Machine-readable What it is Standing
1 /api/v1/schema/gxp-obs-v1.json The observation record: one retrieval of one address on one day, plus the source-query records the same file carries. Published. Nothing in it is corrected, narrowed or withdrawn by any later version, and its address keeps resolving.
2 /api/v1/schema/gxp-obs-v2.json Version 1 with two record families added: the SIGNAL, one entry a publisher listed on a surface this archive reads, and the EVIDENCE STATE, what this record holds for an entry and by what act. Published. It carries version 1’s two families unchanged and supersedes nothing; the two documents resolve side by side.

The documents

Validation floor
/api/v1/schema/series.json is the JSON Schema for the observation records, and the other record types have their own under the same path. Those state shape. These documents state meaning and limits, in x-gxplex-does-not-claim and x-gxplex-absent-means on every property — annotations a validator ignores and a consumer reads.
The records themselves
/api/v1/series.jsonl, one record per line, with a header record carrying the provenance envelope and that build's own figures. observations.json carries the same records grouped by subject, signals.json the signal store, and coverage.json the eight evidence states with a census over the corpus.
One family is not served per entry
The API publishes the evidence state definitions and a census over the corpus. It publishes no evidence state on an individual entry, and a census cannot be read down to one. The limit is stated in the document as well as here.

Two families in one file

Exactly one of instrument_id and source_id is present on every record of the published series. A consumer that reads them alike will treat a query result as a document observation. That rule governs the series file; across the whole spec the families live in different stores and are told apart by their own keys, which each version's document states.

Version 1

The observation record: one retrieval of one address on one day, plus the source-query records the same file carries.

What no observation claims

  • An observation is a record of a RETRIEVAL. It is evidence about what an address returned to this archive on a day, and it is not evidence about what a document required of anyone, at any time.
  • A day with no observation is not a day on which nothing changed. Absence of a record is absence of a measurement.
  • A run of unchanged digests is not a statement that a document is stable. It is a statement that this archive's normalisation of one address did not move across the days it looked.
  • The series begins when this archive began observing, not when the document began. Nothing in it supports a claim about the document before the first record.
  • No record carries a status, a lifecycle position or an interpretation. Those live on other surfaces of this record, each with its own act and its own date, and none of them is derivable from an observation.
  • Nothing here is signed. The digests prove what these bytes were once they were retrieved and normalised here; they prove nothing about who published them.

The document observation

One retrieval of one address on one day. Keyed by instrument_id.

The document observation — fields, limits and absences
Field Type What it records What it does not claim Absent or null means
instrument_id string The id this series was recorded under — a directory name in this archive's observation store. It does not assert that the subject is a regulatory instrument, and it does not assert that a published page exists for it. This archive also observes listing pages, feed endpoints and redirects, and their series carry ids of the same shape. Absent on the source-query family, where `source_id` names the subject instead. The two families are disjoint and both are carried in one file.
source_id string The id of a queried source, on records that describe a query against a publisher API rather than a retrieval of one document. A source-query record is not an observation of a document. It records what a query returned, and nothing in it is evidence about any one document's text. Absent on the document-observation family, which carries `instrument_id` instead.
observed_date string | null The day this archive observed the address. It is not a publication date, not an effective date, not a date the document changed, and not a date the publisher did anything. It is the day a fetch was made from here. Never absent on a published observation record: a record with no day is not an observation and is not served.
observed_at string | null The observation instant, projected to the day at the publication boundary. It carries no time of day. The stored record keeps its instant; retention is not publication, and an intra-day figure would invite a reader to order events this record cannot order. Absent where the store holds no instant for the record.
sequence integer The nth observation of this subject on this day, counted in the order the store recorded them. A repeated (subject, day) pair is not duplication. This store is append-only with designed same-day revalidation, and `sequence` is what disambiguates several fetches in one day. A repeated (subject, day, digest) triple is a revalidation; only DIFFERING digests on one day are a fork. Never absent. A record with no sequence could not be addressed.
url string The address the fetch was aimed at. It is testimony about where the request went, not a statement about what answered. Where `final_url` differs, that value is the record's address and this one is the question that was asked. Absent on some source-query records, which name their subject by `source_id` and their request by `query`.
final_url string | null The address that answered. A redirect is a publisher's own statement about what supersedes what, and this field reports it. It does not say the two addresses hold the same document, and it does not characterise the relation between them. Absent where no answering address was recorded. That is a gap in this record, not evidence that the asked address answered.
redirected boolean Whether the answering address differs from the asked one. It says nothing about why, and nothing about whether the document at the two addresses is the same document. Absent where no redirect chain was recorded for the fetch.
http_status integer | null observed: 200, 202, 403, 404, 406, 500, 503, 0 The status code the response carried. A status code asserts transport success and never artifact identity: 200 does not assert that the response is the document that was asked for. `0` is not a status — it is the browser path, which records no HTTP code, and reading it as a failure would be reading a method as an outcome. Null where no response was received at all. That is an unmeasured result and not a negative one: it does not report that the document is gone.
fetch_method string observed: direct, browser, api, blocked, failed, no_adapter, unobserved_no_credential How the fetch was made. The values are not a ranking and not a quality scale. `blocked` records how a host answered this client on that day; it is a measurement of a transport, not a judgement of a publisher. The list is what the store holds today, not a closed set this record claims for the future. Absent where no fetch was attempted for the record.
content_hash string | null SHA-256 of the NORMALISED text of the retrieval, where text was retrieved. A normalised text is not the document. The digest covers what the normaliser produced from the response, so two equal digests are two equal normalisations and not two identical documents, and two differing digests may differ over something the normaliser kept and a reader would not have noticed. The digest is also not a signature: it proves what these bytes were, never who published them. Null means nothing was retrieved to hash. It does not mean the document was empty, and it does not mean the document was unchanged. An absent digest is an absent measurement.
previous_hash string | null The digest this record's digest was compared against. It names the comparison this record made, not the previous state of the document. Where `series_restarted` is present the comparison crosses a referent change and the two digests are of different things. Null on the first record of a series and after a restart. Null is not "no earlier observation exists" — it is "this record compared against nothing".
changed boolean Whether this record's digest differs from the one it was compared against. It is a fact about two normalisations of one address. `false` does not assert that the document did not change: a change outside the normalised text, a change and a reversion between two observations, or any change on a day with no observation, is invisible to it. `true` does not assert that the publisher revised anything — a template, a date stamp or a session token in the response can move a digest. Absent where no digest was taken. Absence is not `false`: nothing was compared.
content_length integer Characters of normalised content. It measures the normalisation, not the document, and it is not a measure of how much a document says. Written as 0 wherever no digest was taken. A 0 here is the absence of a measurement and never a measured empty document.
content_type string observed: text/html, application/pdf, application/json, The media type the response declared. It is the publisher's own declaration, reported. It was not verified against the bytes, and the empty string is a response that declared none rather than a response that was empty. Absent where no response was received.
raw_bytes integer | null Bytes in the response as received. It is the size of a response on a day, not the size of a document. A refusal page and a document are both responses. Absent where no response was received.
fetch_duration_ms integer How long this fetch took, in milliseconds. It is a fact about one network path on one day, not a property of the publisher and not a measure of a service. Absent where no fetch completed.
error string | null observed: HTTP 403, HTTP 404, HTTP 500, TimeoutError, URLError, no adapter The transport error, where one occurred. An error is an UNMEASURED result and never a negative one. It does not report that the document is gone, that it is unchanged, or that the publisher refused this record specifically — it reports that no measurement was obtained. Null where the fetch returned without a transport error.
ia_url string | null A third-party archive copy of this retrieval, where one was made. It is not this record's copy and this record does not warrant it. A value here says a submission was made and answered, not that the copy still resolves. Null where no copy was made from here. It is not evidence that no third-party copy exists.
ia_submitted_at string | null The day the third-party archive submission was made. It records a submission, not a successful capture. Absent where no submission was made.
ia_unmeasured object Why the third-party archive submission produced no measurement, with the cause and the day. It is the reason a measurement is missing. It is not a failure of the publisher and not a property of the document. Absent where the submission was measured, either way.
normalised_text_path string | null Where the normalised text of this retrieval is retained in this archive. Retention is not publication: the file is not served, and a path here is not an address a consumer can fetch. It is what makes a digest checkable HERE. Absent where nothing was retained. That is not evidence that the document has no text — restricted documents are observed and hashed and their text is never published.
normalised_from string observed: pdf-text What the normalised text was extracted from, where extraction was not from the response body directly. It names the extraction path, not the fidelity of the extraction. Text taken out of a PDF is a reading of a layout. Absent where the normalisation was taken from the response as received.
series_restarted object That this record begins a new comparison series, with the cause — the referent this archive hashes for the subject moved. A restart is a change in what this record MEASURES, never a change in the document. A digest before a restart and a digest after it are digests of different things, and comparing them across the boundary would report a document change that did not happen. Absent on every record that continues its series.
referent_assertion object The check that the bytes retrieved are the bytes of the subject this record names — a marker sought in the response, a minimum size, and a verdict. It confirms that a response is the artifact that was asked for. It does not confirm that the artifact is current, correct or complete, and its absence on a record is not evidence that the referent was wrong. Absent on records taken before the check existed and on records where no marker is declared for the subject.
rescoped_from string The address this subject was observed at before a re-scope. A re-scope is a correction to what this archive tracks. It says nothing about the publisher. Absent on every record of a subject that was never re-scoped.
rescoped_on string | null The day the subject was re-scoped. It is a date in this archive's history, not the publisher's. Absent on every record of a subject that was never re-scoped.
rescope_reason string Why the subject was re-scoped, in this archive's own words. It is this archive's account of its own change. It is a lead and never evidence about the document: evidence is attributable to a voice that is not this one. Absent on every record of a subject that was never re-scoped.
chrome_excluded array Named page furniture removed before normalisation, so a banner that changes daily does not read as a document change. It records what the normaliser dropped. It does not assert that what remains is the document, and a digest is a digest of what survived this list. Absent where nothing was excluded for the subject.
schema_version integer The version of the RECORD SHAPE this line was written under. It is not the version of this specification, and not the version of the API. Three version numbers exist here and they move independently. Absent on records written before the field existed.

The source query

One query put to a publisher API on one day. Keyed by source_id, carried in the same file, and not an observation of any document.

The source query — fields, limits and absences
Field Type What it records What it does not claim Absent or null means
query string The query put to a publisher API. It records what was asked, not what exists at the publisher. Absent on the document-observation family.
returned integer How many records the query returned. It counts a response. It is not a count of what the publisher holds, and a zero is a zero for THIS query on THAT day. Absent on the document-observation family.
total_available integer | null The total the publisher reported for the query. It is the publisher's own figure, reported. It was not verified here. Absent where the publisher reported no total.
new_since_last_run integer How many returned records this archive had not seen before. New TO THIS ARCHIVE. It is not a count of what the publisher newly issued, and a record can be new here because a query changed. Absent on the document-observation family.
new_record_ids array The publisher's identifiers for those records. The identifiers are the publisher's, carried unaltered. Absent on the document-observation family.
api_version string The API version the publisher reported. Reported, not verified. Absent where the publisher reported none.
api_last_updated string | null The last-updated value the publisher reported. It is the publisher's statement about its own data, reported. It is not an observation of a document. Absent where the publisher reported none.
licence_class string observed: open, restricted, unknown What this archive recorded of the terms the queried source states. It describes this archive's own handling and never what a publisher permits. It is not a licence and not a grant. Absent where no reading is recorded.
jurisdiction string The jurisdiction the queried source belongs to. A fact about the source, not about any document it returned. Absent on the document-observation family.
sector string observed: cross_sector, pharma, medical_device The sector the queried source covers, as recorded here. It describes the SOURCE this archive queried. Nothing here classifies any document by product class, by lifecycle position or by reach: that is a judgement, and this record does not make one. Absent on the document-observation family.
source_type string observed: api How the source is read. It names the access path, not the authority of the source. Absent on the document-observation family.
content_chars integer Characters in the normalised response to the query. It measures a response, not a corpus. Absent where the response was not normalised.

Diagnostic annotations

Present on two records of the published series. They are declared rather than left to a permissive schema because one of them is named in a way a consumer will misread.

Diagnostic annotations — fields, limits and absences
Field Type What it records What it does not claim Absent or null means
status integer observed: 200, 500 An HTTP status code, recorded under this name on two diagnostic records. It is NOT a document status, a lifecycle position or a verification state. It is a transport code, and this record publishes document statuses on other surfaces with their own acts and dates. Read `http_status` for the transport code of every ordinary record. Absent on every record except the two that carry a diagnostic note.
finding string observed: http_500, spa_redirects_to_root A short machine token naming what a diagnostic read established about an address. It is this archive's own account of a retrieval problem. It is a lead about this archive's reach, never evidence about the document or the publisher. Absent on every ordinary observation.
note string The same finding in prose, in this archive's own words. This archive's own prose is reported class by definition, however accurate. It is a lead and never evidence: what counts as evidence here is attributable to a voice that is not this one. Absent on every ordinary observation.
method string observed: browser How the diagnostic read was made, recorded under this name on two records. It is the same kind of value as `fetch_method` under a second name on two records. Two names for one fact is a defect in the store and it is reported here rather than tidied away, because a spec that hid it would leave a consumer to discover it. Absent on every ordinary observation.

Version 2

Version 1 with two record families added: the SIGNAL, one entry a publisher listed on a surface this archive reads, and the EVIDENCE STATE, what this record holds for an entry and by what act.

What no signal claims

  • A signal is a record that a publisher LISTED something on a surface this archive read, on a day. It is not a change, not a revision, not an obligation and not a deadline.
  • A signal attached to a document is not evidence that the document moved. Document change is measured by digests over retrievals of the document itself, on a different record family in this same spec.
  • A count of signals is a count of listings this archive read. It is not activity, not attention and not importance, and nothing here ranks one publisher, class or period against another.
  • A day with no signal is not a day on which no publisher listed anything. It is a day this archive read no listing carrying one.
  • The class vocabulary is the one filed for this record. A publisher's own word that names none of it is carried as the publisher's word and is never mapped onto the nearest member.
  • Distance from the source is a position on a stated scale. It is not a reliability rating, and this record publishes none.

What no evidence state claims

  • An evidence state says what this record holds for an entry and by what act. It says nothing about the document — not its standing, not its force, not its quality and not whether it is current.
  • The eight are ordered by the PRECEDENCE the classifier applies, which is not a ranking. Sorting them and reading the result as a grade is the reading this document exists to refuse.
  • "Verified by a person" is an act performed on a recorded day. It is not a property the document acquired and it does not expire into falsehood; it ages, and the date is served beside it.
  • The evidence state is the SECOND axis. The publisher's own assertion about the document is the first, and `unknown` remains an honest value of it. Neither axis is derivable from the other.
  • A count of entries in a state is a count of this archive's holdings. It is not coverage of anything outside this archive, which is a population this record has never enumerated.

The signal

One entry a publisher listed on a surface this archive reads. It is not an observation of a document and it is not a change.

The signal — fields, limits and absences
Field Type What it records What it does not claim Absent or null means
id string | null The identifier the collector recorded for this entry. It is not the publisher's identifier for the document, and two signals carrying one id are not two readings of one document. The record cannot be addressed, cited or de-duplicated. It is counted apart from the records that can be, and it is not dropped.
source string The collector whose store this record was appended to. It names a store in this archive, not the publisher. Two sources may read one publisher and one source may read several. A record with no source is a record this reader cannot place.
source_url string | null The address the publisher gave for the entry. It is not evidence that the address was requested. Read `source_url_provenance` before treating it as a retrieval: an address declared inside a listing and never fetched is the ordinary case. The listing carried no address for the entry.
source_url_provenance string | null Whether the address above was answered by a request, or only declared by the publisher inside the listing. A declared address is a publisher's claim about where something lives. It is not a status code and it is not an identity check. A store written before the collector recorded the distinction.
published string | null The day the publisher states for the entry. It is not the day this archive read it, not the day the entry appeared on the surface, and not a date on which anything took effect. The publisher stated no date, or stated one this reader could not reduce to a day. Both are an absent day here.
retrieved_at string | null The day this archive read the surface the entry was on. It dates the READ, not the entry. An entry read today may have been listed for years, and nothing here says when it appeared. No retrieval day is recorded, which is an absent measurement rather than an entry that was never read.
title string | null The publisher's own title for the entry, verbatim, under the word ceiling this record publishes. A title is what a publisher called something. It is not a description of the document, not a summary and not this record's words. Either the publisher gave none, or it runs over the ceiling and is WITHHELD WHOLE rather than trimmed — a truncated quote is a misquote. `title_over_quote_cap` says which.
title_over_quote_cap boolean | null The collector's own verdict, taken at write time from the one home that holds the ceiling. True does not mean the title is unavailable. It means this record does not publish it, and the entry's own address stands in its place. A store written before the collector recorded the verdict. A reader may then count the words itself, and says that it did.
content_hash string | null A digest of the entry as it was read. It digests the ENTRY as listed, never the document the entry points at. Two equal digests are two identical listings, which is a fact about a listing page. No digest was taken. It is not an unchanged entry.
kind_all array Every class value the publisher stated for this entry, verbatim and in the publisher's own document order. These are the publisher's words, not a taxonomy. Nothing maps them to one another and nothing here says two publishers using one word mean one thing. Empty is a publisher that stated none. The field being absent is a store written before the collector recorded them, and the two are not the same fact.
signal_class string | null The class, where the publisher stated one that is in the filed vocabulary and the record's own state agrees with its own value. It is the publisher's statement carried through, never this record's reading of what an entry is about. No class is inferred from a title, a designator or a numeral. Read `signal_class_state`. Five readings produce a null here and they are different facts, two of them facts about the RECORD rather than about the publisher.
signal_class_state string Which reading produced the class: the publisher stated one in the vocabulary, stated one outside it, stated none, the record carries no state at all, or the record's own state and value contradict each other. `no-class-self-stated` is a publisher that named none on the surface read. It is not a statement that the entry has no class. A store written before the field existed. Nobody asked, which is not the same as a publisher stating none.
signal_class_basis string | null The collector's own account of how it reached the verdict. It is a derivation, not a citation. The evidence is the publisher's own tokens in `kind_all`. The collector recorded no account.
authority string | null The host the entry was published on. A host is not a body and is not a person. No author, editor or contributor element is read into any field here, ever, and a host cannot be one. No host could be read from the address.
instrument_id string | null The tracked entry the collector matched this signal to, by the publisher's own stated identifiers. A match is a fact about identifiers. It is not a claim that the signal is ABOUT the document, and it is not a claim that the document changed. The signal matched no tracked entry: an official mention with no subject in this corpus. It is not a failed match and it is not a signal about nothing.
attached_source_ids array Tracked entries named by every address the publisher declared for this entry. A SUPERSET of `instrument_id`, never a replacement for it. An address naming an entry is not the entry being the subject. A consumer that read the first element as the subject would be choosing one the collector declined to choose. Empty is a publisher whose declared addresses name no entry this corpus holds. The field being absent is a store written before it existed.
addresses_declared integer | null How many addresses the publisher declared for the entry. It counts declarations, not retrievals. None of them was necessarily requested. The record carries no figure. It is never coalesced to a number.
distance_from_source integer | null Distance from the source on the filed scale: 0 primary, 1 official notice, 2 official related record, 3 third-party. It is a position, not a quality. A distance of 3 says a record is third-party; it says nothing about whether it is accurate, and nothing here ranks the four. NULL IS NOT ZERO, and zero is the strongest claim this scale makes — that the record IS the primary document. A null is an entry that declared no address to measure from, and this store writes no zero at all.
distance_from_source_label string | null The scale's own word for the value beside it. The label is the scale's, not the publisher's. Where a record's own label disagrees with the scale, a reader withholds rather than reconciling: neither side adjudicates the other. No distance was read, so there is no label to carry.
origin string | null Whether the record came from a publisher over the wire, or from a synthetic fixture authored in this repository. The writer refuses to append a fixture-sourced record to the live store. This field is a disclosure at the surface and not a second fence: it exists for the day a file is placed in the store by a hand. The record declares no origin, and NOTHING HERE ASSUMES THE SAFE CASE. A reader states the absence rather than treating it as a wire read.
surface_class string | null What kind of publisher surface the entry was listed on. It classes the SURFACE this archive read, not the document and not the entry. The collector recorded no class for the surface.

The evidence state

What this record holds for an entry and by what act — axis two. The objects are the state definitions and the census rows that carry them.

The evidence state — fields, limits and absences
Field Type What it records What it does not claim Absent or null means
key string The evidence state's own identifier — one of the eight this record classifies entries into. The eight are states of THIS RECORD'S HOLDINGS. None of them is a statement about a document, its quality, its standing or its force. A state with no key is not a state this record can be asked about.
label string The reader's words for the state. A label is not a grade. "Verified by a person" is an act this record performed on a day, not a property the document has. No reader-facing wording is published for the state.
states string One sentence saying what this record holds when an entry is in this state, and nothing about the document. It describes a holding. It does not say what a reader should conclude, and it is not a confidence statement. No sentence is published for the state, which would leave a consumer to infer one — the inference this document exists to remove.
from_the_filed_order boolean Whether the state is one the filed order names, or one this record added because the order's list did not partition the world. An added state is not a lesser state. An order's enumerated options are a hypothesis about the world, and a fact rounded onto the nearest listed option is a fact this record would have got wrong. The provenance of the state is not recorded.
count integer | null On the axis census: how many entries of the stated population this record classifies into the state. A count over the corpus is not a per-entry field. This record serves no evidence state on an individual entry through the API today, and a consumer cannot derive one from a census. The object is a state DEFINITION rather than a census row. A zero on a census row is a measured zero and keeps its place, because a state that emptied and a state nothing was ever classified into wear one number.

Consuming this record: developers. Method: how this record is maintained. Found an error? Write to corrections@gxplex.com.

Last updated: