Elasticsearch

WIP - auto-generated documentation. This can be used to imply the index document structures, but it is recommended to refer to the Elasticsearch instance directly.

Objects used to interface with Elasticsearch implementations.

class app.domain.references.models.es.AnnotationDocument(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: GenericNestedDocument

Persistence model for useful annotation fields in Elasticsearch.

annotation_type: str = InstrumentedField[annotation_type]

label: str = InstrumentedField[label]

scheme: str = InstrumentedField[scheme]

value: bool | None = InstrumentedField[value]

class app.domain.references.models.es.EnhancementContentDocument(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: GenericNestedDocument

Persistence model for enhancement content in Elasticsearch.

We define anything we want to explicitly index here and map as dynamic in the parent document.

annotations: list[AnnotationDocument] | None = InstrumentedField[annotations]

clean() → None

Automatically called when saving a document.

We utilise to provide a final barrier to prevent RAW enhancements being indexed.

enhancement_type: EnhancementType = InstrumentedField[enhancement_type]

class app.domain.references.models.es.EnhancementDocument(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: GenericNestedDocument

Persistence model for enhancements in Elasticsearch.

content: EnhancementContentDocument = InstrumentedField[content]

created_at: datetime | None = InstrumentedField[created_at]

classmethod from_domain(domain_obj: Enhancement) → Self

Create a persistence model from a domain model.

id: UUID = InstrumentedField[id]

reference_id: UUID = InstrumentedField[reference_id]

robot_version: str | None = InstrumentedField[robot_version]

source: str = InstrumentedField[source]

to_domain(reference_id: UUID) → Enhancement

Create a domain model from this persistence model.

visibility: Visibility = InstrumentedField[visibility]

class app.domain.references.models.es.ExternalIdentifierDocument(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: GenericNestedDocument

Persistence model for external identifiers in Elasticsearch.

classmethod from_domain(domain_obj: LinkedExternalIdentifier) → Self

Create a persistence model from a domain ExternalIdentifier object.

identifier: str = InstrumentedField[identifier]

identifier_type: ExternalIdentifierType = InstrumentedField[identifier_type]

other_identifier_name: str | None = InstrumentedField[other_identifier_name]

reference_id: UUID = InstrumentedField[reference_id]

to_domain(reference_id: UUID) → LinkedExternalIdentifier

Convert the persistence model into a Domain ExternalIdentifier object.

class app.domain.references.models.es.ReferenceDocument(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: GenericESPersistence[Reference], ReferenceSearchFieldsMixin

Lean persistence model for references in Elasticsearch.

This document stores only the fields needed for search operations: - visibility and duplicate_determination for filtering - search fields (title, abstract, authors, etc.) from ReferenceSearchFieldsMixin

Full reference data (including identifiers) is stored in PostgreSQL and hydrated from there when needed. Identifier lookups are done via PostgreSQL, not ES. Nested structures (identifiers, enhancements) are preserved only in the RobotAutomationPercolationDocument for percolation queries.

class Index

Bases: object

Index metadata for the persistence model.

name = 'reference'

duplicate_determination: DuplicateDetermination | None = InstrumentedField[duplicate_determination]

classmethod from_domain(domain_obj: Reference) → Self

Create a persistence model from a domain model.

to_domain() → Reference

Create a minimal domain model from this persistence model.

Since ES is now search-only, full hydration should be done from PostgreSQL. This returns a minimal Reference with only the ID and visibility.

visibility: Visibility = InstrumentedField[visibility]

class app.domain.references.models.es.ReferenceDomainMixin(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: InnerDoc

Mapping of Reference domain model to Elasticsearch document.

duplicate_determination: DuplicateDetermination | None = InstrumentedField[duplicate_determination]

enhancements: list[EnhancementDocument] = InstrumentedField[enhancements]

classmethod from_domain(reference: Reference) → Self

Create a ReferenceDomainMixin from a Reference domain model.

identifiers: list[ExternalIdentifierDocument] = InstrumentedField[identifiers]

to_domain() → Reference

Create a domain model from a ReferenceDomainMixin.

visibility: Visibility = InstrumentedField[visibility]

class app.domain.references.models.es.ReferenceSearchFieldsMixin(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: InnerDoc

Mixin to project Reference fields relevant to various search strategies.

Currently this holds fields for identifying candidate canonicals during deduplication and for searching references by query.

abstract: str | None = InstrumentedField[abstract]

The abstract of the reference.

annotations: list[str] | None = InstrumentedField[annotations]

Every true annotation on the reference.

These are in format <scheme>[/<label>].

Examples:

• classification:taxonomy:Outcomes/Stroke

• classification:taxonomy:Intervention/Climate policy instruments

• inclusion:destiny (No label)

authors: list[str] | None = InstrumentedField[authors]

The authors of the reference.

These are ordered by:

• First author

• Middle authors in alphabetical order

• Last author

evaluated_schemes: list[str] | None = InstrumentedField[evaluated_schemes]

Every scheme that has been evaluated for this reference.

Combining this with annotations allows you to determine which annotations were evaluated as false.

Examples:

• inclusion:destiny

• classifier:taxonomy:Outcomes

classmethod from_domain(reference: Reference) → Self

Create the ES ReferenceDeduplicationMixin.

classmethod from_projection(projection: ReferenceSearchFields) → Self

Create a ReferenceCandidateCanonicalMixin from the search projection.

inclusion_destiny: float | None = InstrumentedField[inclusion_destiny]

The destiny inclusion score for this reference.

This is used to apply custom thresholds for inclusion. If you just want to know if the reference was included per the default threshold, check for inclusion:destiny in annotations.

publication_date: date | None = InstrumentedField[publication_date]

The publication date of the reference.

publication_year: int | None = InstrumentedField[publication_year]

The publication year of the reference.

title: str | None = InstrumentedField[title]

The title of the reference.

class app.domain.references.models.es.RobotAutomationPercolationDocument(meta: Dict[str, Any] | None = None, **kwargs: Any)

Bases: GenericESPersistence[RobotAutomation]

Persistence model for robot automation percolation in Elasticsearch.

This model serves two purposes in order to fully define the index: a persistence layer for queries that dictate robot automation, and a percolator layer to convert domain models to queryable documents that run against said queries.

class Index

Bases: object

Index metadata for the persistence model.

name = 'robot-automation-percolation'

changeset: ReferenceDomainMixin | None = InstrumentedField[changeset]

classmethod from_domain(domain_obj: RobotAutomation) → Self

Create a percolator query from a domain model.

classmethod percolatable_document_from_domain(percolatable: ReferenceWithChangeset) → Self

Create a percolatable document from a domain model.

Parameters:

percolatable (ReferenceWithChangeset) – The percolatable document to convert.

Returns:

The persistence model.

Return type:

RobotAutomationPercolationDocument

query: dict[str, Any] | None = InstrumentedField[query]

reference: ReferenceDomainMixin | None = InstrumentedField[reference]

robot_id: UUID | None = InstrumentedField[robot_id]

to_domain() → RobotAutomation

Create a domain model from this persistence model.