Robot Automations

        sequenceDiagram
    Actor RO as Robot Owner
    participant R as Robot
    participant DR as Data Repository
    participant ES as Elasticsearch

    RO->>+DR: POST /robot/<robot_id>/automation/ : percolator query
    DR->>-ES: Register percolator query

    alt On Import Batch
        DR->>DR: Ingest References
        DR->>ES: Percolate new References
        loop For each matching robot
            DR->>R: Batch Enhancement Request with matching References
        end
    else On Batch Enhancement
        DR->>DR: Ingest Enhancements
        DR->>ES: Percolate new Enhancements
        loop For each matching robot
            DR->>R: Batch Enhancement Request with matching References
        end
    else On Single Enhancement
        DR->>DR: Ingest Enhancement
        DR->>ES: Percolate new Enhancement
        loop For each matching robot
            DR->>R: Single Enhancement Request with matching Reference
        end
    end
    

        flowchart LR
    G_R([Reference]) --> G_R1[Ingest Reference]
    G_R1 --> G_P[(Persistence)]
    G_R1 --> G_AUTO{Robot Automation Percolation}
    G_AUTO --> G_ROBOT[["Robot(s)"]]
    G_ROBOT --> G_R2[Ingest Enhancement]
    G_R2 --> G_P
    G_R2 --> G_AUTO
    

Context

Robot automations allow Batch Enhancement Requests to be automatically dispatched based on criteria on incoming references or enhancements. This is achieved through a percolator query registered by the robot owner in the data repository using the /robot/<robot_id>/automation/ endpoint.

A batch of imports or enhancements will be processed together, meaning that automated robots will receive a single batch request containing all references or enhancements that matched the automation criteria.

Percolation

The automation criteria is implemented as an Elasticsearch percolator query. Percolation is the inverse of a traditional Elasticsearch search: the query is stored in the index, and the document is used to search. When writing a percolator query, the key question is: “What shape should new references and/or enhancements have to automatically request enhancements from this robot?”.

Query context is implicit when the percolator query is registered - i.e. the top-level element of RobotAutomationIn.query should not be query.

Importantly, the percolator query matches on changesets. On a reference import, this is of course the entire reference, but on an enhancement import, it is the enhancement itself. The query may therefore need to handle both cases, as in the example below. It is guaranteed that only one of reference or enhancement will be provided for each percolation document.

Safeguards

There is a simple cycle-checker in place to prevent a batch enhancement request from triggering an automatic enhancement request to the same robot.

Cycles involving multiple robots are however possible, so caution should be taken when considering robot automation criteria.

Examples

The following examples are used in DESTINY to orchestrate robot automations.

Request Missing Abstract

This percolator query matches only on new references that do not have an abstract, and that do have a DOI (as the abstract robot requires DOIs to function).

{
    "bool": {
        "must": [
            {
                "nested": {
                    "path": "reference.identifiers",
                    "query": {
                        "term": {"reference.identifiers.identifier_type": "DOI"}
                    },
                }
            }
        ],
        "must_not": [
            {
                "nested": {
                    "path": "reference.enhancements",
                    "query": {
                        "term": {
                            "reference.enhancements.content.enhancement_type": "abstract"
                        }
                    },
                }
            }
        ],
    }
}

Request Domain Inclusion Annotation

This percolator query matches on new references that have an abstract, or new enhancements that are abstracts. This is an example of how the orchestration starts to piece together - if the above automation is executed, and an abstract is created, this automation will then be triggered.

{
    "bool": {
        "should": [
            {
                "nested": {
                    "path": "reference.enhancements",
                    "query": {
                        "term": {
                            "reference.enhancements.content.enhancement_type": "abstract"
                        }
                    },
                }
            },
            {
                "term": {
                    "enhancement.content.enhancement_type": "abstract"
                }
            }
        ],
        "minimum_should_match": 1,
    }
}