aerOS Federator

Introduction 

The aerOS Federator serves as a management service responsible for controlling the establishment and maintenance of federation mechanisms among the multiple aerOS domains that form the IoT-Edge-Cloud continuum.

Features 

This component is intended to provide an additional layer of automatization above Orion-LD to avoid direct interaction with the context brokers of the continuum when it comes to federation management, as well as federated backup mechanisms for federation critical data (e.g. domains registry).

The core federation functionalities are provided by the context broker through the establishment of Context Source Registrations (CSR), which allows an Orion-LD instance to retrieve infor-mation (in NGSI-LD entities format) from another Orion-LD instances, which in the aerOS continuum means that data from one domain can be obtained just by calling the context broker of another domain once a proper registration has been performed. This capability is directly linked with the aerOS Data Fabric, but the aerOS Federator component has been designed to act as the starting point of this mechanism (domain discovery) and is only focused on the data related with the management of the continuum, which follows the aerOS continuum ontology. Finally, it also acts as the building block for the aerOS distributed domain repository.

{
  "id": "urn:ngsi-ld:SampleCSRegistration",
  "type": "ContextSourceRegistration",
  "information": [
    {
      "entities": [
        {
          "type": "Domain"
        },
        {
          "type": "LowLevelOrchestrator"
        },
        {
          "type": "InfrastructureElement"
        }
      ]
    }
  ],
  "contextSourceInfo": [
    {
      "key": "Authorization",
      "value": "urn:ngsi-ld:request"
    }
  ],
  "mode": "inclusive",
  "operations": [ "retrieveOps"],
  "endpoint": "https://upv-domain.aeros-project.eu/orion-ld",
  "hostAlias": "UPVDomain"
}

Enhancement of Orion-LD for aerOS domain federation

Queries for retrieving contextual data as NGSI-LD entities actually stored in different context brokers play a main role in the aerOS federation. Thus, in the domain of distributed systems, particularly within the context of entity federation, the pagination of entity query results stands as a significant technical challenge. This challenge escalates when the queries involve dynamic attribute values, at which point the task transitions from merely difficult to seemingly unfeasible. For effective pagination, it is imperative to establish a consistent set of resulting NGSI-LD entities across different paginated queries. Otherwise, without this consistency, the realization of pagination is not viable.

These challenges have been extensively deliberated within the ETSI ISG CIM, in the course of standardizing the NGSI-LD API. Consequently, two distinct solutions have been formulated:

Entity Maps: this solution involves freezing the set of resulting entities as a list that only includes their IDs.
Snapshots: this approach goes a step further by freezing the entities themselves, storing them within a local database.

After some technical discussions, the Entity Maps strategy has been adopted for the aerOS project in order to reduce the amount of exchanged data among the brokers, aligning with its implementation in Orion-LD for the MVP of the project.

Going into more technical details, the process begins with an initial query to Orion-LD API to obtain a set of NGSI-LD entities (GET /ngsi-ld/v1/entities?<queryParameters>) during which the Entity Map is generated and stored in the broker. The response includes the Entity Map’s ID in an HTTP header (NGSILD-EntityMap), along with the first batch of entities in the payload body. Subsequent paginated requests must provide this Entity Map ID, along with offset/limit URL parameters for pagination.

An Entity Map is structured as an array mapping entity IDs to arrays of Context Source Registration IDs, indicating the registrations associated with each entity. In the aerOS project, where attributes of entities are not distributed across multiple brokers, the registration ID arrays typically contain a single entry. Furthermore, for entities hosted locally in the broker, a special identifier (@none) is used. Finally, armed with the information of the Entity Map, the broker is fully informed to dispatch distributed requests and compile the responses into an array of entities. Upon processing all responses, the broker then furnishes this array to the original requester.

{
  "urn:ngsi-ld:entities:E1": [ "urn:ngsi-ld:registrations:R1" ],
  "urn:ngsi-ld:entities:E2": [ "urn:ngsi-ld:registrations:R6" ],
  "urn:ngsi-ld:entities:E3": [ "urn:ngsi-ld:registrations:R0" ],
  "urn:ngsi-ld:entities:E4": [ "@none" ],
  "urn:ngsi-ld:entities:En": [ "urn:ngsi-ld:registrations:Rn" ]
}