############################# aerOS installation ############################# .. contents:: :local: :depth: 3 .. note:: All the used container images and Helm charts are only available for the aerOS project partners, but all this content will be publicly delivered in the near future under an open-source license. The formal description of the aerOS installation procedure (without technical aspects) is available in the section "3.2 aerOS installation" of "D5.2 Integration, evaluation plan and KPIs definition". This guide doesn't aim to cover the installation of the Computing Resource that will become aerOS Infrastructure Elements. For instance, the installation of K8s clusters and Docker. Please, **bear in mind that aerOS domains can be of K8s nature or just Docker environments**. This is because the only mandate is for them to be able to run containerised workloads. Please, select the option that suits your particular case better. .. note:: All the content enclosed in **<>** are just placeholders, so please, replace them with the appropiate value Prerequirements ================ Kubernetes ------------- - Create the *common-deployments secret* to be able to pull all the aerOS container images: .. code:: bash kubectl create secret docker-registry aeros-common-deployments --docker-server=registry.gitlab.aeros-project.eu --docker-username= --docker-password= - Add common-deployments (named as *aeros-common*) Helm chart repository: .. code:: bash helm repo add --username --password aeros-common https://gitlab.aeros-project.eu/api/v4/projects/65/packages/helm/stable Docker ----------------------- - Docker login to aerOS Gitlab registry to be able to pull all the aerOS container images: .. code:: bash docker login registry.gitlab.aeros-project.eu -u -p 1. Entrypoint domain ===================== 1. Install Orion-LD Context Broker ----------------------------------- Kubernetes ~~~~~~~~~~ - In K8s install the Helm chart setting the *DomainID* parameter: .. code:: bash helm install orion-ld aeros-common/orion-ld --set broker.image.repository=registry.gitlab.aeros-project.eu/aeros-public/common-deployments/orion-ld --set broker.image.tag=1.0.4-mvp --set broker.imagePullSecrets[0].name=aeros-common-deployments --set broker.args.brokerId= --set broker.service.ports.api.nodePort=31550 --set broker.args.traceLevel="70-99" .. note:: - You can directly reach (without passing through KrakenD) the Orion-LD API through http://k8s-master-node-ip:31550 for testing purposes Docker ~~~~~~~~~~ - In Docker machines, get the *docker-compose.yaml* file and then edit it to set the *DomainID* parameter: .. code:: bash wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/orion-ld/1.0.0/docker-compose.yaml .. note:: - You can directly reach (without passing through KrakenD) the Orion-LD API through http://k8s-master-node-ip:31550 for testing purposes 2. Install Keycloak and OpenLDAP ---------------------------------- 2.1 Install OpenLDAP ~~~~~~~~~~~~~~~~~~~~~ - In K8s, add the official Helm chart repository and then install the Helm chart: .. code:: bash helm repo add helm-openldap https://jp-gouin.github.io/helm-openldap/ .. code:: bash helm install openldap helm-openldap/openldap-stack-ha --set ltb-passwd.ingress.enabled=false --set phpldapadmin.ingress.enabled=false --set phpldapadmin.service.type=NodePort --set persistence.size=100Mi --set replication.enabled=false --set replicaCount=1 --debug 2.2 Create users, roles and organizations in OpenLDAP ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Currently it must be done in the *phpldapadmin* dashboard. In the future, it'll be able to be done using the Portal. - List of defined roles in aerOS according to D3.1 and D3.2: - System administrator - Domain administrator - Vertical user - IoT service deployer - Data producer - To access to this dashboard, you have to port-forward the *openldap-phpldapadmin* service: .. code:: bash kubectl port-forward service/openldap-phpldapadmin 81:80 and then access to it via http://localhost:81 - Now, **first**, create these *organizational units* (Click on the name -> Create child entry -> New Organizational Unit): - *groups* - *roles* - *users* 2.3 Install Keycloak ~~~~~~~~~~~~~~~~~~~~~~ .. warning:: If Keycloak will be exposed behind a reverse proxy (e.g. through a K8s Ingress), you must set the PROXY_ADDRESS_FORWARDING env var to *true* .. warning:: Take into consideration that Keycloak must be reachable by the user's browser when accessing to the Management Portal. Therefore, it can be exposed through a K8s ingress, through a NodePort or exposing the Docker container port to the host machine. It will depend on the specific needs or network configuration of the pilot. Kubernetes ^^^^^^^^^^^ - Expose via NodePort: .. code:: bash helm install idm aeros-common/idm --set keycloak.service.ports.keycloak.nodePort= --debug - Expose via Ingress (default ingress configuration, please check the values): .. code:: bash helm install idm aeros-common/idm --set keycloak.envVars.proxyAddressForwarding=true --set keycloak.ingress.enabled=true --debug Docker ^^^^^^^ .. code:: bash wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/idm/1.0.0/docker-compose.yaml .. code:: bash docker compose up -d 2.4 Keycloak configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Access to Keycloak URL with the admin credentials (by default admin/Pa55w0rd). 2. Create a new Realm. 3. Go to *User federation* -> *Add Ldap providers* -> Fill in the form with the proper OpenLDAP configuration. 4. Create a new Client for the Management Portal. .. note:: This Client configuration will be used for the Management Portal installation. 3. Create Domain and Organization entities -------------------------------------------- - The Domain entity creation will be automatically done by the aerOS Federator in the future. - The User, Role and Organization entities will be automatically created by the LDAP Collector, which will be included in the Data Fabric. - Until the LDAP Collector flow is ready, at least, create an Organization and a Domain entity in Orion-LD: .. code:: bash curl --location 'http:///ngsi-ld/v1/entities' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:ngsi-ld:Organization:", "type": "Organization", "name": { "type": "Property", "value": "" } }' .. code:: bash curl --location 'http:///ngsi-ld/v1/entities' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:ngsi-ld:Domain:", "type": "Domain", "description": { "type": "Property", "value": "This is the first example domain from UPV" }, "publicUrl": { "type": "Property", "value": "" // -> it MUST be the public URL of the domain which will point to the KrakenD API Gateway }, "owner": { "type": "Relationship", "object": ["urn:ngsi-ld:Organization:"] }, "isEntrypoint": { "type": "Property", "value": true }, "domainStatus": { "type": "Relationship", "object": "urn:ngsi-ld:DomainStatus:Functional" } }' 4. Install KrakenD with the proper configuration -------------------------------------------------------- Kubernetes ~~~~~~~~~~ Install the Helm chart. Some additional guidelines will be provided to add additional endpoints to the KrakenD configuration (e.g. a pilot wants to expose an internal/custom non-aerOS API via KrakenD): .. code:: bash helm install api-gateway aeros-common/api-gateway --set krakend.config.keycloakUrl= --set krakend.config.keycloakRealm= --debug Docker ~~~~~~ First download and configure the krakend.json file .. code:: bash wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/api-gateway/1.0.0/krakend.json Download the Docker compose file and edit it to include the *krakend.json* file as a volume .. code:: bash wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/api-gateway/1.0.0/docker-compose.yaml Finally, run the Docker Compose file: .. code:: bash docker compose up -d 5. Install Self-* modules in all the IEs of the domain -------------------------------------------------------- .. note:: Currently, use the container images of the Self-awareness with the tag *1.1.0-legacy-entity-id* .. warning:: Before installing the Self-* modules, check the Orion URL in Docker compose file or in the chart's values file. Kubernetes ~~~~~~~~~~ Check the values files and install the charts: .. code:: bash helm install self-awareness aeros-common/self-awareness --set hardwareinfo.image.tag=1.1.0-legacy-entity-id --set powerconsumptionarm64.image.tag=1.1.0-legacy-entity-id --set powerconsumptionamd64.image.tag=1.1.0-legacy-entity-id --debug .. code:: bash helm install self-orchestrator aeros-common/self-orchestrator --debug Docker ~~~~~~ 1. Download the Docker compose files .. code:: bash wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/self-awareness/1.0.0/self-awareness_docker-compose.yaml .. code:: bash wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/self-orchestrator/1.0.0/self-orchestrator_docker-compose.yaml 2. Edit the Docker Compose files with the proper configuration parameters. 3. Install it: .. code:: bash docker compose up -d 6. Install the Management Portal -------------------------------------------------------- .. warning:: Take into consideration that the Management Portal must be reachable by the user's browser. Therefore, it can be exposed through a K8s ingress, through a NodePort or exposing the Docker container port to the host machine. It will depend on the specific needs or network configuration of the pilot/environment. First, check the Keycloak's Realm and Client that you created in `step #2.4 <#keycloak-configuration>`__ for the management portal because they are needed for the installation. Keep in mind that the installation must be done to align the access URL of the portal with the "redirect URIs" that you configured in #2.4.4. However, if this URL changes or even you want to add more URLs, you can update the configuration of the Client in Keycloak and it will be transparent for the Management Portal because the Portal only needs to know the URL, the Realm and the Client that you've configured in Keycloak. Detailed instructions will be provided as this is the most "manual" or different step for each continuum installation. Kubernetes ~~~~~~~~~~ The installation is done using Helm chart, so first, check the *frontend.keycloak* and *backend.envVars.keycloakUrl* objects of the values file to edit it if needed: - Expose via NodePort: .. code:: bash helm install management-portal aeros-common/management-portal --set frontend.service.ports.web.nodePort= --debug - Expose via Ingress (default ingress configuration, please check the values): .. code:: bash helm install management-portal aeros-common/management-portal --set frontend.ingress.enabled=true --debug Docker ~~~~~~ 1. Create a *config* folder, download the *keycloak.json* and *managementportal.conf* files inside it. Then edit both files. .. code:: bash wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/management-portal/1.0.0/keycloak.json wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/management-portal/1.0.0/managementportal.conf 2. Download the Docker compose file and edit it. .. code:: bash cd .. wget --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/management-portal/1.0.0/docker-compose.yaml 3. Run the Docker compose file. .. code:: bash docker compose up -d 7. Install HLO and Redpanda -------------------------------------------------------- 7.1 Redpanda ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. warning:: - DO NOT use a virtualized CPU (e.g. kvm64) because the Redpanda installation will fail. - For machines with SELinux enabled, it is needed a customization of the official Helm chart. Kubernetes ^^^^^^^^^^ Install the official redpanda chart using these custom values, so save these values into a file named *aeros-values.yaml*: .. code:: yaml statefulset: initContainers: setDataDirOwnership: enabled: true replicas: 1 tls: enabled: false monitoring: enabled: false scrapeInterval: 30s storage: persistentVolume: size: 500Mi If Prometheus has been previously installed in the K8s cluster, you can enable the monitoring of Redpanda by setting *monitoring -> enabled: true* .. code:: bash helm repo add redpanda https://charts.redpanda.com helm install redpanda redpanda/redpanda -n redpanda --create-namespace -f aeros-redpanda-values.yaml --debug Docker ^^^^^^ Check the official documentation: https://docs.redpanda.com/current/get-started/quick-start/ 7.2 Topics creation in Redpanda ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ List of used topics by the HLO: - allocator2deployment - data2allocator - fe2data There are **two options** to create the needed topics for the HLO in Redpanda: 1. *port-forward* the port 8080 of the *redpanda-console* pod or service, and then access to it via http://localhost:forwarded-port (in this command the forwarded-port is 8081). Finally, go to Topics -> Create topic. .. code:: bash kubectl port-forward service/redpanda-console 8081:8080 2. Create them using these commands. You can access to the console to check if they have been created: - K8s: .. code:: bash kubectl exec -it redpanda-0 -n redpanda -- rpk topic create allocator2deployment kubectl exec -it redpanda-0 -n redpanda -- rpk topic create data2allocator kubectl exec -it redpanda-0 -n redpanda -- rpk topic create fe2data - Docker: .. code:: bash docker exec -it redpanda-0 rpk topic create allocator2deployment docker exec -it redpanda-0 rpk topic create data2allocator docker exec -it redpanda-0 rpk topic create fe2data 7.3 HLO ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Currently, all components can be installed using an *umbrella* Helm chart which installs all the Helm charts of the HLO components in the same K8s cluster. It's available in the `Helm chart repository `__ of the common-deployments Gitlab repository. .. code:: bash helm install hlo aeros-common/hlo-deployments --debug In the future, more detailed instructions will be provided to install each HLO component separately, and for the HLO installation in Docker machines. 8. Install LLO(s) -------------------------------------------------------- In aerOS, the LLOs are based on the K8s Operator Pattern, which provide a framework to automatically control the lifecycle management of resources. This does not mean that LLOs only support the management of K8s workloads, but that actually leverage this widely used and tested resource management framework to finally reflect the desired state of services (to be deployed in a certain IE), with a common way to express the Implementation Blueprints (custom K8s Custom Resources) that is independent on the container management framework used in the selected IE, thus agnostic for the HLO. .. warning:: As aerOS LLOs are based on K8s Operators, the LLO API and the LLO Operators (LLO K8s, LLO Docker, ...) must be installed in K8s clusters. 8.1 Prerequirements ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. warning:: - Remember that each domain must contain at least one K8s cluster to be able to run the LLO Operators. - At least one LLO must be installed in each domain. Install the LLO API: .. code:: bash curl --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/llo/1.0.0/cr-api-deployment.yaml | kubectl apply -f - 8.2 NGSI-LD entities ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The automatic creation of *LowLevelOrchestrator* entities and linking to the underlying InfrastructureElement* entities (value *lowLevelOrchestration* attributes) of the domain is still pending. It will be performed by the LLO itself using the *containerTechnology* attribute of the *InfrastructureElement* entities. Therefore, currently, please first manually create the LLO entities (one for each LLO in the domain) in Orion-LD before installing the LLO(s): .. code:: text curl --location 'http:///ngsi-ld/v1/entities' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:ngsi-ld:LowLevelOrchestrator:", // -> it MUST MATCH with the "lowLevelOrchestrator" attribute of the all IE entity to be orchestrated by this LLO. "type": "LowLevelOrchestrator", "description": { "type": "Property", "value": "LLO example of type Kubernetes" }, "domain": { "type": "Relationship", "object": "urn:ngsi-ld:Domain:" // -> it MUST MATCH with the ID of the domain that you're installing (same as in step #3) }, "orchestrationType": { "type": "Relationship", "object": "urn:ngsi-ld:OrchestrationType:Kubernetes" // -> Container management technology (Docker or Kubernetes) } }' then, update the *lowLevelOrchestrator* attribute of all the IEs managed by the created LLOs: .. code:: bash curl --location --request PATCH 'http:///ngsi-ld/v1/entities/?local=true' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "lowLevelOrchestrator": { "type": "Relationship", "object": "urn:ngsi-ld:LowLevelOrchestrator:" } }' 8.3 Install LLO K8s ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Install the LLO K8s Operator in the K8s cluster: .. code:: bash curl --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/llo/1.0.1/k8s-operator-sdk-deployment.yaml | kubectl apply -f - 8.4 Install LLO Docker ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The guidelines for installing the LLO Docker will be fine-tuned in the near future. 1. Install the LLO Docker Operator (in the K8s cluster) .. code:: bash curl --header "PRIVATE-TOKEN: " https://gitlab.aeros-project.eu/api/v4/projects/65/packages/generic/llo/1.0.0/docker-operator-deployment.yaml | kubectl apply -f - 2. Deploy the `Docker API `__ **in each Docker IE** of the Domain 3. Create a K8s CR of type *DockerInfrastructureElement* for each Docker IE. This must be done in the same K8s cluster in which the LLO Docker Operator has been installed in order to link the Operator with the Docker API of the Docker IE. .. code:: yaml apiVersion: llo.aeros-project.eu/v1alpha1 kind: DockerInfrastructureElement metadata: labels: app.kubernetes.io/name: urn_ngsi-ld_IE_continuum-docker-1 app.kubernetes.io/instance: urn_ngsi-ld_IE_continuum-docker-1 app.kubernetes.io/part-of: aeros-llo-docker app.kubernetes.io/managed-by: aeros-project.eu app.kubernetes.io/created-by: aeros-llo-docker name: continuum-docker-1 spec: id: urn:ngsi-ld:IE:continuum-docker-1 apiUrl: http://192.168.1.124:8000 version: 25.0.3 authentication: 2. Other domains ===================== 1. Install Orion-LD Context Broker ----------------------------------- Create needed *Domain* (with the *isEntrypoint* attribute set to *false*) and *Organization* (only if the domain belongs to a new organization-, ...) entities as in `step #3 from Entrypoint domain <#create-domain-and-organization-entities>`__. 2. Create Context Source Registrations --------------------------------------- Create the needed NGSI-LD Context Source Registrations (CSRs) in all the context brokers. In the future it'll be done by the aerOS Federator component, nevertheless, currently it will be guided by the UPV. Currently, each Orion-LD must have 3 CSR (organizations, infrastructure and services) for each existing domain in the continuum without counting its own domain (e.g. 3 CSR for a continuum with 2 domains, 6 for 3 domains...). Example of CSR with 2 domains (entrypoint domain and the new one) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. In the **Entrypoint domain**: .. code:: text curl --location 'http:///ngsi-ld/v1/csourceRegistrations' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:aeros:federation::organizations", "type": "ContextSourceRegistration", "information": [ { "entities": [ { "type": "Organization" } ] } ], "contextSourceInfo": [ { "key": "Authorization", "value": "urn:ngsi-ld:request" } ], "mode": "inclusive", "operations": [ "retrieveOps" ], "hostAlias": "", // The domain ID of the new domain used in the Orion's Helm installation command "endpoint": "/orionld" // The "publicUrl" attribute of the newly created Domain entity }' .. code:: text curl --location 'http:///ngsi-ld/v1/csourceRegistrations' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:aeros:federation::infrastructure", "type": "ContextSourceRegistration", "information": [ { "entities": [ { "type": "Domain" }, { "type": "LowLevelOrchestrator" }, { "type": "InfrastructureElement" } ] } ], "contextSourceInfo": [ { "key": "Authorization", "value": "urn:ngsi-ld:request" } ], "mode": "inclusive", "operations": [ "retrieveOps" ], "hostAlias": "", // The domain ID of the new domain used in the Orion's Helm installation command "endpoint": "/orionld" // The "publicUrl" attribute of the newly created Domain entity }' .. code:: text curl --location 'http:///ngsi-ld/v1/csourceRegistrations' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:aeros:federation::services", "type": "ContextSourceRegistration", "information": [ { "entities": [ { "type": "Service" }, { "type": "ServiceComponent" }, { "type": "NetworkPort" }, { "type": "InfrastructureElementRequirements" } ] } ], "contextSourceInfo": [ { "key": "Authorization", "value": "urn:ngsi-ld:request" } ], "mode": "inclusive", "operations": [ "retrieveOps", "updateOps", "mergeEntity" ], "hostAlias": "", // The domain ID of the new domain used in the Orion's Helm installation command "endpoint": "/orionld" // The "publicUrl" attribute of the newly created Domain entity }' 2. In the **NEW domain**: .. code:: text curl --location 'http:///ngsi-ld/v1/csourceRegistrations' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:aeros:federation::organizations", "type": "ContextSourceRegistration", "information": [ { "entities": [ { "type": "Organization" } ] } ], "contextSourceInfo": [ { "key": "Authorization", "value": "urn:ngsi-ld:request" } ], "mode": "inclusive", "operations": [ "retrieveOps" ], "hostAlias": "", // The domain ID of the Entrypoint domain used in the Orion's Helm installation command "endpoint": "/orionld" // The "publicUrl" attribute of the Entrypoint Domain entity }' .. code:: text curl --location 'http:///ngsi-ld/v1/csourceRegistrations' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:aeros:federation::infrastructure", "type": "ContextSourceRegistration", "information": [ { "entities": [ { "type": "Domain" }, { "type": "LowLevelOrchestrator" }, { "type": "InfrastructureElement" } ] } ], "contextSourceInfo": [ { "key": "Authorization", "value": "urn:ngsi-ld:request" } ], "mode": "inclusive", "operations": [ "retrieveOps" ], "hostAlias": "", // The domain ID of the Entrypoint domain used in the Orion's Helm installation command "endpoint": "/orionld" // The "publicUrl" attribute of the Entrypoint Domain entity }' .. code:: text curl --location 'http:///ngsi-ld/v1/csourceRegistrations' \ --header 'aerOS: true' \ --header 'Content-Type: application/json' \ --data '{ "id": "urn:aeros:federation::services", "type": "ContextSourceRegistration", "information": [ { "entities": [ { "type": "Service" }, { "type": "ServiceComponent" }, { "type": "NetworkPort" }, { "type": "InfrastructureElementRequirements" } ] } ], "contextSourceInfo": [ { "key": "Authorization", "value": "urn:ngsi-ld:request" } ], "mode": "inclusive", "operations": [ "retrieveOps", "updateOps", "mergeEntity" ], "hostAlias": "", // The domain ID of the Entrypoint domain used in the Orion's Helm installation command "endpoint": "/orionld" // The "publicUrl" attribute of the Entrypoint Domain entity }' 3. Install Self-* in IEs ----------------------------------- Same steps as above. 4. Install HLO & Redpanda ----------------------------------- Same steps as above. 5. Install LLO(s) ----------------------------------- Same steps as above. 3. Add a new IE in an existing domain ====================================== 1. Install Self-* modules in IE. Orion-LD must be reachable by the new IE. 2. Install additional LLOs if needed. You have to create a new LLO entity for the additional LLO and update the IE entity to point to the new LLO entity (see `step #8 from Entrypoint domain <#install-llo-s>`__).