aerOS KrakenD API Gateway

Introduction

KrakenD is an open-source, high-performance API Gateway designed to facilitate and manage API requests between clients and services. It acts as an intermediary, handling incoming API requests, routing them to the appropriate backend services, and then aggregating the responses before sending them back to the client. KrakenD is known for its simplicity, flexibility, and efficiency, making it a popular choice for organizations looking to streamline their API management processes.

Features

  • High Performance: KrakenD is built to handle high loads with low latency, ensuring efficient request processing and response times.

  • Scalability: It supports horizontal scaling, allowing it to handle increasing traffic by distributing the load across multiple instances.

  • Ease of Use: Configuration is straightforward, often done through a simple JSON or YAML file without the need for extensive programming.

  • Flexibility: It supports various functionalities such as rate limiting, caching, authentication, and authorization, making it adaptable to different use cases.

  • Security: KrakenD includes built-in security features like JWT validation, OAuth2 integration, and request throttling to protect against common API threats.

Place in architecture

KrakenD acts as the API Gateway of every aerOS domain and thus should always be the entrypoint of communications into the domain.

User guide

A list of the enabled endpoints can be seen in the testing section.

There are 3 possible installations

  • Helm Chart (Recommended)

  • Manually using the k8s manifests

  • Docker

The files for non-helm installations can be found in the Gitlab repo. Should work with other K8s cluster environments with modifications to the KrakenD configuration file, see more in testing.

Prerequisities

Keycloak needs to be previously installed and properly configured in order to access the protected endpoints. Otherwise every petition will return a 401 Unauthorized response.

Installation

This section will cover the 3 ways of installing KrakenD in your deployment. For help creating custom endpoints see the Values section. You can also use the file in helm-chart/ as a template.

You can clone the repository to get the required files. This will be necessary if you do not use the Helm installation method.

git clone https://gitlab.aeros-project.eu/wp3/t3.4/api-gateway.git

Manually using the k8s manifests

Once you have access to the desired K8s cluster, use the command kubectl to install KrakenD. Please note that you’ll need to have the files locally for this installation.

Also, please note that you’ll have to manually modify the krakend.json file to fit your deployment, modifying the keycloak field where krakend verifies the token and also modifying the backend of the endpoints so it fits your deployment.

From the root folder of the api-gateway cloned repo:

kubectl create -f ./k8s-manifests/krakend-service.yaml
kubectl create configmap krakend-config --from-file krakend.json
kubectl create -f ./k8s-manifests/krakend-deployment.yaml

Docker installation

For this installation simply execute the docker compose up -d command from the root folder of the api-gateway cloned repo.

Also, please note that you’ll have to manually modify the krakend.json file to fit your deployment, modifying the keycloak field where krakend verifies the token and also modifying the backend of the endpoints so it fits your deployment.

Configuration options

This section will cover how to manually add new endpoints to your deployment, as well as how to edit the configuration without breaking anything. We will be using the helm installation as a template, but the basis applies to all three installations.

Please note we will not cover the entire helm deployment and its configuration or all of its fields such as the ingress section. You can enable the default Ingress from the values.yaml file by enabling it in the ingress section, however we feel like this is too much a private part of the pilots deployment to cover it reliably.

The two relevant files for the Helm installation are ./helm/values.yaml and ./helm/templates/_endpoints.tpl.

Values

This file contains all variables that are used in the deployment, the three relevant sections are:

krakend:
...
    service:
        type: NodePort
        ports:
        api:
            port: 8080
            targetPort: 8080
            containerPort: 8080
            nodePort: ""
            protocol: TCP
    config:
      hloAllocatorUrl: http://hlo-allocator-service.default.svc.cluster.local:8082
      hloFeUrl: http://hlo-fe-service.default.svc.cluster.local:8081
      keycloakUrl: https://keycloak.cf-mvp-domain.aeros-project.eu
      keycloakRealm: keycloak-openldap
      logLevel: DEBUG
      orionCacheDuration: 3600
      orionUrl: http://orion-ld-broker.default.svc.cluster.local:1026
      sharedCacheDuration: 900
    additionalEndpoints:
      -  endpoint: /test/anotherendpoint
         method: POST
         roles:
            - DomainAdministrator
            - IoTDeployer
         queryStrings:
            - q
            - testq
         backend:
            url: /placeholder/custom/backend
            method: PATCH
            host: https://orion-broker-url:1026
         inputHeaders:
            - X-Auth-Token
            - Authorization
            - Content-Type

Service is where the ports that will be used by KrakenD are set up for Kubernetes.

Config is where the predetermined values for some of the configuration will be set up. For example you may want to change the Orion URL to the one that fits your deployment, or you may want to set up several custom lists for the allowed roles and headers. Everything here must follow the parameters of Helm.

AdditionalEndpoints is where every new endpoint you wish to add must be located, all new endpoints must follow the structure set by the example explained below:

- endpoint:  - The exact string resource URL you want to expose
  method - The method supported by this endpoint
  roles: - Array: only tokens matching one of the roles here will be 
           allowed through (Optional)
  queryStrings: - Array: only query strings listed here will be allowed though into the 
                  backend (To allow all query strings add a single "*" element
                   -wildcard- in the array)
  inputHeaders: - Array: only headers listed here will be allowed though into the 
                  backend (To allow all headers add a single "*" element
                   -wildcard- in the array)
  backend:
    url: - The path inside the service (no protocol, no host, no method)
    method: - The method supported by the backend
    host - The hosts that will be receiving the petitions

Developer guide

First, verify the correct installation with the following command:

kubectl get all

If both the service and deployment are running you can test the deployment by making a petition to Keycloak to retrieve a user token:

curl --location --request POST 'https://YOUR_KEYCLOAK/auth/realms/YOUR_REALM/protocol/openid-connect/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'client_id=YOUR_CLIENT' \
--data-urlencode 'client_secret=YOUR_SECRET' \
--data-urlencode 'username=YOUR_USER' \
--data-urlencode 'password=YOUR_PASSWORD'

Please note that you’ll have to change the CURL command to fit your requirements. Once you have the user token you’ll have to send it as a header to the desired endpoint. A list of the currently enabled endpoints is:

/orionld/ngsi-ld/v1/entities/{entityId}/attrs/{attrId}
PATCH
--
/orionld/ngsi-ld/v1/entities/{entityId}
PATCH
--
/orionld/ngsi-ld/v1/entities/{entityId}
DELETE
--
/orionld/ngsi-ld/v1/entities/{entityId}/attrs/{attrId}
DELETE
--
/orionld/ngsi-ld/v1/subscriptions/{subscriptionId}
DELETE
--
/hlo_fe/services/{service_id}
GET
--
/hlo_fe/services/{service_id}
POST
--
/hlo_fe/services/{service_id}
DELETE
--
/hlo_fe/services/{service_id}
PUT
--
/hlo_al/services/{service_id}
POST
--
/hlo_al/services/{service_id}/overlay
DELETE
--
/hlo_al/services/{service_id}/service_components/{service_component_id}/
PUT
--
/hlo_al/services/{service_id}/service_components/{service_component_id}/
GET
--
/hlo_al/services/{service_id}/service_components/{service_component_id}/
DELETE
--
/orionld/ngsi-ld/v1/entities
GET
--
/orionld/ngsi-ld/v1/entities/{entityId}
GET
--
/entities
GET
--
/entityOperations/create
POST
--
/entityOperations/merge
POST
--
/jsonldContexts
GET
--
/jsonldContexts
POST
--
/entities/{entityId}/attrs/{attrId}
DELETE
--
/entities/{entityId}/attrs/{attrId}
PATCH
--
/entities/{entityId}/attrs/{attrId}
PUT
--
/csourceSubscriptions
GET
--
/csourceSubscriptions
POST
--
/entityOperations/upsert
POST
--
/types/{type}
GET
--
/entityOperations/query
POST
--
/csourceSubscriptions/{subscriptionId}
DELETE
--
/csourceSubscriptions/{subscriptionId}
GET
--
/csourceSubscriptions/{subscriptionId}
PATCH
--
/entityOperations/delete
POST
--
/entityOperations/update
POST
--
/jsonldContexts/{contextId}
DELETE
--
/jsonldContexts/{contextId}
GET
--
/subscriptions
POST
--
/subscriptions
GET
--
/csourceRegistrations
GET
--
/csourceRegistrations
POST
--
/subscriptions/{subscriptionId}
DELETE
--
/subscriptions/{subscriptionId}
GET
--
/subscriptions/{subscriptionId}
PATCH
--
/attributes/{attrId}
GET
--
/entities
POST
--
/entities/{entityId}
PATCH
--
/entities/{entityId}
PUT
--
/entities
DELETE
--
/entities/{entityId}
DELETE
--
/entities/{entityId}
GET
--
/entities/{entityId}/attrs
POST
--
/attributes
GET
--
/types
GET
--
/csourceRegistrations/{registrationId}
DELETE
--
/csourceRegistrations/{registrationId}
GET
--
/csourceRegistrations/{registrationId}
PATCH
--
/orionld/ex/v1/notify
POST
--
/orionld/ex/v1/notification/{subscriptionId}
POST
--
/federator/health
GET
--
/federator/v1/domains
GET
--
/federator/v1/domains
POST
--
/federator/v1/domains/local
GET
--
/federator/v1/domains/local
DELETE
--
/federator/v1/domains/{domainName}
DELETE
--
/iota_api
POST

Authors

Universitat Politècnica de València

License

This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.

THE SOFTWARE IS PROVIDED "AS IS WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to <http://unlicense.org/>

Notice (dependencies)

Keycloak needs to be previously installed and properly configured in order to access the protected endpoints. Otherwise every petition will return a 401 Unauthorized response.