Data Management and Interoperability Tool

THEROS data management and harmonization platform aims to tackle the heterogeneity and variability of various data sources used for monitoring, verification, and traceability by implementing a comprehensive data management and harmonization platform. This platform will provide essential APIs to facilitate data retrieval from different THEROS components, ensuring seamless integration.  A methodology utilizing data translators will be adopted to automatically transform managed resources into standardized digital formats, models, and ontologies. Furthermore, geocomputational and analytical processes will be employed to link outputs from Earth Observation services, photonics, IoT sensing devices, and DNA authenticity kits with transaction data managed by the blockchain traceability system, enabling efficient consumption by the verification engine. The platform will operate on a cloud-based centralized infrastructure, ensuring efficient resource management, scalability, and large-scale monitoring.

Picture26

THEROS Toolbox infrastructure

Key Components

Datasources Connectors:

  • These components facilitate the data collection from the various data sources of the THEROS project. They are created to match the specific needs and protocols that each data provider support so that data can be retrieved from IoT Sensor Network, Earth Observation Services, MEMS photonics systems and the DNA authenticity kit.

Data Translators:

  • These components convert data received from the connectors into a standardized format so that they can be used by the platform ensuring uniform data access. Each data translator is custom made as each data source has different formats and data fields.

THEROS Toolbox API:

  • The Toolbox API implements the OGC SensorThings API specification and is the central component that integrates data from all translators. It serves as a single point where all data can be retrieved in a unified way by various data consumers.

Traceability & Verification Connector:

  • This component act as a middleware between the data management and harmonization platform and the blockchain-based traceability and verification engine. This connector’s function is to selectively transfer harmonized data to the verification engine based on the needs of the business rules that are going to be applied.

Harmonization Platform Monitor:

  • All components of the platform are deployed in a Kubernetes cluster and for each traffic, CPU usage and memory is monitored.  By monitoring, these metrics, any performance bottleneck can be detected and actions like scaling the services deployments can be taken.

Interfaces

Monitoring of the deployed harmonization platform components:

  • IoT Sensor Network connector implements two APIs one for retrieving raw sensor data and another API that retrieve the output of the IoT analytics engine.

  • MEMS photonics systems connector implements one API for retrieving data from the MEMS backend server

  • DNA authenticity kit connector implements a graphical user interface that the user imports the data from the portable device and an API to retrieve them to the platform.

  • Earth Observation Services connector that consumes data from the Marker provider API and retrieves marker information about each parcel.

Traceability & Verification Connector:

  • This connector is a client to the REST API of the Traceability and Verification Component which sends part of the harmonized data for verification.

THEROS Toolbox API:

  • The main interface from which any data consumer can retrieve data that are collected by all THEROS data producers. It implements the SensorThings API that has a REST API endpoint.

Keycloak Identity Provider:

  • The interface where all components authenticate requests using its endpoint and ensure that users have rights to access the data and services have permissions to send and receive data.

Services

THEROS Toolbox API:

  • The Toolbox API is a core service in the THEROS architecture, integrating diverse data from various sources into a unified system. This centralized API conforms with the Open Geospatial Consortium (OGC) SensorThings API specification, ensuring consistency, interoperability, and accessibility.

Keycloak Identity Provider:

  • Keycloak is an open-source identity and access management solution which provides authentication and authorization services. It supports both user authentication and service-to-service secure communication, ensuring secure access control and identity management. This service supports OAuth 2.0 and OpenID Connect protocols, making it possible to secure APIs and user authentication, using enterprise grade security best practices.

    Keycloak is integrated to all graphical user interfaces and services that interact with each other in THEROS as shown in figure below.

Picture28

THEROS Identity provider

Usage and applications

Data consumers can use the Toolbox API to integrate the data streams coming from various sources, into their applications and also further process the data if needed. This unified interface allows for easy access of standardized data, enabling the other digital tools to not worry about the different data formats and protocols that the data providers support. This harmonized approach simplifies data querying and filtering and by using a consistent format, it also simplifies data parsing. Keycloak, deployed as an identity provider, serves as the secure authentication and authorization solution for both user and service-to-service communications. With Keycloak, each digital tool can implement single sign-on (SSO) enhancing user experience and security. By managing user roles and permissions through Keycloak, all appropriate access controls are in one place, reducing the risk of unauthorized access. This identity management solution ensures compliance with state-of-the-art security standards, and secure interactions between users and services.