Decentralized and Microservice-Oriented Data Integration for External Data Sources

Data lakes offer good opportunities to centrally use heterogeneous data for analytical questions in companies. However, there are also challenges and risks regarding missing reference architectures, accessibility or usability. By using modern architecture patterns such as microservices, data can alternatively be managed in a technically and organizationally decentralized manner. Easily accessible interfaces and microservice architecture patterns can maintain important data lake characteristics, such as accessibility and the provision of metadata. Thus, costs can be saved, data can be held accountable in the respective domains, and at the same time interfaces for analytical questions can be provided. The paper illustrates the idea in the form of a work-in-progress paper using the integration of external data sources as an example.