Georg Schwarz
What Microservices Can Learn From Enterprise Data Integration

What Microservices Can Learn From Enterprise Data Integration

research microservices architecture

Note: This is an opinionated summary of a research paper by the first author. The opinions expressed exceed the scientific insights from the study. You can download the original study here.

TL;DR

  • Microservices underutilize proven integration strategies from enterprise systems, particularly at the UI and database layers
  • There is no perfect solution: Every integration approach (API calls, middleware, via frontend, database views) involves trade-offs between coupling, complexity, and consistency
  • We need a combination of strategies: Design interactions coming from the user-interface, use a combination of synchronous and asynchronous integration mechanisms among microservices

The Problem With Microservices Data Integration

You’ve built your microservices architecture. Each service owns its data, maintains its own database, and exposes a clean API. Everything is decoupled and independent; exactly as it should be. But then comes the inevitable question: “How do we query data across multiple services?”

If you’ve been solving this with service-to-service API calls or custom aggregation layers, you’re not alone. But there are more ways to do it, so let’s go over those!

Findings

We analyzed how microservices handle data integration and compared it to established Enterprise Information Integration (EII) theory. EII describes integration across four architectural levels: client/UI layer, server application layer, middleware, and database layer. Microservices are more opinionted in this regard:

  1. Microservices favor application-level integration

    • REST and RPC-based service calls dominate current practice
    • Direct API composition is the go-to solution for cross-service queries

  2. Middleware is used selectively and cautiously

    • Simple pub-sub messaging is recommended for event-driven architectures
    • “Keep your middleware dumb, and keep the smarts in the endpoints” — ESBs with transformation logic are explicitly rejected

  3. UI-level integration is underutilized but viable

    • Modern web clients can integrate data from multiple services, yet most projects use monolithic UIs
    • The opportunity: Micro-frontends enable integration at the presentation layer, extending microservices principles to the frontend

  4. Database-level integration is completely rejected

    • Direct database access between services violates microservices principles
    • Shared databases create coupling and restrict technology choice
    • The gap: Database views as versioned interfaces and read replicas remain unexplored

No Perfect Solution

When you need to query data across multiple services, your team always faces a trade-off:

  • Service-to-service API calls: Can create chatty, slow request chains and tight coupling
  • Pub-sub middlewares: Adds complexity with delivery guarantees and eventual consistency
  • Micro-frontends: Are a relatively new topic and seem complex to implement yet
  • Database integration: Are not considered at all, although databases have solved many challenges in the past that we try to solve now on the application layer

Open Questions (For Practitioners and Researchers)

  1. Database views as service interfaces: Could versioned views provide read-only integration while maintaining independent deployability, although sacrificing technology independence to a certain point?

  2. Middleware with intelligence: While “dumb pipes, smart endpoints” is the mantra, smarter middleware abstractions such as service meshes emerge and gain in popularity. How much intelligence is okay, and where is the sweet spot of this trade-off?

  3. UI-layer integration deserves more attention: Client-side data composition and micro-frontends can solve many integration problems without backend complexity, yet they’re often overlooked. Should we start thinking about cutting into microservices already from the user-interface perspective?

  4. Combination of approaches: It is unlikely that there is a one-size-fits-all solution, we might even need a combination of approaches for some systems. So how to combine those in a meaningful way?

Practical Takeaways

  1. Consider the full architectural stack for integration:

    • UI layer: Use micro-frontends or backend-for-frontend patterns to compose data client-side or in dedicated aggregation services
    • Application layer: REST and service calls work, but watch for cascade failures and latency
    • Middleware: Pub-sub messaging enable loose coupling and asynchronous data replication
    • Database layer: Generally rejected, but consider whether versioned views could work for your read-heavy scenarios

  2. Push more integration to the UI: Modern web clients are capable of calling multiple services and composing results. This reduces backend complexity and can improve user experience.

    In general, we should think about the whole architectural integration design with the user interface level. In most scenarios using the well known approach of RESTful HTTP between the UI and the backend, we can simplify backend interactions a lot when supplying more context data in the requests rather than getting data from other microservices (when this does not open up attack vectors).

  3. Combining application layer and middleware solutions: Among microservices, it often makes sense to use a combination of a synchronous solution (e.g., RESTful HTTP) and an asynchronous solution (e.g., AMQP). For a specific microservice interaction, the choice should depend on the nature of the business logic that is implemented.

  4. Event-driven integration is underrated: Message brokers and event streams can handle many integration scenarios through event sourcing, CQRS, and data replication without synchronous coupling. The technologies handle a lot of error scenarios for you by providing at-least-once or exactly-once delivery guarantees, letting you focus on your business logic. You need to worry less about handling timeouts, retries, and preventing service instance overload, but there is a learning curve as well.

Learn More

📄 Download the full paper (PDF)

Published at the 53rd Hawaii International Conference on System Sciences (HICSS-53), 2020.

Citation:

Schwarz, G.-D., & Riehle, D. (2020). What Microservices Can Learn From
Enterprise Information Integration. In Proceedings of the 53rd Hawaii
International Conference on System Sciences (HICSS).

License: This article is licensed under the CC BY-NC-ND 4.0 license, which permits non-commercial sharing and distribution with appropriate credit to the original authors, but does not allow modifications or adaptations.

This research was conducted at the Friedrich-Alexander University Erlangen-Nürnberg, examining the intersection of modern microservices architecture and established enterprise integration patterns.