Eventuate - Event Sourcing and CQRS

When building an application based on microservices architecture, there are two major patterns for service to service communication.

Serivce to Service communication

Synchronous, Request/Response Communication

In light-4j, this means to use Client module to call other services in the request/response fashion regardless light-rest-4j, light-graphql-4j or light-hybrid-4j is used.

Asynchronous, Message-Based Communication

The eventual consistency framework light-eventuate-4j is designed to facilitate asynchronous communication between services built on top of light-rest-4j, light-grahpql-4j and light-hybrid-4j.

Service communication is through events and every service maintain its own aggregates to serve consumer independently.

Which Style to Choose

For the pros and cons of each communication patterns, please see communication pattern

Why light-eventuate-4j

For microservices implementation, developing business transactions that update entities that are owned by multiple services is a challenge, as is implementing queries that retrieve data from multiple services; for a complicated application that has too many services interact with each other. It is essential to choose an asynchronous message-based style for service communication and transaction management with eventual consistency.

For a large scale application, an event driven approach gives you more flexibility and scalability and requests from client don’t need all the services to be up and running the exact time.

In addition to event driven application, Event Sourcing and CQRS are getting popular these days for high scale applications.

Once you’ve decided that asynchronous message-based communication is best for your use case, you need to make some architecture decisions to choose the best combination of frameworks to build your microservices.

For large scale applications, the event driven approach gives you more flexibility and scalability and requests from client don’t need all the services to be up and running the exact time.

In addition to event driven application, Event Sourcing and CQRS are getting populate these days for high scale applications.

light-eventuate-4j is a framework that provides user to build Event Sourcing and CQRS services without worrying about the underline infrastructure. It provides an event store and APIs to publish and subscribe events to/from clients. You can build your services with light-rest-4j, light-graphql-4j or light-hybrid-4j and they are all seemlessly integrated with light-eventuate-4j framework which provide event store and event producing and consuming client for services built on top of other light-4j frameworks.

Another alternative solution would be light-tram-4j which is another event driven framework but without the complicated event sourcing to deal with.

light-eventuate-4j components:

  • light-4j, light-rest-4j, light-graphql-4j and light-hybrid-4j provide platforms to build microservices in REST, Graphql and Hybrid/RPC style.

  • Mysql, PostgreSql or Oracle for event store to persist events

  • Zookeeper is used to manage Kafka cluster

  • Apache Kafka is used as message brokers for publishing/subscribing events

  • A console/admin UI to monitor the message broker and communication between services

  • A hybrid command server to host all hybrid command services in case light-hybrid-4j is used

  • A hybrid query server to host all hybrid query services in case light-hybrid-4j is used

  • A RESTful CDC service that is responsible to publish events to Kafka from event store if light-rest-4j is used.

Architecture

Here is a list of architecture decisions for light-eventuate-4j:

Design

Component

Data Change Capture
  • eventuate-cdc-connector-common - common utilities shared by all CDC modules
  • eventuate-cdc-connector-mysql-binlog - a trailing transaction log implementation of CDC with mysql binlog
  • eventuate-cdc-connector-polling - a generate CDC implemented by polling database table periodically
  • eventuate-cdc-connector-test-util - test utility for cdc connectors
  • eventuate-cdc-polling-server - a restful service that uses polling connector
  • eventuate-cdc-server - a restful service that uses mysql-binlog connector
  • eventuate-cdc-service - a hybrid service that uses mysql-binlog connector
Common
  • eventuate-common - common interfaces or classes shared by both client and server
  • eventuate-event - event and event related interfaces and implementations
Client
  • eventuate-client - common client utilities and classes
  • eventuate-client-jdbc - client JDBC classes for db repository
  • eventuate-client-jdbc-common - client JDBC API common
  • eventuate-client-test - client test cases
  • eventuate-client-test-domain - client test case account management domain model (account, money)
  • eventuate-client-test-example - client test case services that use test-domain
  • eventuate-client-test-util - utility classes for client test cases
Server
  • eventuate-server-common - common components for server
  • eventuate-server-jdbc - JDBC event store API
  • eventuate-server-kafka - Kafka Publisher API

CDC server vs service

In light-eventuate-4j we have two different CDC implementations mysql binlog and polling. The first only works with MySql database and it is the most efficient way. The second polling is a generic implementation that is suitable for any SQL database like Postgres or Oracle. For each implementation, we need to start a server to allocate a thread to do the CDC work. There are two different ways to implement the server. Read More »

Concepts

Light-eventuate-4j conceptual model Entity An entity is a business object such as an Order. An entity is an instance of an EntityType. In Domain-Driven Design terms, an entity is an aggregate. Each entity has an ID, unique among instances of the same entity type, which you can choose or allow Eventuate to assign to you. An entity has a sequence of one or more events. You can perform the following operations on an entity Read More »

How to handle duplicated events in Kafka

As we are using Kafka as our messaging broker for event sourcing and Kafka guarantees messages are delivered to consumers at least once. that means we will have duplicated events that have to be handled gracefully. There are several ways to handle duplicate messages. Idempotency Vector clock Read More »

Why not use kafka as eventstore

When using Kafka as pipeline for event sourcing, people will ask why not use Kafka as event store instead of a database. It can persist events and keep it for as long as it requires. There are several drawbacks in doing so. Kafka only guarantees at least one delivery and there are duplicates in the event store that cannot be removed. Due to immutability, there is no way to manipulate the event store when the application evolves and events need to be transformed. Read More »

Event Sourcing

Event sourcing is an event-centric approach to persistence; A service that uses event sourcing persists each aggregate as a sequence of events. When it creates or updates an aggregate, the service saves one or more events in the database, which is also known as the event store. It reconstructs the current state of an aggregate by loading the events and replaying them. In functional programming terms, a service reconstructs the state of an aggregate by performing a functional fold/reduce over the events. Read More »

Event Driven Architecture

An Eventuate application consists of four types of modules, each with different roles and responsibilities: Command-side module Query-side view updater module Query-side view query module Outbound gateway module Note that this is the logical architecture of the application. The modules can either be deployed together as a monolithic application or separately as standalone microservices. Command-side modules The command-side of an Eventuate application consists of one or more command-side modules. Read More »

Command Query Responsibility Segregation

CQRS Pattern CQRS stands for Command Query Responsibility Segregation. As the name suggests, we split the application into two parts: Command-Side and Query-Side. Now, one will ask, what do “command” and “query” mean? Commands change the state of the object or entity, also called modifiers or mutators. Queries return the state of the entity and do not change anything. Another term for them is “accessors.” Why Is It Required? Read More »

Communication Patterns

In a monolithic application, components invoke one another via method or function calls. In contrast, a microservices-based application might be distributed on multiple Docker containers or VMs or multiple machines. Each service instance is typically a process and services must interact using an inter-process communication mechanism. Patterns There are two major styles in client/service or service/service communication. Synchronous, Request/Response Based Communication The client expects a timely response from the service and might even block while it waits. Read More »

Example

Read More »

Light-eventuate-4j Tutorial

Currently, there are several tutorial to teach users how to use the framework step by step. The source code for these tutorials can be found in eventuate example on github.com Read More »

Getting Started with light-eventuate-4j

Building microservices that communicated with events Read More »