This project demonstrates the use of Command and Query Responsibility Segregation (CQRS) and Event Sourcing with Pivotal Application Service (a.k.a Pivotal Cloud Foundry or PAS). The code consists of two Spring Boot microservices built using the open source Axon Framework and a user interface constructed using Node.js and React.js.
Note: If you just want to experience Axon Trader without having to build and host the code for yourself, simply navigate your browser to https://axontrader.cfapps.io/.
This demo will only intended to run in a compatible Pivotal Application Service environment. This environment must also have a marketplace containing the required "brokered" services. Examples of compatible PAS environments would include Pivotal Web Services (PWS), any enterprise PCF foundation with the correct services, or locally on your PC using PCF-Dev (if started with the required marketplace services running).
In this tutorial we will use Pivotal Web Services (PWS) as our PAS as it meets all of the requirements set out above. If you don't have a PWS account, you can sign up and receive free credits here.
In order to follow along with this tutorial...
- You must have the cf-cli installed (you can test this test by calling
cf versionin your terminal). - You must have access to a Pivotal Application Service instance.
- You must have marketplace services for MySQL, RabbitMQ, Spring Cloud Services Registry and Spring Cloud Services Config Server in PAS.
- You must have the
bashterminal in order to run the provided scripts.
To begin, clone this repository and cd to it's directory in your terminal window. There you'll notice there is a folder for the trader-app api, a folder for the trading-engine microservice and a folder for the trader-ui user interface (amongst others).
You don't have to explore the code in these folders in order to run the Axon Trader application, but you might want to take a look if you're curious how it's written.
To build and run Axon Trader application in your Pivotal Web Services space, read on. If you're using an enterprise instance of Pivotal Application Service the instructions are the same (as long as you have the required marketplace services available).
The Axon Trader Reference Architecture is built using Java, Spring Boot and Node.js (bootstrapping React.js single page application). To run, the Axon Trader applications requires certain PAS Marketplace Services to be available in the "space" where you will host and run the applications. These marketplace services are ClearDB (MySQL), CloudAMQP (RabbitMQ), Spring Cloud Services Config Server and Spring Cloud Services Service Registry.
To pave your PWS space with these marketplace services, simply run the pave.sh script provided. The pave script isn't rocket science, there's no magic, it just saves you typing cf create-service multiple times. Obviously it needs some details from you about your PWS space in order to run, as illustrated below...
# you need to provide this script with your PCF username, password, org and space...
./pave.sh <PWS-USERNAME> <PWS-PASSWORD> <PWS-ORG> <PWS-SPACE>Note: In the
pave.shscript we have chosen only the free PWS service plans for each of the marketplace services. These free plans won't cost you a penny to run but they do have usage limitations (like a 7 day lifetime in the case of the Spring Cloud Services), so make yourself aware of these and make other choices as you feel appropriate.
Following the completion of the pave.sh script, a quick call to cf services in your terminal window should provide you with a list of all of the marketplace services you just created. These services will now be available for use by the Axon Trader applications (which is good, because the Axon Trader applications depend on them to run).
Note: If this was your first time using the Pivotal Application Service, notice how easy it is to create services via the marketplace. Pivotal Application Service is designed to offer a low code, low friction developer experience.
The configuration of the backend microservices in the Axon Trader application is handled by Spring Cloud Services Config. In the pave.sh script, when commissioning Spring Cloud Services Config, a file called config-server-setup.json is used to tell the service where it can find the configuration for the microservices. If you wish to change this location, edit the file. The config repository can be cloned from here.
Now the PWS space is "paved" we can build and "push" our microservice applications and their user interface to it. A script called deploy-backend.sh has been provided to do this to save you a little more repetitive typing...
# you need to provide the deploy script with your PCF username, password, org and space...
$ ./deploy-backend.sh <PWS-USERNAME> <PWS-PASSWORD> <PWS-ORG> <PWS-SPACE>If you open the script you'll notice how easy it is to use
cf pushto run your applications in the cloud. In a couple of minutes you can have microservices with externally accessible URLs and which will keep running no matter what.
When the script has finished, you should see a list of your currently deployed apps and their URLs...
name requested state instances urls
esrefarch-demo-trader-app started 1/1 esrefarch-demo-trader-app.cfapps.io
esrefarch-demo-trader-ui started 1/1 esrefarch-demo-trader-ui.cfapps.ioNote: Take note of the URL for the
esrefarch-demo-trader-app, you'll need it for the next task.
Finally, we need to create some dummy data for the application (companies, users, and cash). To help with this, we have provided an actuator endpoint that accepts a POST request but doesn't require any content. You can use curl to trigger this as shown below.
curl -sL -X POST ${your-trader-app-url}/actuator/data-initializerIf this returns true, you should now have users, companies and orderbooks ready to be used once the UI is in place, which we'll do in the next section.
When we "pushed" the Axon Trader App to Pivotal Web Services a few moments ago, it was assigned a unique and random route URL. We now need to tell the Axon Trader UI about this unique URL so that it can communicate with the backend.
Edit the HostURLsMapping constant in the file trader-app-ui/src/utils/config.js and bind your axon trader UI and APP urls to each other as follows...
export const HostURLsMapping = {
"localhost": "http://localhost:8080",
"your-trader-UI-hostname": "https://your-trader-APP-URL",
}So if your trader UI hostname was esrefarch-demo-trader-ui.cfapps.io and your trader app URL was https://esrefarch-demo-trader-app.cfapps.io then your configuration file would contain
"esrefarch-demo-trader-ui.cfapps.io": "https://esrefarch-demo-trader-app.cfapps.io"as a key-value pair in theconfig.jsJSON file.
Now this configuration is set, we can build and push the Axon Trader UI to PWS.
./deploy-frontend.sh <PWS-USERNAME> <PWS-PASSWORD> <PWS-ORG> <PWS-SPACE>In a few minutes, the UI should have now have started and you will be in a position to test the application as described in the next section.
You should have a replica of the Axon Trader application running in your own PWS space. This means you are ready to test it. To begin, we need the Axon Trader UI URL. You can discover all the URLs for all your PWS application by calling cf apps from your command line...
cf apps
Getting apps in org esrefarch / space myspace as me@me.com...
OK
name requested state instances urls
esrefarch-demo-trader-app started 1/1 esrefarch-demo-trader-app.cfapps.io
esrefarch-demo-trader-ui started 1/1 esrefarch-demo-trader-ui.cfapps.io
esrefarch-demo-trading-engine started 1/1 esrefarch-demo-trading-engine.cfapps.ioCopy the url for the esrefarch-demo-trader-ui application from the urls column and paste it into your browser window. You should see something like this...
If the User Interface is there you can use the Axon Trader UI application and check that everything is working as expected. You should be able to impersonate users, view the dashboard and the order books and place trades. If you forgot to initialise the data you won't have any users, companies or orderbooks. Go back to section 2. and complete the initialisation exercise before you continue.
The following tasks are all optional.
This section on continuously delivering Axon Trader is purely mentioned here for completeness. You can manually refresh the Axon Trader applications at any time simply by repeating the steps above.
For the continuous integration and deployment needs of the Axon Trader application, the team have chosen to use Concourse-ci. You can view our team's Axon Trader build monitor here at any time. Remember: builds can be green as well as red. Past performance is no guarantee of future results.
All of the configuration and scripts required to recreate our build pipeline on your own Concourse server is included in the ci folder under the project root. The only thing that's missing is the private.yml file which we use to specify some secrets and additional config required by the pipeline.
Here is an example of ci/private.yml so you can recreate it...
---
cf-trading-engine-url: http://your-trading-engine.cfapps.io
cf-trader-app-url: http://your-trader-app.cfapps.io
cf-trader-ui-url: http://your-trader-ui.cfapps.io
cf-endpoint: api.run.pivotal.io
cf-user: me@pivotal.io
cf-password: ************
cf-org: myorg
cf-space: prod
skip-ssl: false
webhook: https://hooks.slack.com/your-webhook-url
git-uri: git@github.com:pivotalsoftware/ESarch.git
git-private-key: |
-----BEGIN RSA PRIVATE KEY-----Note: For the
git-uriyou could use your own fork if you don't want to track our repo.
To install and run the pipeline on Concourse, login using fly then run set-pipeline using the pipeline.yml and the private.yml as shown below. Finally unpause-pipeline so make it capable of being run. The commands for this are illustrated below...
fly -t wings login -c http://your-concourse-url
fly -t wings set-pipeline -p bw-esrefarch-demo -c ci/pipeline.yml -l ci/private.yml
fly -t wings unpause-pipeline -p bw-esrefarch-demoIf you have finished trying the Axon Trader application and you would like to clear down your PWS space, a script has been provided that will stop the applications, remove the services and delete everything.
./teardown.sh <PWS-USERNAME> <PWS-PASSWORD> <PWS-ORG> <PWS-SPACE>Once completed, the teardown.sh script should leave you with an empty space and no services. To check this is the case you could call...
cf apps
cf services
cf network-policiesYou should see that each is empty. If not, use the PWS dashboard UI at run.pivotal.io to clear things manually or use the cf-cli's `cf help -a" command to find and execute the commands you need.
The local Spring profile plus docker-compose.yml run the whole Axon Trader on a laptop with no Pivotal/TAS services. Compose stands in for the TAS-bound services (MySQL 5.7 for enginedb/appdb, RabbitMQ 3.7 for rabbit, the discovery-server Eureka for registry); the local profile points both apps at those containers. There is no Config Server any more: the Spring Cloud Services config client was removed and the values ESarch-conf served are vendored in each app's application-kubernetes.properties (see deploy/kubernetes/README.md).
Prerequisites: JDK 8, Docker with Compose v2, Node.js (20 is known to work) with npm, plus curl and jq for the smoke script.
# 1. Build and test (JDK 8), then build the container images into the local Docker daemon with Jib
./mvnw clean verify
./mvnw -DskipTests package jib:dockerBuild
(cd trader-app-ui && npm ci && CI=true npm test)
# 2. Backing services: MySQL 5.7 (:3306), RabbitMQ 3.7 (:5672, UI on :15672), Eureka discovery-server image (:8761)
docker compose up -d --wait
# 3. Backend apps (trading-engine on :8081, trader-app on :8080), either as images in Compose, which also starts
# the UI image and the reverse proxy from 4a (http://localhost:8088)...
docker compose --profile apps up -d --build --wait
# ...or as jars, each in its own terminal
SPRING_PROFILES_ACTIVE=local java -jar trading-engine/target/trading-engine.jar
SPRING_PROFILES_ACTIVE=local java -jar trader-app/target/trader-app.jar
# 4a. UI image (nginx) behind a local reverse proxy that stands in for the GKE Ingress: http://localhost:8088
# serves the UI at / and trader-app at /api (only /api/command/ and /api/query/ are routed).
# Already running with the `apps` profile; with the jars, start it on its own:
docker compose --profile ui up -d --build --wait
# 4b. Or the dev server on :3000, calling trader-app directly (the bundle defaults to same-origin /api)
(cd trader-app-ui && REACT_APP_API_ROOT=http://localhost:8080 npm start)
# 5. Smoke test: newman collection, ci/tasks/e2e-test-both-sides.sh, then a matched buy/sell checked in the portfolio view.
# With 4a, proxyURL sends the UI checks and the orders through the proxy and adds deep-link and routing checks.
proxyURL=http://localhost:8088 ci/local/verify-local.sh # or just ci/local/verify-local.sh with 4bJib builds esarch/trading-engine, esarch/trader-app and esarch/discovery-server (no Dockerfile, no local JDK in the image): Eclipse Temurin 8 JRE base pinned by digest, user 1000:1000, JAVA_TOOL_OPTIONS=-XX:MaxRAMPercentage=75.0 -XX:+ExitOnOutOfMemoryError instead of the buildpack memory calculator, and two tags, the 12-character git SHA and latest. Override with -Dimage.registry=... / -Dimage.tag=...; jib:build pushes instead of loading into Docker. Compose runs latest unless ESARCH_IMAGE_TAG (and ESARCH_IMAGE_REGISTRY) say otherwise.
The UI calls the API same-origin under /api by default. To point an image at another API root without rebuilding, mount a replacement for /usr/share/nginx/html/config.js containing window.AXON_TRADER_CONFIG = { apiRoot: 'http://localhost:8080' }; (see trader-app-ui/public/config.js). REACT_APP_API_ROOT still works at build time. ci/local/smoke-ui-image.sh checks the image and the proxy against a stub API, without the Java services. The proxy uses host networking to reach the backends on 127.0.0.1 (Linux; on Docker Desktop enable host networking first).
The backends bind to 127.0.0.1 only (the local profile has no auth in front of the trading API; set ESARCH_BIND_ADDRESS to change it). ci/local/verify-local.sh exits non-zero if newman, the e2e script or the portfolio check fails. It waits until trader-app can route commands to trading-engine and loads the demo data through /actuator/data-initializer, so open http://localhost:3000 afterwards to trade in the UI. The local passwords default to throwaway values; override them with ESARCH_MYSQL_USER, ESARCH_MYSQL_PASSWORD, ESARCH_MYSQL_ROOT_PASSWORD, ESARCH_RABBITMQ_USER and ESARCH_RABBITMQ_PASSWORD (set the same values for Compose and the apps). The local profile lets Hibernate manage the schema with ddl-auto=update, as on TAS, so data survives restarts; docker compose down -v wipes it. To run with the production schema handling (Flyway migrations and ddl-auto=validate), start the apps with the kubernetes profile instead. ci/local/verify-kubernetes-profile.sh does that against the same Compose stack, sets only the environment variables listed in deploy/kubernetes/README.md, and checks that orders and portfolios survive a restart. With ESARCH_RUNTIME=image it runs the Jib images (mvn -B -DskipTests package jib:dockerBuild) on the Compose network instead of the jars, as the pods would.
.github/workflows/ci.yml runs on every pull request and on pushes to master: ./mvnw verify, the UI npm test and build, the UI image smoke test (ci/local/smoke-ui-image.sh), then it builds the four images (trading-engine, trader-app, discovery-server with Jib, trader-ui with its Dockerfile) tagged with the full git SHA and scans them with Trivy. The scan fails on any critical CVE that is not in ci/trivy/critical-baseline.yaml, the pinned list of criticals already shipped by the Spring Boot 2.0.3 / Java 8 stack (each entry is tied to the exact package version and the file expires on 2027-05-01). The job summary has a per-image table and the JSON reports are uploaded as the trivy-reports artifact.
On master the scanned images are pushed to Artifact Registry through Workload Identity Federation (no service account key). Set these repository variables from the outputs of infra/terraform/envs/staging; until all three are set the push job is skipped with a notice:
| Variable | Terraform output | Example |
|---|---|---|
GCP_WORKLOAD_IDENTITY_PROVIDER |
ci_workload_identity_provider |
projects/123456789/locations/global/workloadIdentityPools/github/providers/github |
GCP_CI_SERVICE_ACCOUNT |
ci_service_account |
ci-publisher@my-project.iam.gserviceaccount.com |
GCP_ARTIFACT_REGISTRY |
artifact_registry_url |
europe-west1-docker.pkg.dev/my-project/esarch |
The pushed digests are listed in the publish job summary. They are then deployed to staging and, behind the production environment approval, promoted to production without a rebuild: see docs/deploy-gke.md for the release, rollback (helm rollback, previous digest) and local kind workflow. The Concourse pipeline (ci/pipeline.yml) still exists alongside it.
CQRS on itself is a very simple pattern. It only prescribes that the component of an application that processes commands should be separated from the component that processes queries. Although this separation is very simple in and of itself, it provides a number of very powerful features when combined with other patterns. Axon provides the building blocks that make it easier to implement the different patterns that can be used in combination with CQRS.
Commands are typically represented by simple and straightforward objects that contain all data necessary for a command handler to execute it. A command expresses its intent by its name. In Java terms, that means the class name is used to figure out what needs to be done, and the fields of the command provide the information required to do it.
The Command Bus receives commands and routes them to the Command Handlers. Each command handler responds to a specific type of command and executes logic based on the contents of the command. In some cases, however, you would also want to execute logic regardless of the actual type of command, such as validation, logging or authorization.
The command handler retrieves domain objects (Aggregates) from a repository and executes methods on them to change their state. These aggregates typically contain the actual business logic and are therefore responsible for guarding their own invariants. The state changes of aggregates result in the generation of Domain Events. Both the Domain Events and the Aggregates form the domain model.
Repositories are responsible for providing access to aggregates. Typically, these repositories are optimized for lookup of an aggregate by its unique identifier only. Some repositories will store the state of the aggregate itself (using Object Relational Mapping, for example), while others store the state changes that the aggregate has gone through in an Event Store. The repository is also responsible for persisting the changes made to aggregates in its backing storage.
Axon provides support for both the direct way of persisting aggregates (using object-relational-mapping, for example) and for event sourcing.
The event bus dispatches events to all interested event listeners. This can either be done synchronously or asynchronously. Asynchronous event dispatching allows the command execution to return and hand over control to the user, while the events are being dispatched and processed in the background. Not having to wait for event processing to complete makes an application more responsive. Synchronous event processing, on the other hand, is simpler and is a sensible default. By default, synchronous processing will process event listeners in the same transaction that also processed the command.
Event listeners receive events and handle them. Some handlers will update data sources used for querying while others send messages to external systems. As you might notice, the command handlers are completely unaware of the components that are interested in the changes they make. This means that it is very non-intrusive to extend the application with new functionality. All you need to do is add another event listener. The events loosely couple all components in your application together.
In some cases, event processing requires new commands to be sent to the application. An example of this is when an order is received. This could mean the customer's account should be debited with the amount of the purchase, and shipping must be told to prepare a shipment of the purchased goods. In many applications, logic will become more complicated than this: what if the customer didn't pay in time? Will you send the shipment right away, or await payment first? The saga is the CQRS concept responsible for managing these complex business transactions.
Since Axon 3.1 the framework provides components to handle queries. The Query Bus receives queries and routes them to the Query Handlers. A query handler is registered at the query bus with both the type of query it handles as well as the type of response it providers. Both the query and the result type are typically simple, read-only DTO objects. The contents of these DTOs are typically driven by the needs of the User Interface. In most cases, they map directly to a specific view in the UI (also referred to as table-per-view).
It is possible to register multiple query handlers for the same type of query and type of response. When dispatching queries, the client can indicate whether he wants a result from one or from all available query handlers.
For more information on Axon visit the Axon documentation. For more information on Axon Trader visit the Axon Trader wiki.
