Repository files navigation

ftgogo (food-to-gogo) is a Golang implementation oftheFTGO application described in thebook"Microservice Patterns" by Chris Richardson. Alibraryedat was developed to provide for Golang many of the solutionsthatEventuate, the framework used by FTGO, provides for Java.

This repository exists to demonstrate the patterns and processes involved when constructing a distributed applicationusingevent-driven architecture.

This repository started as a Golang clone of the FTGO demonstration application but as time goes on it will grow todemonstrate additional microservice patterns and techniques.

1. Architecture
   1. Clean Architecture
   2. Code Layout
   3. Services
2. Design
   1. Event-Driven Architecture
      1. Sagas
      2. Outbox Pattern
      3. Message Deduplication
   2. CQRS
   3. Event Sourcing
   4. Backend-For-Frontend
   5. GRPC and Protocol Buffers
   6. Testing
      1. Specifications / Acceptance Tests
      2. Integration Tests
      3. Unit Tests
3. Other
   1. Tracing
   2. Metrics/Instrumentation
   3. Mono-repository
   4. Shared code
   5. Monolith
   6. Type Registration
4. Changes from FTGO

Docker - Everything is built and run from a docker compose environment.

Open a command prompt and then execute the following docker command

NOTE: The first time you bring everything up the init script for Postgres will run automatically. The services will crash-loop for a bit because of that. Eventually things will stabilize.

docker-compose up

UseCtrl-C to stop all services.

Not recommended but each service can be run usinggo run .. You'll need to use an.env file or set some environmentvariables to properly run the service.

Usego run . --help to see all the flags and environment variables that can be set.

The colors used for each component in the above diagram align with the ring colors used in clean architecture diagram.

In this implementationofclean architecture I am workingmainly withHexagonal Architecture or Ports & Adapters termsfrom those methodologies.

• Primary Adapters (Driver Adapters)
  • Adapter implementations are in/internal/handlers
  • Port interfaces are in/internal/application/service.go
  • Primary AdaptersUSE the interface the applicationIMPLEMENTS
• Secondary Adapters (Driven Adapters)
  • Adapter implementations are in/internal/adapters
  • Port interfaces are in/internal/application/ports
  • Secondary AdaptersIMPLEMENT the interface the applicationUSES

Bringing the interfaces and implementations together looks about the same for all services. Below is an example fromtheConsumer Service.

package mainimport ("github.com/stackus/ftgogo/consumer/internal/adapters""github.com/stackus/ftgogo/consumer/internal/application""github.com/stackus/ftgogo/consumer/internal/domain""github.com/stackus/ftgogo/consumer/internal/handlers""github.com/stackus/ftgogo/serviceapis""shared-go/applications")funcmain() {svc:=applications.NewService(initService)iferr:=svc.Execute();err!=nil {panic(err)    }}funcinitService(svc*applications.Service)error {serviceapis.RegisterTypes()domain.RegisterTypes()// DrivenconsumerRepo:=adapters.NewConsumerRepositoryPublisherMiddleware(adapters.NewConsumerAggregateRepository(svc.AggregateStore),adapters.NewConsumerEntityEventPublisher(svc.Publisher),    )app:=application.NewServiceApplication(consumerRepo)// Drivershandlers.NewCommandHandlers(app).Mount(svc.Subscriber,svc.Publisher)handlers.NewRpcHandlers(app).Mount(svc.RpcServer)returnnil}

• Services exist within a capability or domain folder. Within that folder you'll find the following layout.

  /"domain"        - A capability or domain that is a subdomain in the larger application domain|-/cmd           - Parent for servers, cli, and tools that are built using the code in this domain| |-/cdc         - CDC (Change Data Capture) server. If the service publishes messages it will also have this| |-/service     - Primary service for this capability|-/internal      - Use the special treatment of "internal" to sequester our code from the other services  |-/adapters    - Driven Adapter implementations.  |-/application - Application core folder. Processes under this will implement business rules and logic  | |-/commands  - CQRS commands. Processes that apply some change to the subdomain  | |-/ports     - Application interfaces that the Driven Adapters implement.  | |-/queries   - CQRS queries. Processes that request information from the subdomain  | |-service.go - Application interface and implementation that is used by the handlers, the Driver Adapters.  |-/domain      - The definitions and the domain rules and logic  |-/handlers    - Driver Adapter implementations

This layout is an example of organizing code to achieve clean architecture. You do not need to use this layout to haveimplemented clean architecture with Go. I've made and am likely to make more minor adjustments to this layout and do notconsider it perfect or the "one".

• accounting-service - theAccounting Service
• consumer-service - theConsumer Service
• delivery-service - theDelivery Service
• kitchen-service - theKitchen Service
• order-service - theOrder Service
• order-history-service - theOrder History Service
• restaurant-service theRestaurant Service

• accounting-cdc - theAccounting CDC Service
• consumer-cdc - theConsumer CDC Service
• kitchen-cdc - theKitchen CDC Service
• order-cdc - theOrder CDC Service
• restaurant-cdc theRestaurant CDC Service

• customer-web - theCustomer Web Gateway Service
• store-web - theStore Web Gateway Service

Asynchronous messaging handles all inter-service communication. The exception is the communication from the BFF/UI layerto the downstream services.

The same three sagas found inFTGO have been implementedhere in theorder-service.

• CreateOrderSaga
  • saga responsible for the creation of a new order
    
• CancelOrderSaga
  • saga responsible for the cancelling and releasing of order resources like tickets and accounting reserves
    
• ReviseOrderSaga
  • saga responsible for the processing the changes made to an open order
    

An implementation of theoutbox pattern can be used to ensureall messages arrive at their destinations. It provides the solution tothedual write problem. Any service that publishes messages is actuallypublishing the message into the database. A CDC sibling service then processes the messages from the database andpublishes the message into NATS Streaming. This process provides at-least-once delivery.

TODO

This will be a new feature added to theedat library.

Each service divides the requests it receives into commands and queries. Using a simple designdescribedhere byThree Dots Labs all ofour handlers can be setup to use a command or query.

This is a very limited in scope implementationofCQRS. It is valid in that we have two thingswhere before we had one. Command and query have been segregated to separate responsibilities.

TheOrder History Service provides an order by consumer read-model.

Several services use event sourcing and keep track of the changes to aggregates using commands and recorded events.Check out theOrder aggregate for an example.

The project now demonstrates thebackend-for-frontend pattern withthe addition of a Customer-Web service. These types of services are purpose built API Gateways that serve a specificclient experience.

The addition of these BFFs also provide a place to implement cross-cutting concerns such as authorization andauthentication. I've tried to add demonstrations of the capabilities of what a BFF might do for a microservicesapplication.

With the addition of the first BFF, GRPC is now used in place of HTTP by the handlers. GRPC is asubjectivelybetterchoice for communication between your applications Api Gateway or BFF than using HTTP with REST. The move to GRPC wasdone leaving the contracts previously used by HTTP endpoints unchanged as much as possible.

Executable specifications written in Gherkin have been added to most services. You can find these in the/featuresdirectories.

These specifications can be executed using thegodog Cucumber tool. The Makefiletargetrun-feature-tests can be used to run all the specifications across all services. Below is an example outputfrom running theConsumer Registration specification.

Feature: Register ConsumerScenario: Consumers can be registered            # features\register_consumer.feature:4WhenI register a consumer named"Able Anders" # register_consumer.go:15 -> *FeatureStateThenI expect the command to succeed           # feature_state.go:72 -> *FeatureStateScenario: Consumers must be registered with a name                              # features\register_consumer.feature:8WhenI register a consumer named""                                           # register_consumer.go:15 -> *FeatureStateThenI expect the command to fail                                             # feature_state.go:64 -> *FeatureStateAndthe returned error message is"cannot register a consumer without a name" # feature_state.go:80 -> *FeatureStateScenario: Duplicate consumer names do not cause conflicts # features\register_consumer.feature:13GivenI register a consumer named"Able Anders"         # register_consumer.go:15 -> *FeatureStateWhenI register another consumer named"Able Anders"    # register_consumer.go:15 -> *FeatureStateThenI expect the command to succeed                    # feature_state.go:72 -> *FeatureState

TODO

TODO

New requests into the system will be given value that is put into three containersRequestID, aCorrelationID, andaCausationID.

At each request boundary, -> HTTP, or -> GRPC, or -> Message, a newRequestID is generated, if a previousRequestIDexists in the current context it is moved into theCausationID.

• At each boundary a newRequestID is generated
• TheCausationID is set to the previousRequestID
• TheCorrelationID value remains the same into every new request that results from the original request.

These three values can be used to build a map of a request through the system.

The tracking and management of these IDs is a featureofedat.

Prometheus metrics for each service are available athttp://localhost:[port]/metrics. The order-service has a fewadditional counters. See the order-servicecode for more information.

This demonstration application is a mono-repository for the Golang services. I chose to use as few additional frameworksas possible, so you'll find there is also quite a bit of shared code in packages under/shared-go

/shared-go is named the way it is because I intended to build one of the services in another language. I didn't butleft the name the way it was.

This code exists simply to make it easier for me to build this demonstration. Applying DRY to microservices is acode-smell.

Still a work-in-progress.

Industryblogs,books,andtalks typically all suggest applications startwith a monolith, single deployable application, before developing an application using microservices. Whether you have alegacy monolith application or are starting a new application the last step before microservices is to refactor ordesign the monolith to be loosely-coupled.

The service capabilities can all be run together in a monolith to demonstrate what that might look like. It'll at beatrepresent the final form of a monolith that has been broken up by feature. This kind of monolith has several names.The "Majestic Monolith", the "Loosely-Coupled Monolith", or the "Modular Monolith".

Note: The monolith came into existence after the development of the microservices. It may not give the best example of what a monolith might look like just before switching to microservices.

Commands, Events, Snapshots, and other serializable entities get registered in groups ineach/"domain"/internal/domain/register_types.go and in the child packages ofserviceapis. This type registration isa feature ofedat/core and is not unique to this application.

The purpose of doing this type registration is to avoid boilerplate marshalling and unmarshalling code for every struct.

I intend for this demonstration to exist as a faithful Golang recreation of the original. If a difference exists eitherbecause of opinion or is necessary due of the particulars of Go, I will try my best to include them all here.

• I've kept most API requests and responses the same "shape" but routes are prefixed with/api and usesnake_caseinstead ofcamelCase for property names.
• In FTGO many apis and messages that operated on Tickets used the OrderID as the TicketID. I could have done the samebut chose to let the Ticket aggregates use their own IDs. The TicketID was then included in responses and messageswhere it was needed.
• Order-History is not using DynamoDB. The purpose of Order-History is to provide a "view" or "query" service, and itshould demonstrate using infrastructure best suited for that purpose. For now, I'm using Postgres but intend touseElasticsearch soon.
• TheOrderService->createOrder method I felt was doing too much.Thecommandimplementation creates the order like before, but the published entity event that results from that command is now thecatalyst for starting the CreateOrderSaga.
• To better demonstrate a "Backend-For-Frontend", orders take a consumersaddressId. Consumers will register one ormore addresses now. Thecustomer-web service uses datafrom another service to complete the modified CreateOrder command.

• Tests. Examples of testing these services. Both Unit and Integration are still missing but executable specificationshave been added.
• Api-Gateway. I haven't gotten around to creating the gateway. "Backend-for-Frontend"s have been added.

Just like the original the following are outside the scope of the demonstration.

• Logins & Authentication The "Backend-for-Frontend"s have implemented this in a very insecure way and only fordemonstration purposes.
• Accounts & Authorization The "Backend-for-Frontend"s have implemented this in a very insecure way and only fordemonstration purposes.
• AWS/Azure/GCP or any other cloud deployment instructions or scripts
• Tuning guidance
• CI/CD guidance
• Chaos Testing - although feel free to terminate a service or cdc process for a bit and see if it breaks (it shouldn't)

DEPRECATED

TODO: Rewrite. The instructions below no longer work with the changes made to the demo, primarily from the addition of the BFFs.

Postman can be used to load the api collection for an easy demo.

With the application running using one of the commands above you can make the following calls to see the processes,events, and entities involved.

• Register Consumer
• Sign In Consumer
• Add Consumer Address
• Create Order
• Consumer Service: Register Consumer
• Restaurant Service: Create Restaurant
• Order: Create Order

LoadingFTGOGO.postman_collection.json into Postman will provide pre-built calls with semi-random data that can beused to test the above.

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

From time to time I expect to make improvements that may be breaking. I provide no expectation that local copies of thisdemonstration application won't be broken after fetching any new commit(s). If it does fail to run; simply remove therelated docker volumes and re-run the demonstration.