MQTT is an OASIS standard protocol for connected device applications that providesbidirectional messaging using a publish-and-subscribe broker architecture. TheMQTT protocol is lightweight to reduce the network overhead, and MQTT clientsare very small to minimize the use of resources on constrained devices. Onesolution for organizations who want to support connected device applications onGoogle Cloud is to run a standalone MQTT broker on Compute Engine orGKE. To deploy an MQTT broker in your organization, youneed to make several key decisions which affect the overallarchitecture; in particular, load-balancing and the deployment environment. Thisdocument describes an architecture for deploying an MQTT broker, the coreapplication in an MQTT deployment, on Google Cloud. It also describes thedecisions that you need to make when you deploy this broker, and how they impactthe architecture.

This document is part of a series of documents that provide information aboutIoT architectures on Google Cloud. The other documents in this series include thefollowing:

• Connected device architectures on Google Cloud overview
• Standalone MQTT broker architecture on Google Cloud (this document)
• IoT platform product architecture on Google Cloud
• Best practices for running an IoT backend on Google Cloud
• Device on Pub/Sub architecture to Google Cloud
• Best practices for automatically provisioning and configuring edge and bare metal systems and servers

The following diagram shows an MQTT broker architecture running onGoogle Cloud.

The architecture in the preceding image is composed as follows:

• The MQTT broker is deployed as a cluster of three instances that are connectedto the Cloud Load Balancing service. For the cloud load balancer, you canchoose from one ofseveral load-balancing products,which are described later in this document.
• The broker cluster includes a device credential store and a deviceauthentication and authorization service. The cluster connects with thebackend workloads through Dataflow or Pub/Sub.
• On the client side, edge gateways provide bidirectional communicationbetween edge devices and the MQTT broker cluster through MQTT over TLS.

Generally, we recommend that you deploy the MQTT broker application for thisarchitecture in a cluster for scalability. Factors such as theclustering functionality, the scale-up and scale-down cluster management, datasynchronization, and network partition handling are addressed by the specificbroker implementations.

Architectural considerations and choices

The following sections describe the different architectural choices andconsiderations that you must make for a standalone MQTT broker architecture, andthe impact that these choices have on the architecture.

Connected devices

Internet-connected edge devices publish their telemetry events and otherinformation to the MQTT broker. To implement the standalone MQTT brokerarchitecture that's described in this document, the device needs to have anMQTT client, the server certificate public key for TLS authentication, and thecredentials needed to authenticate with the MQTT broker.

In addition, edge devices generally have connectors to local sensors, to on-premisesdata systems, and to other devices that do not have internet access or IPconnectivity. For example, the edge device can serve as a gateway for otherlocal constrained devices connected to the gateway using BLE, to a wiredconnection, or to another near-field protocol. A detailed specification of theconnected device architecture is outside the scope of this guide.

Load balancing

In the architecture, an external load-balancing service is configured betweenthe public network and the MQTT broker cluster. This service provides severalimportant networking functions, including distribution of incoming connectionsacross backend nodes, session encryption, and authentication.

Google Cloud supports several load balancer types. To choose the best loadbalancer for your architecture, consider the following:

• mTLS. mTLS handles both encryption and device authentication methods,while standard TLS handles only encryption and requires a separate deviceauthentication method:

  • If your application uses mTLS for device authentication and needs toterminate the TLS tunnel, we recommend that you use anexternal passthrough Network Load Balancer or anexternal proxy Network Load Balancer with a target TCP proxy.External proxy Network Load Balancers terminate the TLS session and proxy the connection tothe broker node, along with any authentication credentials that are contained in themessage. If you need the client connection information as part of theauthentication scheme, you can preserve it in the backend connection byenabling the PROXY protocol.
  • If your application doesn't use mTLS, we recommend that you use anexternal proxy Network Load Balancer with a target SSL proxy to offload the external TLS and SSL processing to the load balancer.External proxy Network Load Balancers terminate the TLS session and proxy the connection tothe broker node, along with any authentication credentials that are contained in themessage. If you need the client connection information as part of theauthentication scheme, you can preserve it in the backend connection byenabling the PROXY protocol.

• HTTP(S) endpoints. If you need to expose HTTP(S) endpoints,we recommend that you configure a separateexternal Application Load Balancer for these endpoints.

For more information about the load balancer types that Cloud Load Balancingsupports, seeSummary of Google Cloud load balancers.

Load balancing strategy

Any load-balancing service distributes connections from edge devices across thenodes in the cluster according to one of several algorithms orbalancing modes.For MQTT, asession affinity load-balancing strategy is better than randomload balancing. Because MQTT client-server connections are persistentbidirectional sessions, state is maintained on the broker node that stopsthe connection. In a clustered configuration, if a client disconnects and thenreconnects to a different node, the session state is moved to the new node,which adds load on the cluster. This issue can be largely avoided by using sessionaffinity load balancing. If clients frequently change their IP addresses, theconnection can break, but in most cases session affinity is better for MQTT. Sessionaffinity is available in all Cloud Load Balancing products.

Device authentication and credential management

MQTT broker applications handle device authentication and access controlseparately from Google Cloud. A broker application also provides its owncredential store and management interface. The MQTT protocol supports basicusername and password authentication in the initial Connect packet, and thesefields are also frequently used by broker implementations to support other formsof authentication such as X.509 Certificate or JWT authentication. MQTT 5.0 alsoadds support for enhanced authentication methods that use challenge andresponse-style authentication. The authentication method that's used depends onthe choice of MQTT broker application and your connected device use case.

Regardless of the authentication method that the broker uses, the brokermaintains a device credential store. This store can be in a local SQL databaseor a flat file. Some brokers also support the useof a managed database service such as Cloud SQL. You need a separate serviceto manage the device credential store and handle any integrations with otherauthentication services such as IAM. The development of thisservice is outside the scope of this guide.

For more information about authentication and credential management, seeBest practices for running an IoT backend on Google Cloud.

Backend workloads

Any connected device use case includes one or more backend applicationsthat use the data ingested from the connected devices. Sometimes, theseapplications also need to send commands and configuration updates to thedevices. In the standalone MQTT broker architecture in this document, incomingdata and outgoing commands are both routed through the MQTT broker. There aredifferent topics within the broker's topic hierarchy to differentiate between thedata and the commands.

Data and commands can be sent between the broker and the backend applications inone of several ways. If the application itself supports MQTT, or if it can bemodified to support MQTT, the application can subscribe directly to the brokeras a client. This approach enables you to use the MQTT Pub/Subbidirectional messaging capability directly by using your application to receivedata from and send commands to the connected devices.

If your application does not support MQTT, there are several other options. Inthe architecture described in this document, Apache Beam provides an MQTT driver,which allows bidirectional integration with Dataflow and other Beamdeployments. Many brokers also have plugin capabilities that support integrationwith services like Google Pub/Sub. These are typically one-wayintegrations for data integration, although some brokers support bidirectionalintegration.

Use cases

An MQTT broker architecture is particularly well suited for the device usecases that are described in the following sections.

Standards-based data ingestion from heterogeneous devices

When you want to collect and analyze data from a large fleet of heterogeneousdevices, an MQTT broker is often a good solution. Because MQTT is a widelyadopted and implemented standard, many edge devices have built-in support forit, and lightweight MQTT clients are available to add MQTT support to devicesthat don't. The publish-and-subscribe paradigm is also a part of the MQTTstandard, so MQTT-enabled devices can take advantage of this architecturewithout additional implementation work. By contrast, devices that connect toPub/Sub must implement the Pub/Sub API or use thePub/Sub SDK. Running a standards-compliant MQTT broker onGoogle Cloud thus provides a simple solution for collecting data from awide range of devices.

When your connected devices are not controlled by your application but by athird party, you might not have access to the device system software, and themanagement of the device itself would be the other party's responsibility. Inthat circumstance, we recommend that you run an MQTT broker and provideauthentication credentials to the third party to set up the device-to-cloudcommunication channel.

Bidirectional communication for multi-party application integration

The bidirectional messaging capability of MQTT makes it very suitable for amultiparty-mobile-application use case such as on-demand food delivery or alarge-scale web chat application. MQTT has low protocol overhead, and MQTTclients have low resource demands. MQTT also features publish-and-subscriberouting, multiple quality of service (QoS) levels, built-in message retention,and broad protocol support. An MQTT broker can be the core component of ascalable messaging platform for on-demand services applications and similar usecases.

Edge-to-cloud integrated messaging

Because of the standardization and low overhead that MQTT offers, it can also bea good solution for integrating on-premises and cloud-based messagingapplications. For instance, a factory operator can deploy multiple MQTT brokersin the on-premises environment to connect to sensors, machines, gateways, andother devices that are behind the firewall. The local MQTT broker can handle allbidirectional command and control and telemetry messaging for the on-premisesinfrastructure. The local broker can also be connected by two-way subscriptionto a parallel MQTT broker cluster in the cloud, allowing communication betweenthe cloud and the edge environment without exposing the on-premises devices andsystems to the public internet.

What's next

• Learn how to connect devices and build IoT applications on Google Cloud usingIntelligent Products Essentials.
• Learn about practices forautomatically provisioning and configuring edge andbare metal systems and servers.
• For more reference architectures, diagrams, and best practices, explore theCloud Architecture Center.