This document is one part of a two-part series on extendingCloud Logging andCloud Monitoring to include on-premises infrastructure and apps.

• Log on-premises resources with BindPlane (this document): Read about howLogging supports logging from on-premises resources.
• Monitor on-premises resources with BindPlane:Read about how Monitoring supports monitoring of on-premisesresources.

You might consider using Logging and Monitoringfor logging and monitoring of your on-premises resources for the followingreasons:

• You want a temporary solution while you move infrastructure toGoogle Cloud and you want to log and monitor your on-premisesresources until they're decommissioned.
• You might have a diverse computing environment with multiple clouds andon-premises resources.

In either case, with the Logging and MonitoringAPIs andBindPlane,you can gain visibility into your on-premises resources. This document isintended for DevOps practitioners, managers, and executives who are interestedin a logging strategy for resources in Google Cloud and their remainingon-premises infrastructure and apps.

Ingesting logs with Logging

You can get logs into Logging by using the API in two supportedways:

• Use BindPlane from observIQ to ingest logs from youron-premises or other cloud sources.
• Use the Cloud Logging API directly from your app or by using a custom agent.

Using BindPlane to ingest Logging logs

The following diagram shows the architecture of how BindPlane ingests logs andthen how those logs are ingested into Logging.

BindPlane enables users to remotely deploy and manage agents on thehosts they want to gather logs from. For more information, read aboutthe architecture of BindPlane.This option requires the least amount of effort to deploy because it requiresconfiguration to set up rather than development.

Advantages:

• Requires configuration, not development.
• Included in the cost of usingLogging.
• Is a supported configuration by Logging product and support.
• Can extend to logs not provided by the default configuration.

Disadvantages:

• Requires the use of a third-party tool.
• Might need to provide a custom configuration if the log source isn'tprovided by default. The provided list of logs is available inSources.

Using the Logging API directly

The following diagram shows the architecture of how logs are collected byinstrumentation and ingested into Logging.

Using the APIs directly means that you need to instrument your applications tosend logs directly to the API or develop a custom agent to send logs to the API.This is the option that requires the highest level of effort because it requiresdevelopment effort.

Advantages:

• Provides flexibility because you can implement the instrumentation withclient logging libraries.

Disadvantages:

• Requires separate solution for infrastructure logs, such as a custom agent.
• Requires code instrumentation, which might mean higher cost to implement.
• Requires the use of batching and other scalable ingestion techniques forproper ingestion performance.
• Support is provided for the Logging API only, notcustom-developed code.

Using BindPlane

This document covers using BindPlane from observIQ to ingest logsinto Logging. Because it's included in the cost ofLogging, BindPlane doesn't require development and provides aproduct-supported solution.

Agents, sources, and destinations

For detailed information about agents, sources, and destinations, seethe BindPlaneQuickstartGuide.

Example use case

Enterprise customers use BindPlane to ingest logs in the followingon-premises logging scenarios:

1. Custom parsing and filtering of log data from custom application logs.
2. Collection of operating-system events from Linux or Windows virtual machines.
3. Ingestion of syslog streams from network or other compatible devices.
4. Collection of Kubernetes system and application logs.

Send logs from on-premises to Logging

After you set up BindPlane and start sending logs, those logs are sent toLogging. To view, process, and export logs, go to theGoogle Cloud console.The logs are listed asgeneric_node orgeneric_taskresource types. Formore information about the labels included in each resource type, see theLogging resource list.

Cloud Logging supports non-Cloud Logging logs through theuse of two resource types:

• Generic Node: Identifies a machine or other computational resourcefor which no other resource type is applicable. The label values mustuniquely identify the node.
• Generic Task: Identifies an app process for which no other resourceis applicable, such as a process scheduled by a custom orchestrationsystem. The label values must uniquely identify the task.

View logs in Logging

On theLogs Explorer page, theAll resourceslist includesGeneric Node as a resource type.

The list of logs that appear on the page were captured as thegeneric_node resource type.Expand a row to see log entry details.

The log entries use astructured logging format, which provides a richer format for searching the logs because the logpayload is stored as ajsonPayload. The structured logging format makes logsmore accessible because you can use the fields in the payload as a part ofthe search. The BindPlane agent provides a mapping from the original logentry to the structured log entry in Logging.

Conclusion

With logs available in Logging, you can take advantage of thefull use of the Logging features. Logs appear in theGoogle Cloud console. You can export logs with Logging exportsand use them to create metrics and alerts in Monitoring by usinglogs-based metrics.

What's next

• Logging and Monitoring
• BindPlane set-up instructions for Cloud Monitoring and Logging
• Set up logs-based metrics in Logging
• For more reference architectures, diagrams, and best practices, explore theCloud Architecture Center.