A coordinator agent controls the agentic AI system in this example. The coordinator agent invokes an appropriate subagent to trigger the agentic flow. The agents can communicate with each other by using theAgent2Agent (A2A) protocol, which enables interoperability between agents regardless of their programming language and runtime. The example architecture shows agents in asequential pattern and aniterative refinement pattern.

For more information about the subagents in this example, see theAgentic flow section.

• Clearly define the business goal of the agentic AI system and the task that each agent performs.
• Choose anagent design pattern that best meets your requirements.
• Use ADK to efficiently create, deploy, and manage your agentic architecture.

• Design the human-facing agents in the architecture to support natural language interactions.
• Ensure that each agent clearly communicates its actions and status to its dependent clients.
• Design the agents to detect and handle ambiguous queries and nuanced interactions.

• Ensure that the agents have sufficientcontext to track multi-turn interactions and session parameters.
• Clearly describe the purpose, arguments, and usage of the tools that the agents can use.
• Ensure that the agents' responses are grounded in reliable data sources to reduce hallucinations.
• Implement logic to handle no-match situations, such as when a prompt is off-topic.

AI agents introduce certain unique and critical security risks that conventional, deterministic security practices might not be able to mitigate adequately. Google recommends anapproach that combines the strengths of deterministic security controls with dynamic, reasoning-based defenses. This approach is grounded in three core principles: human oversight, carefully defined agent autonomy, and observability. The following are specific recommendations that are aligned with these core principles.

Human oversight: An agentic AI system might sometimes fail or not perform as expected. For example, the model might generate inaccurate content or an agent might select inappropriate tools. In business-critical agentic AI systems, incorporate a human-in-the-loop flow to let human supervisors monitor, override, and pause agents. For example, human users can review the output of agents, approve or reject the outputs, and provide further guidance to correct errors or to make strategic decisions. This approach combines the efficiency of agentic AI systems with the critical thinking and domain expertise of human users.

Access control for agents: Configure agent permissions by using Identity and Access Management (IAM) controls. Grant each agent only the permissions that it needs to perform its tasks and to communicate with tools and with other agents. This approach helps to minimize the potential impact of a security breach, because a compromised agent would have limited access to other parts of the system. For more information, seeSet up the identity and permissions for your agent andManaging access for deployed agents.

Monitoring: Monitor agent behavior by using comprehensive tracing capabilities that give you visibility into every action that an agent takes, including its reasoning process, tool selection, and execution paths. For more information, seeLogging an agent in Vertex AI Agent Engine andLogging in the ADK.

For more information about securing AI agents, seeSafety and Security for AI Agents.

Shared responsibility: Security is a shared responsibility. Vertex AI secures the underlying infrastructure and provides tools and security controls to help you protect your data, code, and models. You are responsible for properly configuring your services, managing access controls, and securing your applications. For more information, see Vertex AI shared responsibility.

Security controls: Vertex AI supports Google Cloud security controls that you can use to meet your requirements for data residency,customer-managed encryption keys (CMEK), network security usingVPC Service Controls, andAccess Transparency. For more information, see the following documentation:

• Security controls for Vertex AI
• Security controls for Generative AI
• Generative AI and zero data retention

Safety: AI models might produce harmful responses, sometimes in response to malicious prompts.

• To enhance safety and mitigate potential misuse of the agentic AI system, you can configure content filters to act as barriers to harmful inputs and responses. For more information, seeSafety and content filters.
• To inspect and sanitize inference requests and responses for threats like prompt injection and harmful content, you can useModel Armor. Model Armor helps you prevent malicious input, verify content safety, protect sensitive data, maintain compliance, and enforce safety and security policies consistently.

Model access: You can set up organization policies to limit the type and versions of AI models that can be used in a Google Cloud project. For more information, seeControl access to Model Garden models.

Data protection: To discover and de-identify sensitive data in the prompts and responses and in log data, use theCloud Data Loss Prevention API. For more information, see this video:Protecting sensitive data in AI apps.

Transport security: The A2A protocol mandates HTTPS for all A2A communication in production environments and it recommendsTransport Layer Security (TLS) versions 1.2 or higher.

Authentication: The A2A protocol delegates authentication to standard web mechanisms like HTTP headers and to standards like OAuth2 and OpenID Connect. Each agent advertises the authentication requirements in its Agent Card. For more information, seeA2A authentication.

Ingress security (for the frontend service): To control access to the application,disable the defaultrun.app URL of the frontend Cloud Run service andset up a regional external Application Load Balancer. In addition to load-balancing incoming traffic to the application, the load balancer handles SSL certificate management. For added protection, you can useGoogle Cloud Armor security policies to provide request filtering, DDoS protection, and rate limiting for the service.

User authentication:

• Users inside your organization: To authenticate internal user access to the frontend Cloud Run service, use Identity-Aware Proxy (IAP). When a user tries to access an IAP-secured resource, IAP performs authentication and authorization checks.
• Users outside your organization: To authenticate external user access to the frontend service, useIdentity Platform orFirebase Authentication. To manage external user access, configure your application to handle a sign-in flow and to make authenticated API calls to the Cloud Run service.

For more information, seeAuthenticating users.

Container image security: To ensure that only authorized container images are deployed to Cloud Run, you can use Binary Authorization. To identify and mitigate security risks in the container images, use Artifact Analysis to automatically run vulnerability scans. For more information, seeContainer scanning overview.

Data residency: Cloud Run helps you meet data residency requirements. Your Cloud Run functions run within the selectedregion.

For more guidance about container security, seeGeneral Cloud Run development tips.

Data encryption: By default, Google Cloud encrypts data at rest by using Google-owned and Google-managed encryption keys. To protect your agents' data by using encryption keys that you control, you can useCMEKs that you create and manage in Cloud KMS. For information about Google Cloud services that are compatible with Cloud KMS, seeCompatible services.

Mitigate data exfiltration risk: To reduce the risk of data exfiltration, create aVPC Service Controls perimeter around the infrastructure. VPC Service Controls supports all of the Google Cloud services that this reference architecture uses.

Access control: When you configure permissions for the resources in your topology, follow the principle ofleast privilege.

Cloud environment security: Use the tools inSecurity Command Center to detect vulnerabilities, identify and mitigate threats, define and deploy a security posture, and export data for further analysis.

Post-deployment optimization: After you deploy your application in Google Cloud, get recommendations to further optimize security by using Active Assist. Review the recommendations and apply them as appropriate for your environment. For more information, seeFind recommendations in Active Assist.

Fault tolerance: Design the agentic system to tolerate or handle agent-level failures. Where feasible, use a decentralized approach where agents can operate independently.

Simulate failures: Before deploying the agentic AI system to production, validate it by simulating a production environment. Identify and fix inter-agent coordination issues and unexpected behaviors.

Error handling: To enable diagnosis and troubleshooting of errors, implement logging, exception handling, and retry mechanisms.

Quota management: Vertex AI supportsdynamic shared quota (DSQ) for Gemini models. DSQ helps to flexibly manage pay-as-you-go requests, and it eliminates the need to manage quota manually or to request quota increases. DSQ dynamically allocates the available resources for a given model and region across active customers. With DSQ, there are no predefined quota limits on individual customers.

Capacity planning: If the number of requests to the model exceeds the allocated capacity, thenerror code 429 is returned. For workloads that are business critical and that require consistently high throughput, you can reserve throughput by usingProvisioned Throughput.

Model endpoint availability: If data can be shared across multiple regions or countries, you can use aglobal endpoint for the model.

Monitoring using logs: By default, agent logs that are written to thestdout andstderr streams are routed to Cloud Logging. For advanced logging, you can integrate the Python logger with Cloud Logging. If you need full control over logging and structured logs, use the Cloud Logging client. For more information, seeLogging an agent andLogging in ADK.

Continuous evaluation: Regularly perform a qualitative evaluation of the output of the agents and thetrajectory or steps taken by the agents to produce the output. To implement agent evaluation, you can use theGen AI evaluation service or theevaluation methods that ADK supports.

Database tools: To efficiently manage database tools for your AI agents and to ensure that the agents securely handle complexities like connection pooling and authentication, use theMCP Toolbox for Databases. It provides a centralized location to store and update database tools. You can share the tools across agents and update the tools without redeploying agents. The toolbox includes a wide range of tools for Google Cloud databases like AlloyDB for PostgreSQL and for third-party databases like MongoDB.

Generative AI models: To enable AI agents to use Google generative AI models like Imagen and Veo, you can useMCP Servers for Google Cloud generative media APIs.

Google security products and tools: To enable your AI agents to access Google security products and tools like Google Security Operations, Google Threat Intelligence, and Security Command Center, useMCP servers for Google security products.

Cost analysis and management: To analyze and manage Vertex AI costs, we recommend that you create baseline metrics for queries per second (QPS) and tokens per second (TPS). Then, monitor these metrics after deployment. The baseline also helps with capacity planning. For example, the baseline helps you determine whenProvisioned Throughput might be necessary.

Model selection: Themodel that you select for your AI application directly affects both costs and performance. To identify the model that provides an optimal balance between performance and cost for your specific use case, test models iteratively. We recommend that you start with the most cost-efficient model and progress gradually to more powerful options.

Cost-effective prompting: The length of your prompts (input) and the generated responses (output) directly affect performance and cost. Write prompts that are short, direct, and provide sufficient context. Design your prompts to get concise responses from the model. For example, include phrases such as "summarize in 2 sentences" or "list 3 key points". For more information, see thebest practices for prompt design.

Context caching: To reduce the cost of requests that contain repeated content with high input token counts, usecontext caching.

Batch requests: When relevant, considerbatch prediction. Batched requests incur a lower cost than standard requests.

Resource allocation: When you create a Cloud Run service, you can specify the amount of memory and CPU to be allocated. Start with the default CPU and memory allocations. Observe the resource usage and cost over time, and adjust the allocation as necessary. For more information, see the following documentation:

• Configure memory limits for services
• Configure CPU limits for services

Rate optimization: If you can predict the CPU and memory requirements, you can save money withcommitted use discounts (CUDs).

Model selection: When you select models for your agentic AI system, consider the capabilities that are required for the tasks that the agents need to perform.

Prompt optimization: To rapidly improve and optimize prompt performance at scale and to eliminate the need for manual rewriting, use theVertex AI prompt optimizer. The optimizer helps you efficiently adapt prompts across different models.

Model selection: Themodel that you select for your AI application directly affects both costs and performance. To identify the model that provides an optimal balance between performance and cost for your specific use case, test models iteratively. We recommend that you start with the most cost-efficient model and progress gradually to more powerful options.

Prompt engineering: The length of your prompts (input) and the generated responses (output) directly affect performance and cost. Write prompts that are short, direct, and provide sufficient context. Design your prompts to get concise responses from the model. For example, include phrases such as "summarize in 2 sentences" or "list 3 key points". For more information, see thebest practices for prompt design.

Context caching: To reduce latency for requests that contain repeated content with high input token counts, usecontext caching.

Resource allocation: Depending on your performance requirements, configure the memory and CPU to be allocated to the Cloud Run service. For more information, see the following documentation:

• Configure memory limits for services
• Configure CPU limits for services

For more performance optimization guidance, seeGeneral Cloud Run development tips.