This page explains various issues, the error messages for these issues, andtroubleshooting steps to resolve the issues.

Connectivity issues

If your Memorystore for Redis instance experiences connectivity issues, thencheck the issues in this section to see if one of them causes the problem.

If an issue in this section doesn't match your issue, then Telnet to one of yourRedis nodes andrun some Redis commands to see if the instanceresponds.

If the node is unresponsive, then see if any of thenetworking issuesare the reason why your node's network connection is blocked. If none of theseissues resolves the problem, then contactCloud Customer Care.

You provision resources in different VPC networks.

To connect to a Memorystore for Redis instance from a Google Cloud resource,such as a Compute Engine VM, you must provision the resources on thesame authorized VPC network as the instance.

Attempting to Telnet to an instance from a resource that's in a different regionor VPC network results in the following error message:

telnet: Unable to connect to remote host: Connection timed out

You delete a VPC network peering.

When you create a Memorystore for Redis instance, you create a VPC peeringbetween your VPC network and an internal Google VPC network.

The network peering uses the following format:

redis-peer-############

If this network peering is deleted, then attempting to Telnet to the instanceresults in the following error message:

telnet: Unable to connect to remote host: Connection timed out

To re-establish the deleted network peering, create another instance. Then,delete this instance. Your original instance now has the network peering that itneeds.

You create firewall rules that block your instance's IP addresses.

If you create egress firewall rules that block either the port (6379) or IPaddress of your Memorystore for Redis instance, then you might experienceconnectivity issues. Don't create network firewall rules that block the IPaddress range of your instance.

CPU usage issues

This section explains CPU usage issues that your Memorystore for Redis instancemight encounter.

Your instance experiences high latency or doesn't respond.

If your Memorystore for Redis instance experiences high latency orunresponsiveness, then these issues might be caused by using the followingresource-intensive Redis commands:

• KEYS
• LRANGE
• EVAL
• HGETALL
• ZRANGE

These commands can put heavy CPU pressure on your instance. We recommend thatyou don't run theKEYS command in production environments. Instead, useSCAN because this command uses fewer resources thanKEYS.

UsingLRANGE to query all or a large subset of your keyspace can demand highCPU resources. To resolve this, we recommend that you reduce the number of keysin the keyset that you query in a single operation.

Using a complex Lua script withEVAL can also cause high CPU usage. Werecommend that you use less-complex Lua scripts.

HGETALL andZRANGE can return a large number of keys, adverselyaffecting the performance of your server. Instead ofHGETALL, we recommendthat you run theHSCAN command. Also, before usingZRANGE, we recommend thatyou limit the size of the range that you use for a query.

Before you run a resource-intensive command, check the size of the datastructures that the command queries to ensure that it doesn't cause latency.

If your instance experiences high latency or unresponsiveness, then do thefollowing:

1. Check your client-side logs to find if any resource-intensive commands ran.If so, then note the date and time.
2. Use Cloud Monitoring toview theredis.googleapis.com/stats/cpu_utilization metric. See if periods ofhigh CPU utilization coincide with the same periods when the resource-intensivecommands ran.

The output buffer of your instance runs out of space.

If the output buffer of your Memorystore for Redis instance runs out of space,then do the following:

• Set a smaller value for themaxmemory-gb parameter.
• Use theallkeys-lrumaxmemory policy.

When the memory of your instance is full, and a new write comes in,Memorystore for Redis evicts keys to make room for the write, based on yourinstance'smaxmemory policy. Theallkeys-lru policy evicts the least recentlyused (LRU) keys from the entire keyset.

We recommend that you monitor your instance'smaxmemory and used memory. Thishelps you to know if your instance reaches the provisioned instance capacity.Also, by reducing the value for themaxmemory-gb parameter, you get more spacefor the overhead.

Networking issues

This section explains networking issues that your instance might encounter.

You exhaust the allocated IP range or a conflicting route exists.

When you create resources within the IP address range dedicated forMemorystore for Redis, you might exhaust all of the IP addresses. Or, theremight be aroute that conflictswith the IP address of the Memorystore for Redis instance that you're trying tocreate.

These issues cause the following error message to appear:

The IP ranges for the connection do not have enough available IPs. Allocate anew range or expand existing range and try again.

To resolve this issue, allocate additional IP addresses or remove the routeconflict discrepancy. For more information, seeIP address range exhaustion.

You don't have a private services access connection established for your network.

If your Memorystore for Redis instance uses theprivate services access connection mode, but aprivate services accessconnection doesn't exist for your network, then you might receive the followingerror message:

Google private service access is not enabled. Enable private service access andtry again

To resolve this issue,establish a private services access connection.

You delete the network peering for the private services access connection.

When you establish aprivate services accessconnection, Memorystore for Redis creates a network peering connection calledservicenetworking-googleapis-com. This connection appears in your project'sVPC network peering page.

If you delete the network peering, then the following error message appears forexisting instances:

telnet: Unable to connect to remote host: Connection timed out

In addition, if you delete the network peering and youcreate an instance, then the following error message appears:

Private services access is not configured correctly. For steps on how to verify the connection, check the documentation.

To resolve this issue,create a private service access connection between your network and the private servicesaccess network by running the following command:

gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --ranges=RESERVED_RANGE_NAME --network=VPC_NETWORK --project=PROJECT_ID

You use conflicting parameters.

If you use both the--reserved-ip-range parameter and the--connect-mode=private-service-accessparameter, then you receive the following error message:

Reserved IP range is not supported for --connect-mode private services access

You can't use both parameters at the same time because Memorystore for Redisdoesn't support the--reserved-ip-range parameter for the private servicesaccess connection mode.

To resolve this issue, do one of the following:

• Use the--reserved-ip-range parameter with the--connect-mode=direct-peeringparameter.
• Don't use the--reserved-ip-range parameter.

You exceed the subnetwork quota for your project.

There's a limit to the number of subnetworks that you can create within yourGoogle Cloud project. If you exceed this quota, then you receive one of thefollowing error messages:

Internal network quota exceeded. Please request higher limit here: https://forms.gle/ZfVduUGq2iSYcYGm8

or

Unable to create instance. Network quota limit has been reached. Please request higher limit here: https://forms.gle/ZfVduUGq2iSYcYGm8

To resolve this issue, either fill out the form that appears in the errormessages or contactCloud Customer Care.

You don't attach your service project to your host project.

If you useShared VPC and you don't attach your serviceproject to your host project, then you receive the following error message:

Invalid network name <network-name>. Project <project-name> referenced is not the host project for <service-project-name>.

To resolve this issue,attach your service project to your host project.

You create an instance in a service project with the direct peering connection mode and designate a Shared VPC network.

If you create a Memorystore for Redis instance in aservice projectand you designate a Shared VPC network from the host project for the instance,then you can't use thedirect peeringconnection mode.

By default, the connection mode is set to direct peering. If you attempt touse this connection mode when you create an instance, and you also choose aShared VPC network from the host project as the value for the--networkparameter, then you receive the following error message:

Authorized_network must exist in the same project as redis instance

To resolve this issue, do one of the following:

• When you create your instance, use the private services access connectionmode
  (--connect-mode=private-service-access).
• Choose an authorized VPC network that's in the same project as your instance.

You use an IP address range for Compute Engine that Memorystore for Redis can't access.

You can't access your Memorystore for Redis instance from aCompute Engine VM that has an IP address in the172.17.0.0/16 range.This range is reserved for an internal component.

To resolve this issue, access your instance from a Compute Engine VMthat has an IP address in a different range.

Issues connecting to your instance from other Google Cloud resources

This section explains issues that your Memorystore for Redis instance mightencounter when other resources attempt to connect to it.

You can't connect to your instance from serverless environments that require a Serverless VPC Access connector

If you can't connect to your Memorystore for Redis instance by using one of theserverless environments that require a Serverless VPC Access connector,then verify whether you set up this connector for your environment. For moreinformation, seeServerless VPC Access connector requirement.

You can't connect to your instance from a Google Kubernetes Engine (GKE) cluster

To connect to your Memorystore for Redis instance from a GKEcluster, you must enable VPC-native/IP aliasing on the cluster.

It's easiest to enable VPC-native/IP aliasing when you create the cluster. Inthe advanced options section, selectVPC Native. For more information, seeCreate a VPC-native cluster.

Identity and Access Management (IAM) issues

This section explains IAM issues that your Memorystore for Redisinstance might encounter.

You want to restore a deleted policy binding for a service account

Memorystore for Redis uses the following service accounts to manage yourinstances:

service-PROJECT_NUMBER@service-networking.iam.gserviceaccount.comservice-PROJECT_NUMBER@cloud-redis.iam.gserviceaccount.com

If you delete the policy bindings for these service accounts, then you can'tcreate instances.

If you attempt to create a Memorystore for Redis instance by using theGoogle Cloud CLI, then you might receive the following error message:

(gcloud.redis.instances.create) FAILED_PRECONDITION: A required IAM policy might be missing. Please run this command:"gcloud projects add-iam-policy-bindingYOUR_PROJECT_ID --member='serviceAccount:service-YOUR_PROJECT_NUMBER@cloud-redis.iam.gserviceaccount.com' --role='roles/redis.serviceAgent'" and try again.

To re-establish the policy binding for theservice-networking service account,run the following command. ReplaceVARIABLES with appropriate values.

gcloud projects add-iam-policy-bindingPROJECT_ID --member='serviceAccount:service-PROJECT_NUMBER@service-networking.iam.gserviceaccount.com' --role='roles/servicenetworking.serviceAgent'

To re-establish the policy binding for thecloud-redis service account, runthe following command. ReplaceVARIABLES with appropriate values.

gcloud projects add-iam-policy-bindingPROJECT_ID --member='serviceAccount:service-PROJECT_NUMBER@cloud-redis.iam.gserviceaccount.com' --role='roles/redis.serviceAgent'

Operation timeout issues

The following issues result in an unresponsive Redis instance and/orinstance/node operation timeouts.

Network partition issues

Sometimes Google Cloud resources cannot communicate across zones within a regiondue to a network partition error in Google Cloud servers. This can cause yourinstance to lose connection, resulting in a timeout error.

After Google Cloud resolves the network partition error for the region or zonewhere your instance is provisioned, connectivity should resume normally.

In this issue, you may see a connectivity error message such as thefollowing:

telnet: Unable to connect to remote host: Connection timed out

If you are not able to identify the cause of the timeout error, reach out toGoogle Cloud Support.

Service project and host project not in the same VPC service control perimeter

If you are usingShared VPC and aVPC service controlperimeter, and your Redis instance creation operation times out, this mayindicate that your service project and host project are not in the sameservice perimeter. Your service project and host project must be in the sameperimeter in order for your Redis instance to communicate with connectingclients over the Shared VPC network.

To see if you are experiencing this issue,check your Redis instance's audit logsfor the following error:

violationReason: "NETWORK_NOT_IN_SAME_SERVICE_PERIMETER"

To resolve this issue, put your host network and your service network in thesame service perimeter.

Troubleshooting import and export issues

This section outlines some common issues you may run into when using import andexport for Memorystore for Redis.

Import and export buttons are disabled in the Google Cloud console

Issue: the user logged into the console does not have theredis.instances.import and/orredis.instances.export permissions required toimport and/or export RDB files.

Solution:grant the permissionsto the user, and refresh the instance details page.

The import operation completed but the data was not restored

If an import operation completes but the data is not restored, first checkeither the Google Cloud console or the command line for an error message, andresolve any issues described by the error message.

If there is a failure during the import process, the instance is recovered usingan empty RDB file. You can attempt to restore the data by importing thesame RDB file again, or using a different RDB file.

The import failed because the RDB file was too big

If you received the error message "Import RDB file gs://bucket/object.rdb sizeexceeds max memory 10GB", you should scale up your instance and retry theimport. You can also try to import a smaller RDB file into your instance.

Troubleshooting Google Cloud CLI issues

If you run into an issue where a gcloud CLI command is unavailable,or if the command behaves differently from how it is documented, try updatingthe gcloud CLI:

gcloud components update

Stopping all ongoing commands and connections for a Redis instance

As Memorystore for Redis is a Google managed product, there are somecommands that areblocked in your Redis instancein order to provide a safe and reliable environment. One of the restrictedcommands isCLIENT, which includesCLIENT KILL, used to stop commands.

If a Redis command is consuming a lot of CPU/RAM utilisation and is affectingyour production environment, you need to restartthe instance (forBasic Tier configurations), or to failover into a replica(forStandard Tier configurations). This restart/failover operation stops allthe commands running on the Redis server, and ends all the ongoingconnections.

Below you'll find commands to perform restarts or failovers for each Memorystorefor Redis configuration.

Stopping commands in Standard tier Memorystore for Redis instances

gcloud redis instances failoverINSTANCE_NAME --data-protection-mode=limited-data-loss

Stopping commands in Basic tier Memorystore for Redis instances

The only way to perform a restart in a Memorystore for Redis instances ischanging its configuration, like scaling up the instance. An example of acommand that you can run to restart your instance can be found below.

gcloud redis instances updateINSTANCE_NAME --regionREGION_ID --sizeNUMBER_GB

After you scale your instance to a different size, you can run another scaling operation to return it to the original size.

Issues with the domain restricted sharing organization policy

Depending on when your instance was created, Memorystore for Redis uses one oftwo different service account formats. To identify which service account formatyour instance uses, seeMemorystore for Redis service account format.

There is a known issue where theiam.allowedPolicyMemberDomains organizationpolicy causes errors when used with Memorystore for Redis instances that usethe[PROJECT_NUMBER]-compute@developer.gserviceaccount.com service accountformat.

In these issues, you may encounter this error:One or more users named in the policy do not belong to a permitted customer.

There are two options to resolve this issue.

1. Create an instance: newly created instances have the appropriateservice account format which is compatible with the organization policy. If itis critical to maintain your cache contents, you can perform an export to backupexisting data and an import to the new instance. A newly created instance has anew service IP address that needs to be configured in your application.
2. Force account access: use this option only if it's infeasible to recreate your instance.