Use Ranger with caching and downscoping

Objective: Learn how to use Ranger properties to enable caching and downscopingon a Dataproc cluster.

Enable caching

This section lists the steps to enable caching with Ranger in order to reduceRanger Key Management System (KMS) round trips to encrypt and decrypttokens.

Note: Ranger caching is available with Dataproc on Compute Engineimage versions starting with image version2.0.92.

  1. Installmemcached on Dataproccluster VMs. By default, the memcached server starts on VM port 11211 (localhost:11211).

    sudo apt-get install -y memcached

  2. Set the following properties in the/etc/dataproc-ranger-gcs-plugin/conf/ranger-gcs-site.xmlRanger config file on Dataproc cluster VMs.

    <property><name>authorization.service.remoteCaching.address</name><value>localhost:11211</value></property>
    <property><name>authorization.service.remoteCaching.class</name><value>com.google.cloud.hadoop.ranger.gcs.authorization.caching.MemcachedCache</value></property>
    <property><name>authorization.service.remoteCaching.encryption.key.uri</name><value>gcp-kms://projects/PROJECT_ID_OF_KMS_KEY/locations/REGION/keyRings/KEYRING_NAME/cryptoKeys/KEY_NAME</value></property>

  3. Restart the authorization service.

    sudo systemctl restart ranger-gcs-plugin-authorization-server

View cache status

You can usetelnet to view Rangercache status.

  1. Installtelnet.

    sudo apt-get install -y telnet

  2. Use telnet to connect tomemcache on VM port11211.

    sudo telnet 127.0.0.1 11211

  3. Usetelnet commands to view cache status, including the following commands:

    • stats items: List the status of cache items.Sample output:
      STAT items:17:number 2STAT items:17:number_hot 0STAT items:17:number_warm 0STAT items:17:number_cold 2
    • stats cachedump: List keys stored in the cache.Sample output:
      stats cachedump 17 2ITEM 0616eeeeb54e23a09505da5bf75cd7fafe733eacf0d07bd7b1ac9cf46d17c188 [3051 b; 1707948281 s]ITEM d23645df9c79290d59ddb1b9710ff04fee37aa0b5de866b9b6d56b54641d68b4 [3078 b; 1707948281 s]
    • flush_all: Invalidate cache items.

Downscope Cloud Storage access tokens

You might need to downscope Ranger access tokens to move up (upscope) theCloud Storage paths that an external Hive table points to.

To move all partitions and subpartitions up to the table level,set thedownscope.table.partition-name.pruning.enabledproperty totrue in theranger-gcs-site.xml config file on Dataproccluster VMs.

<property>  <name>downscope.table.partition-name.pruning.enabled</name>  <value>true</value></property>

Example:

  • Cloud Storage bucket name:gs://warehouse
  • Original access token path:warehouse/hive/table/type=debit/year=2017/month=Aug/day=01/
  • After settingdownscope.table.partition-name.pruning.enabled totrue,upscoped access token path:warehouse/hive/table/
Note: You can verify access token paths in the/var/log/ranger-gcs-plugin-authorization-server.logwhenlog4j.properties is set todebug.

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-02-19 UTC.