This page shows you how to back up and restore a Ranger schema onDataproc with Ranger clusters.

Before you begin

1. Create a bucket if needed. You must have access to a Cloud Storage bucket,which you will use to store and restore a Ranger schema.

   To create a bucket:

   1. In the Google Cloud console, go to the Cloud StorageBuckets page.

      Go to Buckets

   2. Clickadd_boxCreate.
   3. On theCreate a bucket page, enter your bucket information. To go to the next step, clickContinue.
      1. In theGet started section, do the following:
         • Enter a globally unique name that meets thebucket naming requirements.
         • To add abucket label, expand theLabels section (expand_more), clickadd_boxAdd label, and specify akey and avalue for your label.
      2. In theChoose where to store your data section, do the following:
         1. Select aLocation type.
         2. Choose a location where your bucket's data is permanently stored from theLocation type drop-down menu.
            • If you select thedual-region location type, you can also choose to enableturbo replication by using the relevant checkbox.
         3. To set upcross-bucket replication, selectAdd cross-bucket replication via Storage Transfer Service and follow these steps:

            Set up cross-bucket replication

            1. In theBucket menu, select a bucket.

            2. In theReplication settings section, clickConfigure to configure settings for the replication job.

               TheConfigure cross-bucket replication pane appears.

               • To filter objects to replicate by object name prefix, enter a prefix that you want to include or exclude objects from, then clickaddAdd a prefix.
               • To set a storage class for the replicated objects, select a storage class from theStorage class menu. If you skip this step, the replicated objects will use the destination bucket's storage class by default.
               • ClickDone.
      3. In theChoose how to store your data section, do the following:
         1. Select adefault storage class for the bucket orAutoclass for automatic storage class management of your bucket's data.
         2. To enablehierarchical namespace, in theOptimize storage for data-intensive workloads section, selectEnable hierarchical namespace on this bucket.Note: You cannot enable hierarchical namespace in existing buckets.
      4. In theChoose how to control access to objects section, select whether or not your bucket enforcespublic access prevention, and select anaccess control method for your bucket's objects.Note: You cannot change thePrevent public access setting if this setting is enforced at anorganization policy.
      5. In theChoose how to protect object data section, do the following:
         • Select any of the options underData protection that you want to set for your bucket.
           • To enablesoft delete, click theSoft delete policy (For data recovery) checkbox, and specify the number of days you want to retain objects after deletion.
           • To setObject Versioning, click theObject versioning (For version control) checkbox, and specify the maximum number of versions per object and the number of days after which the noncurrent versions expire.
           • To enable the retention policy on objects and buckets, click theRetention (For compliance) checkbox, and then do the following:
             • To enableObject Retention Lock, click theEnable object retention checkbox.
             • To enableBucket Lock, click theSet bucket retention policy checkbox, and choose a unit of time and a length of time for your retention period.
         • To choose how your object data will be encrypted, expand theData encryption section (expand_more), and select aData encryption method.
   4. ClickCreate.

Back up a Ranger schema

1. UseSSH to connectto the Dataproc master node of the cluster with the Ranger schema.Run the commands in this section in the SSH terminal session running onthe master node.

2. Set environment variables.

   BUCKET_NAME=bucket name \  MYSQL_PASSWORD=MySQL password  SCHEMA_FILE=schema filename

   Replace the following:

   • MySQL password: You can open/etc/mysql/my.cnf on the clustermaster node to copy the MySQL password.

   • bucket name: The name of the Cloud Storage bucket touse to store the Ranger schema.

   • schema filename: Specify a filename,without the.sql filename extension. The Ranger schema is savedto this file on the master node, then saved inbucket namein Cloud Storage .

3. Stop Hive services.

   sudo systemctl stop hive-metastore.servicesudo systemctl stop hive-server2.service

4. Prevent changes to the Ranger schema tables.

   mysql -u root -p${MYSQL_PASSWORD}REVOKE ALL PRIVILEGES ON ranger.* from 'rangeradmin'@'localhost';GRANT SELECT ON ranger.* TO 'rangeradmin'@'localhost';FLUSH PRIVILEGES;SHOW GRANTS FOR 'rangeradmin'@'localhost';exit;

5. Save the Ranger schema to an.sql file.

   mysqldump -u root -p${MYSQL_PASSWORD} ranger > ${SCHEMA_FILE}.sql

6. Reset Ranger privileges.

   mysql -u root -p${MYSQL_PASSWORD}REVOKE SELECT ON ranger.* from 'rangeradmin'@'localhost';GRANT ALL PRIVILEGES ON ranger.* to 'rangeradmin'@'localhost';FLUSH PRIVILEGES;SHOW GRANTS FOR 'rangeradmin'@'localhost';exit;

7. Restart Hive and Ranger services.

   sudo systemctl start hive-metastore.service  sudo systemctl start hive-server2.service  sudo systemctl restart ranger-admin.service  sudo systemctl restart ranger-usersync.service

8. Copy the Ranger schema to Cloud Storage.

   gcloud storage cp ${SCHEMA_FILE}.sql gs://${BUCKET_NAME}

Restore a Ranger schema

Note: If you back up then restore the schema to same cluster,you may be able to use the SSH session and environment variables usedin yourRanger schema backup session.

1. UseSSH to connectto the Dataproc master node of the cluster where youwill restore the cluster schema.Run the commands in this section in the SSH terminal session running onthe master node.

2. Set environment variables.

   BUCKET_NAME=bucket name \  MYSQL_PASSWORD=MySQL password  SCHEMA_FILE=schema filename

   Replace the following:

   • MySQL password: You can open/etc/mysql/my.cnf on the clustermaster node to copy the MySQL password.

   • bucket name: The name of the Cloud Storage bucketthat contains the saved Ranger schema.

   • schema filename: The name of the Ranger schemafilename, without the.sql filename extension, saved inbucket name in Cloud Storage.

3. Stop Hive services.

   sudo systemctl stop hive-metastore.servicesudo systemctl stop hive-server2.service

4. Prevent changes to the Ranger schema tables.

   mysql -u root -p${MYSQL_PASSWORD}REVOKE ALL PRIVILEGES ON ranger.* from 'rangeradmin'@'localhost';GRANT SELECT ON ranger.* TO 'rangeradmin'@'localhost';FLUSH PRIVILEGES;SHOW GRANTS FOR 'rangeradmin'@'localhost';exit;

5. Copy the Ranger schema.sql file in Cloud Storage to the clustermaster node.

   gcloud storage cp ${BUCKET_NAME}/${SCHEMA_FILE}.sql .

6. Restore the Ranger schema.This step overwrites the contents of theexisting Ranger schema.

   mysqldump -u root -p${MYSQL_PASSWORD} ranger < ${SCHEMA_FILE}.sql

7. Reset Ranger privileges.

   mysql -u root -p${MYSQL_PASSWORD}REVOKE SELECT ON ranger.* from 'rangeradmin'@'localhost';GRANT ALL PRIVILEGES ON ranger.* to 'rangeradmin'@'localhost';FLUSH PRIVILEGES;SHOW GRANTS FOR 'rangeradmin'@'localhost';exit;

8. Update Ranger configuration files. Change the Ranger DB host to a newdatabase hostname in the following files with the following properties:

   File	Property
   ranger-hdfs-security.xml	ranger.plugin.hdfs.policy.rest.url
   ranger-yarn-security.xml	ranger.plugin.yarn.policy.rest.url

9. Restart Hive and Ranger services.

   sudo systemctl start hive-metastore.service  sudo systemctl start hive-server2.service  sudo systemctl restart ranger-admin.service  sudo systemctl restart ranger-usersync.service