This document describes how to troubleshoot OS Login using the metadata server.For information about setting up OS Login or for step-by-step instructions, seeSetting up OS Login.

You can query the metadata server from within a virtual machine (VM) instance.For more information, seeStoring and retrieving instance metadata.

Before you begin

• If you haven't already, set upauthentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

     gcloudinit

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

     Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.
  2. Set a default region and zone.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

  gcloudinit

  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.

  For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.

Common error messages

The following are examples of common errors you might encounter when you useOS Login.

Cannot find name for group

On some VMs using OS Login, you might receive the following error message afterthe connection is established:

/usr/bin/id: cannot find name for group ID 123456789

Ignore this error message. This error does not affect your VMs.

Failure getting groups

You might see logs similar to the following when you create VMs:

Dec 10 22:31:05 instance-1 google_oslogin_nss_cache[381]: oslogin_cache_refresh[381]: Refreshing group entry cacheDec 10 22:31:05 instance-1 google_oslogin_nss_cache[381]: oslogin_cache_refresh[381]: Failure getting groups, quitting

These logs indicate that your organization doesn't have OS Login Linux groupsconfigured. Ignore these messages.

Failed precondition

You might see an error similar to the following when you connect to the VM usingSSH:

ERROR: (gcloud.compute.ssh) FAILED_PRECONDITION: The specified username or UID is not unique within given system ID.

This error occurs when OS Login attempts to generate a username that alreadyexists within an organization. This is common when a user account is deletedand a new user with the same email address is created shortly after. After auser account is deleted, it takes up to 48 hours to remove the user's POSIXinformation.

To resolve this issue, do one of the following:

• Restore the deleted account.
• Remove the account'sPOSIX informationbefore deleting the account.

Invalid argument

You might see errors similar to the following when you connect to a VM usingSSH or use SCP to transfer files:

ERROR: (gcloud.compute.ssh) INVALID_ARGUMENT: Login profile size exceeds 32 KiB. Delete profile values to make additional space.

ERROR: (gcloud.compute.scp) INVALID_ARGUMENT: Login profile size exceeds 32 KiB. Delete profile values to make additional space.

To resolve these errors, do the following:

1. View your OS Login profile by running thegcloud compute os-login describe-profile command:

   gcloud compute os-login describe-profile

   The output looks similar to the following:

   name: '00000000000000'posixAccounts:...sshPublicKeys: ...:   fingerprint: ...   key: |     ssh-rsa AAAAB3NzaC1yc2...   name: ... ...

2. Review the output to identify any unused SSH keys.

3. Remove any unused keys from the output using thegcloud compute os-login ssh-keys remove command:

   gcloud compute os-login ssh-keys remove --key=KEY

   ReplaceKEY with the keys's fingerprint or the keystring.

To prevent this issue from occurring in the future,add an expiry time for SSH keys.Expired keys are automatically removed from your login profile 48 hours afterexpiry, or when you add a new key to your profile.

HTTP response code: 429

You might see the following error when you attempt to connect to a VM using SSH:

Failed to validate organization userUSERNAME has login permission, got HTTP response code: 429

This issue is caused by the metadata server rate limit of 100 queries persecond per virtual machine instance. This limit cannot be adjusted. To resolve this issue, wait a few seconds, then retry the connection.

To prevent this issue in the future, try the following:

• Implement a retry mechanism in the application code. For more information,see:
  • Exponential Backoff
  • Retry Strategy
• Re-use existing SSH connections.
• Send commands in batches to reduce SSH connections and OS Loginmetadata queries.

Default OS Login metadata entries

Compute Engine defines a set of default metadata entries that servesOS Login information. Default metadata is always defined and set by the server.Default metadata keys are case sensitive.

The following table describes the entries you can query.

Checking if OS Login is configured

Use the Google Cloud console or Google Cloud CLI to query metadatato determine if OS Login is enabled. OS Login is enabled when theenable-oslogin metadata key is set toTRUE in project or instancemetadata. If both instance and project metadata are set, the value set ininstance metadata takes precedence.

Viewing OS Login users

To view the profile information for multiple users, you can specify thepagesize parameter. Replace thepagesize with therequired numeric value.

curl "http://metadata.google.internal/computeMetadata/v1/oslogin/users?pagesize=PAGE_SIZE" -H "Metadata-Flavor: Google"

The output might contain a page token, which can be used in subsequent calls to list more users.

For example, to set thepagesize to1, run the following command:

curl "http://metadata.google.internal/computeMetadata/v1/oslogin/users?pagesize=1" -H "Metadata-Flavor: Google"

To obtain the next user, set thepagesize to1 and thepagetoken to the page token from the previous command's output.

curl "http://metadata.google.internal/computeMetadata/v1/oslogin/users?pagesize=1&pagetoken=PAGE_TOKEN" -H "Metadata-Flavor: Google"

On most distributions, you can also run the Unix commandgetent passwd to retrievethe password entries for organization users.

Viewing a specific OS Login user

To view the profile information for a specific user on your VM, run thefollowing command:

curl "http://metadata.google.internal/computeMetadata/v1/oslogin/users?username=USERNAME" -H "Metadata-Flavor: Google"

ReplaceUSERNAME with the username of the user that youwant to query.

For example, you can perform a request to look up the useruser_example_com.The following command and output show added formatting for improved readability.

curl "http://metadata.google.internal/computeMetadata/v1/oslogin/users?username=user_example_com" -H "Metadata-Flavor: Google"

The output is similar to the following:

{    "loginProfiles": [{        "name": "12345678912345",        "posixAccounts": [{            "primary": true,            "username": "user_example_com",            "uid": "123451",            "gid": "123451",            "homeDirectory": "/home/user_example_com",            "operatingSystemType": "LINUX"        }],        "sshPublicKeys": {            "204c4b4fb...": {                "key": "ssh-rsa AAAAB3Nz...",                "fingerprint": "204c4b4fb..."            }        }    }]}

On most distributions, you can also run Unix commands such asgetent passwd username orgetent passwd uid to retrieve profile information.

To retrieve the SSH keys for a user, you can also run/usr/bin/google_authorized_keysUSERNAME.If no keys are returned, the user might not have the required permissions to loginto the VM.

Checking login permissions

To view login and administrative level permissions, you must provide thepolicy=login&email=LOGIN_NAME query parameters.

1. Query the user profile to get the value of thename field:

   curl "http://metadata.google.internal/computeMetadata/v1/oslogin/users?username=user_example_com" -H "Metadata-Flavor: Google"

2. In the output, take note of thename.

3. Run the followinglogin command using the value ofname:

   curl "http://metadata.google.internal/computeMetadata/v1/oslogin/authorize?policy=login&email=LOGIN_NAME" -H "Metadata-Flavor: Google"

For example, you can query the login permissions for the useruser_example_com that was viewed in theprevious section:

curl "http://metadata.google.internal/computeMetadata/v1/oslogin/authorize?policy=login&email=12345678912345" -H "Metadata-Flavor: Google"

The command output indicates the user is authorized to log in to the VM:

{"success":true}

Checking if your VM has a service account

You can query the metadata server to find the service account associated withyour VM. On your VM, run the following command:

curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/" -H "Metadata-Flavor: Google"

The output is similar to the following:

12345-sa@developer.gserviceaccount.com/default/

If no service account is found, the output is blank.

What's next

• Learn more aboutOS Login.
• Learn howSSH connections to Linux VMs work onCompute Engine.
• For step-by-step instructions, review one of the following:
  • Setting up OS Login.
  • Setting up OS Login with 2-step verification
• ReviewManaging OS Login in an organization