Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

1. Locate build logs:

   1. In Google Cloud console, go to theBuild history page.

      Go to Build history

   2. Select the failed build to see its logs.

2. In the build log, find the error message frompip. For example:

   ERROR: apache-airflow-backport-providers-google 2021.2.5 has requirementgoogle-cloud-logging<3.0.0,>=2.1.1, but you'll have google-cloud-logging1.15.0 which is incompatible.

Locate thepip error in in-cluster build logs

1. Locate build logs:

   1. In Google Cloud console, go to theEnvironments page.

      Go to Environments

   2. Select your environment.

   3. Open theLogs tab.

   4. SelectAll logs >Composer logs >Builds >Worker & Scheduler image.

   5. In theSeverity drop-down list, selectInfo.

2. In the displayed logs:

   1. Locate theinstaller.sh error message. Forexample:

      The command '/bin/sh -c bash installer.sh $COMPOSER_PYTHON_VERSIONfail' returned a non-zero code: 1

   2. The preceding info messages have the detailedpip error. Forexample:

      apache-airflow-backport-providers-google 2021.2.5 has requirementgoogle-cloud-logging<3.0.0,>=2.1.1, but you havegoogle-cloud-logging 1.15.0.

Timeouts during Python package installation

In recent versions, thepip utility changed its behavior when attempting tosatisfy dependencies. If specified dependencies can't be satisfied,pip goesthrough all available versions of a package.

When you install a package, the following happens:

1. If the package isalready preinstalled in your environmentand Python dependencies that you specify don't require any changes, thepackage remains as it is.

   Example:aiodebug==2.3.0 is installed and you specifyaiodebug oraiodebug>=2.

2. If the package isn't already installed in your environment or it doesn'tsatisfy Python dependencies you specify, thenpip tries to use thelatest version that satisfies dependencies. This is either the latestversion of the package if you didn't specify a version, or the latestversion of the package that satisfies the specified restrictions.

   Example: If you specifyaiodebug, then the latest will be installed(2.*.*). If you specifyaiodebug<2, then 1.*.* will be installed.

   If this version causes dependency conflicts with preinstalled packages,pipgoes through all available versions of the package to try tosatisfy the specified dependency. When it happens, the following messageswill be visible in the build log:

   INFO: pip is looking at multiple versions of PYTHON_PACKAGE_NAMEto determine which version is compatible with other requirements.This could take a while.

   Before the change,pip failed with a dependency conflict without checkingother versions.

Going through all available versions can take a significant amount of time andlead to a downgrade of packages or to a timeout during the Python packageinstallation.

Preinstalled packages in Cloud Composer are regularly updatedto the latest versions for security reasons. Therefore, a timeout can alsooccur when you upgrade an environment to a later Cloud Composerversion.

Possible solutions:

• Regularly update custom Python dependencies.

• UsePythonVirtualenvOperatorto isolate code snippets that require conflicting packages.

Conflicts with preinstalled PyPI packages

Some package conflicts occur between custom PyPI packages that you install andthe preinstalled packages.

You can see the full list of preinstalled packages for your version ofCloud Composer on theCloud Composer versions page.

To solve this problem, you can:

• Install a different version of the custom PyPI package.

• Install a different version of the preinstalled package. To do so,install a custom PyPI packagewith the name of the preinstalled package and specify the required version.We do not recommend downgrading preinstalled packages.

• See if laterCloud Composer versions use a differentversion of the preinstalled package. You cancheck for potential PyPI package conflictsbefore you upgrade your environment to a later version ofCloud Composer.

• UsePythonVirtualenvOperator to isolate code snippets that require conflicting packages.

Metadata server unavailable when installing PyPI packages

If your environment uses acustompip.conf file,then you might face a problem when no new PyPI packages can be installed inyour environment. In Cloud Build logs related to the packageinstallation error, you can see the following warning message:

WARNING: Compute Engine Metadata server unavailable on attempt 3 of 3. Reason:timed outWARNING: Authentication failed using Compute Engine authentication due tounavailable metadata server.

This problem is caused by apip.conf file that does not allow access to thedefault package index athttps://pypi.org/simple. For example, if yourpip.conf file onlypermits installing packages from a custom Artifact Registry repository because ofthe redefinedindex-url parameter, installing packages from thedefault package index is not available.

To solve this problem, make sure thathttps://pypi.org/simple package indexis added to yourpip.conf file. For example, if your primary package indexis defined in theindex-url parameter, add thehttps://pypi.org/simpleindex in theextra-index-url parameter.

What's next

• Installing Python dependencies
• Troubleshooting environment updates and upgrades