Theright fitting feature usesApache Beam resource hintsto customize worker resources for a pipeline. The ability to target multipledifferent resources to specific pipeline steps provides additional pipelineflexibility and capability, and potential cost savings. You can apply morecostly resources to pipeline steps that require them, and less costly resourcesto other pipeline steps. Use right fitting to specify resource requirements foran entire pipeline or for specific pipeline steps.

Support and limitations

• Resource hints are supported with the Apache Beam Java and Python SDKs,versions 2.31.0 and later.
• Right fitting is supported with batch pipelines.

• Right fitting is supported with streaming pipelines withhorizontal autoscaling enabled.

  • You can enable it by setting the--experiments=enable_streaming_rightfitting pipeline option.

• Right fitting supportsDataflow Prime.

• Right fitting doesn't support FlexRS.

• When you use right fitting, don't use theworker_acceleratorservice option.

Enable right fitting

To turn on right fitting, use one or more of theavailable resource hints in your pipeline. When you use aresource hint in your pipeline, right fitting is automatically enabled. Formore information, see theUse resource hints section of this document.

Available resource hints

The following resource hints are available.

Resource hint nesting

Resource hints are applied to the pipeline transform hierarchy as follows:

• min_ram: The value on a transform is evaluated as the largestmin_ramhint value among the values that are set on the transform itself and all ofits parents in the transform's hierarchy.
  • Example: If an inner transform hint setsmin_ram to 16 GB, and the outer transform hint in the hierarchysetsmin_ram to 32 GB, a hint of 32 GB is used for allsteps in the entire transform.
  • Example: If an inner transform hint setsmin_ram to 16 GB, and the outer transform hint in the hierarchysetsmin_ram to 8 GB, a hint of 8 GB is used for allsteps in the outer transform that are not in the inner transform,and a 16 GB hint is used for all steps in the inner transform.
• accelerator: The innermost value in the transform's hierarchy takes precedence.
  • Example: If an inner transformaccelerator hint is differentfrom an outer transformaccelerator hint in a hierarchy,the inner transformaccelerator hint is used for theinner transform.

Hints that are set for the entirepipeline are treated as if they are set on a separate outermost transform.

Use resource hints

You can set resource hints on the entire pipeline or on pipeline steps.

Pipeline resource hints

You can set resource hints on the entire pipeline when you run the pipelinefrom the command line.

To set up your Python environment, see thePython tutorial.

Example:

pythonmy_pipeline.py \--runner=DataflowRunner \--resource_hints=min_ram=numberGB \--resource_hints=accelerator="type:type;count:number;install-nvidia-driver" \...

pcoll.apply(MyCompositeTransform.of(...).setResourceHints(ResourceHints.create().withMinRam("15GB").withAccelerator("type:nvidia-l4;count:1;install-nvidia-driver")))pcoll.apply(ParDo.of(newBigMemFn()).setResourceHints(ResourceHints.create().withMinRam("30GB")))

pcoll|MyPTransform().with_resource_hints(min_ram="4GB",accelerator="type:nvidia-tesla-l4;count:1;install-nvidia-driver")pcoll|beam.ParDo(BigMemFn()).with_resource_hints(min_ram="30GB")

Multiple accelerator support

Within a pipeline, different transforms can have different acceleratorconfigurations. These include configurations that require different machinetypes. These transform-level accelerator configurations take precedence over thepipeline-level configuration if one was provided.

Right fitting and fusion

In some cases, transforms set with different resource hints can be executed onworkers in the same worker pool, as part of the process offusion optimization.When transforms are fused, Dataflow executes them in anenvironment that satisfies the union of resource hints set on the transforms.In some cases, this includes the entire pipeline.

When resource hints can't be merged, fusion doesn't occur. For example, resourcehints for different GPUs aren't mergeable, so those transforms aren't fused.

You can also prevent fusion by adding an operation to your pipeline that forcesDataflow to materialize an intermediatePCollection. This isespecially useful when trying to isolate expensive resources like GPUs or highmemory machines from slow or computationally expensive steps which don't needthose special resources. In those cases, it may be helpful to force a fusionbreak between the slow CPU-bound steps and the steps which need the expensiveGPUs or high memory machines and pay the cost of materialization associated withbreaking fusion. To learn more, seePrevent fusion.

Streaming right fitting

For streaming jobs, you can enable right fitting by setting the--experiments=enable_streaming_rightfitting pipeline option.

Right fitting may improve the performance of your pipeline if it involves stages with different resource requirements.

Example: Pipeline with CPU-intensive stage and GPU-requiring stage

An example pipeline that may benefit from right fitting is one that executes a CPU-intensive stage, followed by a GPU-requiring stage. Without right fitting, a single GPU worker pool will need to be configured to execute all pipeline stages, including the CPU-intensive stage. This may lead to under-utilization of the GPU resources when the worker pool is executing the CPU-intensive stage.

If right fitting is enabled and a Resource Hint is applied to the GPU-requiring step, the pipeline will create two separate pools, so that the CPU-intensive stage is executed by the CPU worker pool, and the GPU-requiring stage is executed by the GPU worker pool.

For this example pipeline, the autoscaling table shows that the worker pool executing the CPU-intensive stage,Pool 0, is initially upscaled to 99 workers, and later downscaled to 87 workers. The worker pool executing the GPU-requiring stage,Pool 1, is upscaled to 13 workers:

The CPU Utilization graph shows that workers in both worker pools are demonstrating overall high CPU utilization:

Troubleshoot right fitting

This section provides instructions for troubleshooting common issues related toright fitting.

Invalid configuration

When you try to use right fitting, the following error occurs:

Workflow failed. Causes: One or more operations had an error: 'operation-OPERATION_ID':[UNSUPPORTED_OPERATION] 'NUMBER vCpus withNUMBER MiB memory isan invalid configuration forNUMBER count of 'GPU_TYPE' in family 'MACHINE_TYPE'.'.

This error occurs when the GPU type selected isn't compatible with the machine typeselected. To resolve this error, select a compatible GPU type and machinetype. For compatibility details, seeGPU platforms.

Verify right fitting

You can verify that right fitting is enabled by viewing theautoscaling metrics and verifying that theWorker pool column is visible and lists different pools:

Streaming right fitting performance

Streaming pipelines with right fitting enabled might not always perform better than pipelines without right fitting enabled. For example:

1. The pipeline is using more workers
2. The system latency is higher, or the throughput is lower
3. The worker pool sizes are changing more frequently, or are not stabilizing

If you observe this for your pipeline, you can disable right fitting by removing the--experiments=enable_streaming_rightfitting pipeline option. Also, streaming pipelines with right fitting enabled using accelerator Resource Hints might use more accelerators than is deisirable. If you observe this for your pipeline, you can configure a maximum number of accelerators used by the pipeline by setting the--experiments=max_num_accelerators=NUM pipeline option.