You can configure these settings for individual environments or for all environments in your installation.

Configure individual environments to support large message payloads

If the proxies configured to support large message payloads have endpoints in only one or a few environments in your installation, you can configure the environments to support large payloads. This avoids adding additional memory to environments that will not need to support large payloads.

To configure individual environments to support large message payloads, you can use theenvs.components.runtime properties. Make the following changes to youroverrides.yaml file:

Determining the optimal heap size for a Message Processor for large payload scenarios depends on your specific use case; however, Apigee offers general guidelines to assist in this process.

To calculate heap size per Message Processor (MP), use the following values:

• Q - Max QPS per MP pod
• L - Round Trip latency per request (95th percentile of expected latency)
• C - Total concurrency per MP, the maximum number of requests that can simultaneously persist in the MP at any given time. This is calculated asC = Q * L.
• P_req - Payload size (in MB) of request sent by Client to Apigee
• P_resp - Payload size (in MB) of response received from Target
• S - Safety Factor (Apigee recommends a range of 1.5 - 2)
• BASE_HEAP - Base heap size to account for non-request related resources such as proxy configurations. (Apigee recommends3072Mi)

Accounting for the safety factorS, along with a base heap size, the total heap per MP is calculated as follows:

Heap size per MP =BASE_HEAP + C * (P_req + P_resp) * S

The base overrides configuration for Large Payload scenario is:

envs:  components:    runtime:      resources:        requests:          cpu: 2000m          memory: 2Gi        limits:          cpu: 4000m          memory: 4Gi      cwcAppend:        bin_setenv_max_mem: 3072Mi # base heap size

Example calculation

The following is an example calculation with the following values:

• Max QPS,Q:2
• Latency,L:7s
• Request payload size,P_req:40MB
• Response payload size,P_resp:40MB
• Safety factor,S:1.5
• Base heap size,BASE_HEAP:3072Mi

Heap size =3072 + (2 * 7) * (40+40) * 1.5 =4752Mi

limits.memory is expected to be 1Gi over the recommended heap.

In this example, you would add the following to youroverrides.yaml:

envs:  components:    runtime:      resources:        requests:          memory: 4Gi        limits:          memory:5.75Gi # approximately 1Gi over 4.75Gi      cwcAppend:        bin_setenv_max_mem:4752Mi

Considerations

Traffic Bursts

HPA ensures that MPs scale as QPS goes up. However, you can expect HPA to take about 60 seconds to trigger scaling up. An unreasonably high burst of traffic can lead to Out of Memory (OOM) errors on your MP.

If you anticipate such bursts of traffic, please scale up heap usage by using a safety factor that seems appropriate. For example,S = 2.

Heap =BASE_HEAP + (Q * L) * (P_req + P_resp) *2

Policy Usage

The calculation above does not include Policy usage. Apigee allows customers to make copies of their request / response payloads and this can significantly alter the heap usage.

For example, if you have a JavaScript policy that makes 3 copies of your entire target response, the equation to use would be:

Heap per MP = Base Heap + (Q * L) * (P_req + P_resp * 4) * S

Example policies that could potentially increase memory usage are:

• AssignMessage
• JavaScript Callout
• JavaCallout

Monitoring

The guidelines provided here are intended to serve as a starting point. Monitoring and alerting of memory usage is highly encouraged. Use an increased Safety Factor if Heap usage is consistently high.

See also: