NotificationsYou must be signed in to change notification settings
Fork4.5k
Star63.7k

Make cleanups fixture logging actionable#13553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Draft

joe4dev wants to merge1 commit intomain

base:main

Choose a base branch

fromfix/improve-ci-cleanup-logging

Draft

Make cleanups fixture logging actionable#13553

joe4dev wants to merge1 commit intomainfromfix/improve-ci-cleanup-logging

+21 −1

Conversation

Copy link

Member

joe4dev commentedDec 19, 2025

Motivation

A full CI run (community + pro) creates ~170 warnings "Failed to execute cleanup" (using full-text search from downloaded logs).
The AI-based analysis (likely incomplete, but the list can serve as a starting point) listed 64 occurrences linked to the following 42 test cases:

tests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_streams_on_global_tablestests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_table_crudtests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_route_table_associationtests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_tagstests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_custom_idtests/aws/services/lambda_/event_source_mapping/test_lambda_integration_dynamodbstreams.py::TestDynamoDBEventSourceMapping::test_deletion_event_source_mapping_with_dynamodbtests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycle_delete_functiontests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycletests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_function_name_variationstests/aws/services/opensearch/test_opensearch.py::TestOpensearchProvider::test_domain_lifecycletests/aws/services/route53/test_route53.py::TestRoute53::test_associate_vpc_with_hosted_zonetests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_delete_resolver_endpointtests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_create_resolver_query_log_configtests/aws/services/scheduler/test_scheduler.py::tests_create_schedule_with_valid_schedule_expressiontests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[Invtests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[tests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycle_delete_functiontests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycletests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_function_name_variationstests/aws/services/opensearch/test_opensearch.py::TestOpensearchProvider::test_domain_lifecycletests/aws/services/route53/test_route53.py::TestRoute53::test_associate_vpc_with_hosted_zonetests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_delete_resolver_endpointtests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_create_resolver_query_log_configtests/aws/services/scheduler/test_scheduler.py::tests_create_schedule_with_valid_schedule_expressiontests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[Invtests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[tests/aws/test_multi_accounts.py::TestMultiAccounts::test_multi_accounts_dynamodbtests/aws/services/apigateway/test_apigateway_basic.py::TestAPIGateway::test_multiple_api_keys_validatetests/aws/services/sns/test_sns.py::TestSNSSMS::test_opt_in_phone_numbertests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_streams_on_global_tablestests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_table_crudtests/aws/services/lambda_/event_source_mapping/test_lambda_integration_dynamodbstreams.py::TestDynamoDBEventSourceMapping::test_deletion_event_source_mapping_with_dynamodbtests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_streams_on_global_tablestests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_table_crudtests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_route_table_associationtests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_tagstests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_custom_idtests/aws/services/lambda_/event_source_mapping/test_lambda_integration_dynamodbstreams.py::TestDynamoDBEventSourceMapping::test_deletion_event_source_mapping_with_dynamodbtests/aws/test_multi_accounts.py::TestMultiAccounts::test_multi_accounts_dynamodb

Changes

Clarify the warning log message to include the actionable error message
Enrich the warning log message to suggest that the cleanup might not be necessary upon hitting 404s

Follow-up from the AI Winter hackathon on Automated Log Noise Reduction

Companion PR demonstrating further fixes.

PR demonstrating a fix for ESM:https://github.com/localstack/localstack/pull/13546/changes#diff-2d1f022dc5ab81edd6a3789403f602fd45ac88ed37d8151b5b98d1ac617a2ce3