Description

This PR addresses issues highlighted by our compiler compatibility matrix testing for C++ compilers triaged a "Update expected results". This covers a bucket of issues where the expected results on a specific compiler differ from our expected results, but not in a semantically meaningful way.

These changes are grouped into three categories:

• Improving our C++ standard library stubs - in a number of cases, the difference between our unit tests and real compilers was due to differences in the way we modelled our C++ stubs. Rather than just accepting the differences I've modified the standard library stubs, so the unit tests now better reflect real compilers.
• Accept empty results for impossible to test queries - a couple of queries require compiling multiple compilation units with different compilers. That is not possible within our framework, so I've effectively "disabled" those by committing empty expected result files.
• Accept different expected result files where standard library macros have different parameter names.

Change request type

• Release or process automation (GitHub workflows, internal scripts)
• Internal documentation
• External documentation
• Query files (.ql,.qll,.qls or unit tests)
• External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

• No rules added
• Queries have been added for the following rules:
  • rule number here
• Queries have been modified for the following rules:
  • A14-7-2 - qcc, gcc, clang
  • A15-1-5 - qcc, gcc, clang
  • A18-0-3 - qcc, gcc, clang
  • A26-5-1 - qcc, gcc, clang
  • ERR59-CPP - qcc, gcc, clang
  • M18-2-1 - qcc, gcc
  • M18-7-1 - qcc, gcc, clang
  • M27-0-1 - qcc, gcc, clang
  • MSC50-CPP - qcc, gcc, clang

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

• The structure or layout of the release artifacts.
• The evaluation performance (memory, execution time) of an existing query.
• The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

• Yes
• No

🚨🚨🚨
Reviewer: Confirm that format ofshared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

• Confirmed

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

• Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

Author

• Have all the relevant rule package description files been checked in?
• Have you verified that the metadata properties of each new query is set appropriately?
• Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
• Are the alert messages properly formatted and consistent with thestyle guide?
• Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
  As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
• Does the query have an appropriate level of in-query comments/documentation?
• Have you considered/identified possible edge cases?
• Does the query not reinvent features in the standard library?
• Can the query be simplified further (not golfed!)

Reviewer

• Have all the relevant rule package description files been checked in?
• Have you verified that the metadata properties of each new query is set appropriately?
• Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
• Are the alert messages properly formatted and consistent with thestyle guide?
• Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
  As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
• Does the query have an appropriate level of in-query comments/documentation?
• Have you considered/identified possible edge cases?
• Does the query not reinvent features in the standard library?
• Can the query be simplified further (not golfed!)