this change improves the behavior of functionIncludesFinderWithRegExp indetector.go by changing the regexp to also extract a library header file name. even when it is surrounded by single quotes.

background: when working on a board definition that uses the desktop gcc compiler, we found that the error message emitted, which is used to deduce a library header file name, failed to produce the desired result with the current regular expression. the desktop gcc compiler emits the error message surrounding the header file name with single quotes like this:

g++:

... fatal error: 'Foobar.h' file not found    2 | #include <Foobar.h>

while other compilers, currently used in arduino board definitions, omit the single quotes and therefore can be processed correctly:

avr-g++:

... fatal error: Foobar.h: No such file or directory #include <Foobar.h>

arm-none-eabi-g++:

... fatal error: Foobar.h: No such file or directory    2 | #include <Foobar.h>

riscv32-esp-elf-g++:

... fatal error: Foobar.h: No such file or directory    2 | #include <Foobar.h>

in order to address this issues we have changed the regular expression to the following:

var includeRegexp = regexp.MustCompile(`(?ms)[<"'](\S+)[">']`)

with this change error messages from all compilers are parsed correctly.

this issues has been encountered with desktop gcc compiler on macOS :

$ gcc --versionApple clang version 16.0.0 (clang-1600.0.26.6)Target: arm64-apple-darwin24.1.0Thread model: posixInstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Does this PR introduce a breaking change, and istitled accordingly?

as far as we know this PR does not introduce a breaking change. however, only four compilers have been tested.

PS the old regexp used to work on macOS until recently. apparently the error message emitted by desktop gcc on macOS has been changed, which broke the arduino header file name detection …

improved regexp to also parse error messages with single quotes aroun…

78941c0

…d header file name

Copy link

CLAassistant commentedDec 13, 2024•
edited
Loading

All committers have signed the CLA.

Copy link

codecovbot commentedDec 13, 2024•
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.63%. Comparing base(84fc413) to head(fea0d5b).
Report is 11 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@##           master    #2782      +/-   ##==========================================+ Coverage   67.57%   67.63%   +0.05%==========================================  Files         238      238                Lines       22362    22392      +30     ==========================================+ Hits        15111    15144      +33+ Misses       6062     6060       -2+ Partials     1189     1188       -1

Flag	Coverage Δ
unit	`67.63% <ø> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown.Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report?Share it here.

cmaglie previously requested changes

Dec 18, 2024

View reviewed changes

Copy link

Member

cmaglie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hi@dennisppaul, thanks for the detailed report.

g++:

... fatal error: 'Foobar.h' file not found    2 | #include <Foobar.h>

while other compilers, currently used in arduino board definitions, omit the single quotes and therefore can be processed correctly:

Actually, the matching is done on the#include <...> part (the part of the source code that is quoted in the error message) and not in the error message itself, the missing quotes do not affect the result.

https://regex101.com/r/CYmT4J/1

in order to address this issues we have changed the regular expression to the following:
var includeRegexp = regexp.MustCompile(`(?ms)[<"'](\S+)[">']`)

This works but I'm concerned that it will be too broader, and it would match errors that just contains quoted strings like in this examplehttps://regex101.com/r/2rhB6a/1 (look the latest case).

I would just relax the regex a bit to include the missing case, something like:

varincludeRegexp=regexp.MustCompile(`(?ms)^\s*[0-9 |]*\s*#[ \t]*include\s*[<"](\S+)[">]`)

https://regex101.com/r/nWPv0h/1

Can you add a couple of test cases indetector_test.go too?

make regexp more specific, add test

f1faeaa

Copy link

ContributorAuthor

dennisppaul commentedDec 18, 2024

i understand your concern and i do agree, the regexp is too broad. i tried yours, however, the patterns that work in regex101.com does not seem to work in a similar way in go ( interesting! ). i therefore suggest to use a pattern that is not as generic but still works in go as well:

var includeRegexp = regexp.MustCompile(`#include[ \t]*[<"](\S+)[">]`)

i also added a test:

func TestIncludesFinderWithRegExpPaddedIncludes5(t *testing.T) {output := "/some/path/sketch.ino:23:42: fatal error: 'Foobar.h' file not found" +              "   23 | #include \"Foobar.h\"" +              "      |          ^~~~~~~~~~"include := detector.IncludesFinderWithRegExp(output)require.Equal(t, "Foobar.h", include)}

dennisppaul added2 commits

December 18, 2024 22:54

remove GH test failing escaped quotation marks

78d6d43

fix formatting

c47f78b

dennisppaul requested a review fromcmaglie

December 28, 2024 08:31

Use a more strict regexp / fix unit-test

fea0d5b

cmaglie dismissed theirstale review

January 7, 2025 15:07

Outdated

Copy link

Member

cmaglie commentedJan 7, 2025

i understand your concern and i do agree, the regexp is too broad. i tried yours, however, the patterns that work in regex101.com does not seem to work in a similar way in go ( interesting! ). i therefore suggest to use a pattern that is not as generic but still works in go as well:

The problem was that in the new unit-test you forgot to add the newlines in the test string (see the fix in my latest pushed commit). I've reverted the regex with my last suggestion and the test PASS.