Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Tools - mkbuildoptglobals refactoring & attempt to fix caching#9249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
mcspr wants to merge22 commits intoesp8266:master
base:master
Choose a base branch
Loading
frommcspr:buildopts/refactoring
Open
Show file tree
Hide file tree
Changes from1 commit
Commits
Show all changes
22 commits
Select commitHold shift + click to select a range
8253d94
Tools - mkbuildoptglobals refactoring & attempt to fix caching
mcsprMay 27, 2025
8cdc906
compile args typo
mcsprMay 27, 2025
eac15af
build_out & no explicit clean
mcsprMay 27, 2025
60689bc
drop partial data, handle missing closing comment
mcsprMay 27, 2025
9be7e67
check cache state dir state in ci
mcsprMay 27, 2025
c3b64a2
missing file
mcsprMay 27, 2025
a527244
typo, fix err trap for "test ...something..."
mcsprMay 27, 2025
fe3102f
mention indirect and direct dependency on globals .h contents
mcsprMay 28, 2025
d72ca55
update doc
mcsprMay 28, 2025
35c6221
update docs
mcsprMay 28, 2025
d1c9c7b
restore command-line opts info, newlined debugging
mcsprMay 28, 2025
a3f9218
mkdir parent when writing
mcsprMay 28, 2025
56bac44
useless from None
mcsprMay 28, 2025
eb1a542
multiple quoting fixes from sc & naming
mcsprMay 28, 2025
a9afc33
spacing
mcsprMay 29, 2025
a91da86
$cache_dir & $root quoting
mcsprMay 29, 2025
7f06e65
debug flag passed as fqbn
mcsprJul 1, 2025
664e9d8
direct types
mcsprJul 1, 2025
78380e8
warnings fix v2, propogate errors instead
mcsprJul 1, 2025
2e63e3e
debugging in ci tool checks
mcsprJul 1, 2025
e023926
invalid debug lvl=
mcsprJul 1, 2025
0c67f33
rebase unused var
mcsprJul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
NextNext commit
Tools - mkbuildoptglobals refactoring & attempt to fix caching
Optional include for build-dir header to exclude it from core.a when nobuild opts are created / used. Preventive reads before writes, tooPlus, missing change to makecorever.py doing read before writePrefer to use --output-dir in scripts, tests/device *must* beout-of-tree for caching to work properly (avoid changing core nested dirs)Assume 'aggressive caching' is normal mode of operation, no need for special treatment(can be checked by looking at .d files in the build dir)Even CI uses arduino-cli ~/.cache/arduino/{sketch,core} for buildingAllow sketches without globals.h to share core.a, rebuild otherwise(same sketch cache is retained, core.a & .o's reused)Drop manual logging in favour of 'import logging'. Arduino-CLI might still needtweaks in debug mode, though. Output flushing does not always happen promptly.Allow multiple entries for the same name, which are then merged.Allow spaces between @ and the signature words.Allow to use line right after signature.Syntax warnings show relevant snippet from globals.h
  • Loading branch information
@mcspr
mcspr committedJul 1, 2025
commit8253d946fcf97811adc63bbae34ea1710ceeeecc
2 changes: 1 addition & 1 deletion.github/workflows/build-host.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
# Run host test suite under valgrind for runtime checking of code.
# Also, a quick test that themocking buildswork at all
# Also, a quick test that themock buildsare actually working

name: Build on host OS

Expand Down
20 changes: 18 additions & 2 deletions.github/workflows/build-ide.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -9,8 +9,7 @@ permissions:
contents: read

jobs:

# Examples are built in parallel to avoid CI total job time limitation
# download toolchain and check whether gcc basic .c compilation works
sanity-check:
runs-on: ubuntu-latest
defaults:
Expand All@@ -28,6 +27,23 @@ jobs:
run: |
bash ./tests/sanity_check.sh

# verify that any scripts that will or may be used by the builder are working
tooling-check:
runs-on: ubuntu-latest
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
submodules: false
- uses: actions/setup-python@v5
with:
python-version: '3.x'
- run: |
python ./tools/test_mkbuildoptglobals.py --quiet

# Examples are built in parallel to avoid CI total job time limitation
build-linux:
name: Linux - LwIP ${{ matrix.lwip }} (${{ matrix.chunk }})
runs-on: ubuntu-latest
Expand Down
14 changes: 8 additions & 6 deletionsplatform.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -66,17 +66,19 @@ build.spiffs_blocksize=
build.iramfloat=-DFP_IN_IROM

# Fully qualified file names for processing sketch global options
globals.h.source.fqfn={build.source.path}/{build.project_name}.globals.h
commonhfile.fqfn={build.core.path}/CommonHFile.h
build.opt.fqfn={build.path}/core/build.opt
globals.h.build.source.fqfn={build.source.path}/{build.project_name}.globals.h
globals.h.build.fqfn={build.path}/{build.project_name}.globals.h
common.h.fqfn={build.core.path}/CommonHFile.h
build.opt.fqfn={build.path}/sketch/build.opt
build.opt.flags="@{build.opt.fqfn}"
mkbuildoptglobals.extra_flags=

compiler.path={runtime.tools.xtensa-lx106-elf-gcc.path}/bin/
compiler.sdk.path={runtime.platform.path}/tools/sdk

compiler.libc.path={runtime.platform.path}/tools/sdk/libc/xtensa-lx106-elf
compiler.cpreprocessor.flags=-D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -D_GNU_SOURCE -DESP8266 {build.debug_optim} {build.opt.flags} "-I{compiler.sdk.path}/include" "-I{compiler.sdk.path}/{build.lwip_include}" "-I{compiler.libc.path}/include" "-I{build.path}/core"
compiler.cpreprocessor.extra_flags=
compiler.cpreprocessor.flags=-D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -D_GNU_SOURCE -DESP8266 {build.debug_optim} {build.opt.flags} "-I{compiler.sdk.path}/include" "-I{compiler.sdk.path}/{build.lwip_include}" "-I{compiler.libc.path}/include" "-I{build.path}/core" {compiler.cpreprocessor.extra_flags}

# support precompiled libraries in IDE v1.8.6+
compiler.libraries.ldflags=
Expand DownExpand Up@@ -123,8 +125,8 @@ recipe.hooks.sketch.prebuild.pattern="{runtime.tools.python3.path}/python3" -I "
# This is quite a working hack. This form of prebuild hook, while intuitive, is not explicitly documented.
recipe.hooks.prebuild.1.pattern="{runtime.tools.python3.path}/python3" -I "{runtime.tools.makecorever}" --git-root "{runtime.platform.path}" --version "{version}" "{build.path}/core/core_version.h"

# Handleprocessing sketch global options
recipe.hooks.prebuild.2.pattern="{runtime.tools.python3.path}/python3" -I "{runtime.tools.mkbuildoptglobals}""{runtime.ide.path}" {runtime.ide.version} "{build.path}" "{build.opt.fqfn}" "{globals.h.source.fqfn}""{commonhfile.fqfn}"{mkbuildoptglobals.extra_flags}
# Handlecore & sketch global options
recipe.hooks.prebuild.2.pattern="{runtime.tools.python3.path}/python3" -I "{runtime.tools.mkbuildoptglobals}"{mkbuildoptglobals.extra_flags} build --build-opt "{build.opt.fqfn}"--source-sketch-header"{globals.h.build.source.fqfn}"--build-sketch-header "{globals.h.build.fqfn}"--common-header "{common.h.fqfn}"


## Build the app.ld linker file
Expand Down
3 changes: 1 addition & 2 deletionstests/common.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -132,7 +132,7 @@ function build_sketches()
build_cmd+=${cli_path}
build_cmd+=" compile"\
" --warnings=all"\
" --build-path $build_dir"\
" --output-path $build_dir"\
" --fqbn $fqbn"\
" --libraries $library_path"\
" --output-dir $build_out"
Expand DownExpand Up@@ -311,7 +311,6 @@ function install_core()
printf "%s\n" \
"compiler.c.extra_flags=-Wall -Wextra $debug_flags" \
"compiler.cpp.extra_flags=-Wall -Wextra $debug_flags" \
"mkbuildoptglobals.extra_flags=--ci --cache_core" \
"recipe.hooks.prebuild.1.pattern=\"{runtime.tools.python3.path}/python3\" -I \"{runtime.tools.makecorever}\" --git-root \"{runtime.platform.path}\" --version \"{version}\" \"{runtime.platform.path}/cores/esp8266/core_version.h\"" \
> ${core_path}/platform.local.txt
echo -e "\n----platform.local.txt----"
Expand Down
52 changes: 35 additions & 17 deletionstests/device/Makefile
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,17 @@ SHELL := /bin/bash

ESP8266_CORE_PATH ?= $(shell git rev-parse --show-toplevel)

# arduino-cli core/ & sketch/ build and cache directories
# by default, share build location with every other sketch
CACHE_DIR ?= $(HOME)/.cache/arduino
ifneq ("$(findstring $(ESP8266_CORE_PATH),$(CACHE_DIR))", "")
$(warning "CACHE_DIR is located in ESP8266_CORE_PATH, core.a caching will be disabled")
endif

# binaries (compile --output-dir=...) & test output goes in here.
BUILD_DIR ?= $(PWD)/build

# where to look for BSTest scripts
BS_DIR ?= $(PWD)/libraries/BSTest

PYTHON ?= python3
Expand DownExpand Up@@ -67,7 +77,7 @@ list: showtestlist

all: count tests test_report

$(TEST_LIST): | virtualenv $(TEST_CONFIG) $(BUILD_DIR) $(HARDWARE_DIR)
$(TEST_LIST): | virtualenv $(TEST_CONFIG) $(CACHE_DIR) $(BUILD_DIR) $(HARDWARE_DIR)

.NOTPARALLEL: $(TEST_LIST)

Expand All@@ -78,18 +88,21 @@ showtestlist:
@printf '%s\n' $(TEST_LIST)
@echo "--------------------------------"

$(TEST_LIST): LOCAL_BUILD_DIR=$(BUILD_DIR)/$(notdir $@)
$(TEST_LIST): LOCAL_DATA_IMG=data.img
$(TEST_LIST): LOCAL_BUILD_DIR=$(BUILD_DIR)/$(notdir $@)/
$(TEST_LIST): LOCAL_TEST_RESULT_XML=$(LOCAL_BUILD_DIR)/$(TEST_RESULT_XML)
$(TEST_LIST): LOCAL_DATA_DIR=$(LOCAL_BUILD_DIR)/data/
$(TEST_LIST): LOCAL_DATA_IMG=$(LOCAL_BUILD_DIR)/data.img

define build-arduino
rm -f $(LOCAL_BUILD_DIR)/build.options.json
$(BUILD_TOOL) compile \
$(BUILD_FLAGS) \
--libraries "$(PWD)/libraries" \
--warnings=all \
--build-path $(LOCAL_BUILD_DIR) \
--fqbn=$(FQBN) \
$@
export ARDUINO_BUILD_CACHE_PATH="$(CACHE_DIR)"; \
$(BUILD_TOOL) config dump; \
$(BUILD_TOOL) compile \
$(BUILD_FLAGS) \
--libraries "$(PWD)/libraries" \
--output-dir "$(LOCAL_BUILD_DIR)" \
--warnings=all \
--fqbn=$(FQBN) \
$@
endef

define build-mock
Expand All@@ -98,7 +111,7 @@ define build-mock
$(VENV_PYTHON) $(BS_DIR)/runner.py \
$(BS_FLAGS) \
--name $(basename $(notdir $@)) \
--output $(LOCAL_BUILD_DIR)/$(TEST_RESULT_XML) \
--output $(LOCAL_TEST_RESULT_XML) \
--env-file $(TEST_CONFIG) \
$(call mock_script,$@) \
executable "$(ESP8266_CORE_PATH)/tests/host/bin/$(@:%.ino=%)" || echo ""`
Expand All@@ -111,17 +124,17 @@ define upload-data
(cd $(dir $@) && ./make_data.py ) || echo "Filesystem creation skipped"
@test -d $(dir $@)/data/ && ( \
$(MKFS) \
--create $(dir $@)/data/ \
--create $(LOCAL_DATA_DIR) \
--size 0xFB000 \
--block 8192 \
--page 256 \
$(LOCAL_BUILD_DIR)/$(LOCAL_DATA_IMG) && \
$(LOCAL_DATA_IMG) && \
$(ESPTOOL) \
--chip esp8266 \
--port $(UPLOAD_PORT) \
--baud $(UPLOAD_BAUD) \
--after no_reset \
write_flash 0x300000 $(LOCAL_BUILD_DIR)/$(LOCAL_DATA_IMG) ) \
write_flash 0x300000 $(LOCAL_DATA_IMG) ) \
&& (echo "Uploaded filesystem") \
|| (echo "Filesystem upload skipped")
endef
Expand DownExpand Up@@ -149,7 +162,7 @@ define run-test
$(VENV_PYTHON) $(BS_DIR)/runner.py \
$(BS_FLAGS) \
--name $(basename $(notdir $@)) \
--output $(LOCAL_BUILD_DIR)/$(TEST_RESULT_XML) \
--output $(LOCAL_TEST_RESULT_XML) \
--env-file $(TEST_CONFIG) \
$(call mock_script,$@) \
port $(UPLOAD_PORT) \
Expand All@@ -160,6 +173,7 @@ $(TEST_LIST):
@echo "--------------------------------"
@echo "Running test '$@' of $(words $(TEST_LIST)) tests"
mkdir -p $(LOCAL_BUILD_DIR)
mkdir -p $(CACHE_DIR)
ifneq ("$(NO_BUILD)","1")
@echo Building $(notdir $@)
ifeq ("$(MOCK)", "1")
Expand DownExpand Up@@ -187,13 +201,17 @@ $(TEST_REPORT_HTML): $(TEST_REPORT_XML) | virtualenv
test_report: $(TEST_REPORT_HTML)
@echo "Test report generated in $(TEST_REPORT_HTML)"

$(CACHE_DIR):
@mkdir -p $@

$(BUILD_DIR):
@mkdir -p $(BUILD_DIR)
@mkdir -p $@

virtualenv:
@make -C $(BS_DIR) PYTHON=$(PYTHON) virtualenv

clean:
rm -rf $(CACHE_DIR)
rm -rf $(BUILD_DIR)
rm -rf $(BS_DIR)/virtualenv
rm -f $(TEST_REPORT_HTML) $(TEST_REPORT_XML)
Expand Down
1 change: 0 additions & 1 deletiontests/device/test_libc/test_libc.ino.globals.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
/*@create-file:build.opt@

-fno-builtin
*/
Loading
[8]ページ先頭
©2009-2025 Movatter.jp