After getting a quick overview of Espressif’s ESP32 technology in the first chapter, we are now ready to start on some development with the real hardware. For this, we need to understand the basics and how to use the available tools for the job. It is a learning process and takes some time; however, we’ll have acquired the fundamental knowledge and gained hands-on experience to develop actual applications on ESP32 by the end of the chapter.

In this chapter, we’re going to install the development environment on our machines and use our development kits to run and debug the applications. The topics covered are as follows:

• ESP-IDF
• PlatformIO
• FreeRTOS
• Debugging
• Unit testing

Let’s start by looking into the development framework by Espressif Systems,ESP-IDF.

Throughout the book, we’re going to useVisual Studio Code (VSCode) as ourIntegrated Development Environment (IDE). If you don’t have it, you can find it here:https://code.visualstudio.com/. VSCode is updated monthly, but its version is1.77.3 as of writing this book.

The main development framework that we will use in this chapter and in the book isESP-IDF v4.4.4. It is available on GitHub here:https://github.com/espressif/esp-idf. If you are a Windows user, you can download the installer from this link as another installation option:https://dl.espressif.com/dl/esp-idf/.

ESP-IDF requiresPython 3 to be installed on the development machine. We will also need Python in some of the examples. The Python version that is used is3.10.11.

The examples in the book are developed on Ubuntu 22.04.2 LTS (Linux 5.15.0-71-generic x86_64). However, they compile regardless of the development platform after the installation of ESP-IDF on the development machine.

For hardware, we need both devkits in this chapter,ESP32-C3-DevKitM-1andESP32-S3-Box-Lite.

The source code in the examples is located in the repository found at this link:https://github.com/PacktPublishing/Developing-IoT-Projects-with-ESP32-2nd-edition/tree/main/ch2

ESP-IDF is the official framework by Espressif Systems. It comes with all the necessary tools and an SDK to develop ESP32-based products. After installing ESP-IDF, we willhave:

• An SDK for ESP32 development:ESP-IDF v4.4.4 supports the ESP32, ESP32-S2, ESP32-S3, and ESP32-C3 families of chips.
• GCC-based toolchains for theXtensa andRISC-V architectures.
• GNU debugger (GDB).
• Toolchain for the ESP32Ultra Low Power (ULP) coprocessor.
• cmake andninja build systems.
• OpenOCD for ESP32.
• Python utilities: The most notable one isidf.py and is the one we will use throughout the book. It collects all the development tasks in the same Python script.

The installation of ESP-IDF differs from platform to platform, so make sure to follow the steps as described in the official documentation for your target development machine. The documentation is provided at this link:https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/#installation

My personal choiceof development environment is VSCode on Canonical Ubuntu 22.04 LTS. However, you can install ESP-IDF on any platform without any problem if you follow the guidance in the official documentation. The other IDE option is to use ESP-IDF through it.

On top of ESP-IDF, Espressif offers a VSCode extension in the Visual Studio Marketplace to manage ESP32 projects in a similar fashion to the command-line tools. It makes development in the VSCode environment easier but its capabilities are limited compared to the command-line tools. Its installation is explained here:https://github.com/espressif/vscode-esp-idf-extension. We will use this extension with VSCode to develop the very first ESP32 application.

The first application

In thisexample, we will simply print'Hello World' on the serial output of ESP32 – surprise! Let’s do it step by step:

1. Make sure you have installedESP-IDF v4.4.4. Follow the steps as described in the documentation (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/#installation).
2. Make sure you have installedVSCode and theEspressif IDF extension (https://github.com/espressif/vscode-esp-idf-extension).
3. Plug ESP32-S3-BOX-Lite into a USB port of your development machine and observe that it is shown on the device list (the following command is on a Linux terminal but you can choose any relevant relevant tool on your development machine):

   $lsusb | grep -i espressifBus 001 Device 025: ID 303a:1001 Espressif USB JTAG/serial debug unit

4. Run VSCode and enable the ESP-IDF extension if it is not already enabled.

   

   Figure 2.1: Espressif IDF in VSCode

5. Create an ESP32 project. There are two ways to do that. You can either selectView | Command Palette | ESP-IDF: New Project or press theCtrl+E +N key combination to open theNew Project dialog. Fill in the input boxes as shown in the following screenshot (don’t use any whitespace in the project path, and select ESP32-S3 and the serial port that the devkit is connected to). Then click on theChoose Template button.

   

   Figure 2.2: New project

6. On thenext screen, selectESP-IDF from the drop-down box andsample_project from the list. Complete this step by clicking on theCreate project using template sample_project button. On the bottom right of the screen, a pop-up window will show up asking to open the project in a new window. AnswerYes and a new VSCode window will appear with the new project.

   

   Figure 2.3: Select sample_project

7. In the new VSCode window, you now have the environment for the ESP32 project. You can see the file names in Explorer on the left side of the window.

Figure 2.4: Explorer with VSCode files

8. Renamemain/main.c tomain/main.cpp and edit the content with the following simple and famous code:

   #include<iostream>extern"C"voidapp_main(void){    std::cout <<"Hello world!\n";}

9. To compile, flash, and monitor the application, we can simply press theCtrl + E + D key combination (there are other ways to the same thing: we can click on the fire icon in the bottom menu or we can select the same CP in the command palette in VSCode). The ESP-IDF extension will ask you to specify the flash method. There are three options:JTAG, UART and DFU. We selectUART.

   

   Figure 2.5: Selecting the flash method

10. Then the next step is to select the project – we only havefirst_example. VSCode will open a terminal tab where you can see the compilation output. If everything goes well, it will connect to the serial monitor and all the logs will be displayed as follows.

    

    Figure 2.6: The serial monitor output

11. We have compiled the application, flashed it to the devkit, and can see its output in the serial monitor. To close the serial monitor, pressCtrl +].

Let’s go back to the application code instep 8 and discuss it briefly. In the first line, we include the C++iostream header file to be able to use the standard output (stdout) to print text on the screen:

#include<iostream>

In the next line, we define the application entry point:

extern"C"voidapp_main(void){

extern "C" means that we will next add some C code and the C++ compiler will not mangle the symbol name that comes after this. It isapp_main here. Theapp_main function is the entry point of ESP32 applications, so we will have this function in every ESP32 application that we develop. We only print"Hello world!\n" on the standard output inapp_main:

    std::cout <<"Hello world!\n";}// end of app_main

Thestandard output is redirected to the serial console by default, thus we seeHello world! printed on the screen instep 10 when we monitor ESP32 by connecting its serial port. We’ve completed our first ESP32 application. We can use these steps as a blueprint when starting a new project. Let’s discuss more about the internal workings of ESP-IDF and other files in a typical ESP32 project.

ESP-IDF usescmake as its build configuration system. Therefore, we see theCMakeLists.txt files in various places. The one in the root defines the ESP-IDF project:

cmake_minimum_required(VERSION 3.5)include($ENV{IDF_PATH}/tools/cmake/project.cmake)project(first_project)

The other one ismain/CMakeLists.txt. It registers the application component by providing the source code files and include directories. In our case, it is onlymain.cpp and the current directory ofmain.cpp where it will look for any header files:

idf_component_register(SRCS "main.cpp"                    INCLUDE_DIRS ".")

We frequently edit this file to add new source code files and include directories in our projects.

You mayhave noticed that when we compiled the application, new additions appeared in the project root as can be seen in the following figure:

Figure 2.7: The files and directories after compilation

We can see a new directory,build, and a file,sdkconfig. As the name implies, all the artifacts from the build process are located under thebuild directory. The tools used during the build process generate their files in there. The most interesting files are probably the JSON files. Let’s list them:

Figure 2.8: JSON files in the build directory

The names of these files are self-explanatory regarding their content so I don’t want to repeat them. However, I suggest you check their content after building the project since they can be very helpful for troubleshooting purposes and understanding the build system in general.

sdkconfig in theproject root is important. It is generated by ESP-IDF automatically during the build if there is none. This file contains the entire configuration of the application and defines the default behavior of the system. The editor ismenuconfig, which is accessible fromView | Command Palette | ESP-IDF: SDK Configuration editor(menuconfig) or simplyCtrl + E +G:

Figure 2.9: SDK configuration (sdkconfig)

We will usemenuconfig often to configure our projects, and even provide our custom configuration items to be included insdkconfig (menuconfig is a part of the Linux configuration system,Kconfig, and adapted in ESP-IDF to configure ESP32 applications).

ESP-IDF Terminal

One last thing worthnoting with the ESP-IDF extension is theESP-IDF Terminal. It provides command-line access to the underlying Python scripts that come with the ESP-IDF installation. To open a terminal, we can selectView | Command Palette | ESP-IDF:Open ESP-IDF Terminal or press the key combination ofCtrl + E + T. It opens an integrated command-line terminal. The two powerful and popular tools areidf.py andesptool.py. We can manage the entire development environment and build process by only usingidf.py and we will use this tool a lot throughout the book. Just go ahead and writeidf.py in the terminal to see all the options. As a quick example:

$idf.py clean flash monitorExecuting action: cleanRunning ninja in directory <your_directory>/ch2/first_project/buildExecuting "ninja clean"...[0/1] Re-running CMake...-- Project is not inside a git repository, or git repository has no commits; will not use 'git describe' to determine PROJECT_VER.-- Building ESP-IDF components for target esp32s3-- Project sdkconfig file <your_directory>/ch2/first_project/sdkconfig-- App "first_project" version: 1<The rest of the build and flashing logs. Next comes the application output.>I (0) cpu_start: Starting scheduler on APP CPU.Hello world!

This simple command cleans the project (removes the previous build output if any), compiles the application, flashes the generated firmware to the devkit, and finally starts the serial monitor to show the application output.

Similarly, youcan see whatesptool.py can do by writing its name and pressingEnter in the terminal. The main purpose of this tool is to provide direct access to the ESP32 memory and low-level application image management. As an example, we can useesptool.py to flash the application binary:

$esptool.py --chip esp32s3 write_flash -z 0x10000 build/first_project.binesptool.py v3.2Found 1 serial portsSerial port /dev/ttyACM0Connecting...Chip is ESP32-S3Features: WiFi, BLECrystal is 40MHzMAC: 7c:df:a1:e8:20:30Uploading stub...Running stub...Stub running...Configuring flash size...Flash will be erased from 0x00010000 to 0x00079fff...Compressed 430672 bytes to 205105...Wrote 430672 bytes (205105 compressed) at 0x00010000 in 4.5 seconds (effective 769.1 kbit/s)...Hash of data verified.Leaving...Hard resetting via RTS pin…

Theesptool.py tool isespecially useful for production. It directlycommunicates with the ROM bootloader of Espressif SoCs and is independently available onPyPI so we can install it on any machine for firmware binary management with a simple command, without using the entire ESP-IDF:

$pip install esptool

The Python installation adds two other tools:

• espefuse.py for reading/writing eFuses of ESP32
• espsecure.py formanaging secure boot and flash encryption

We will not useesptool.py and the other two that come with the installation in the examples but you will probably need them in production to flash the firmware images on your IoT devices and enable security features. The official documentation for these tools can be found at this link:https://docs.espressif.com/projects/esptool/en/latest/esp32/index.html

With this overview under your belt, you are now ready to start using VSCode and the ESP-IDF extension together to develop ESP32 projects. The next tool isPlatformIO.

PlatformIO supports many different platforms, architectures, and frameworks with modern development capabilities. It comes as an extension in VSCode and so is very easy to install and configure with just a few clicks. After launching VSCode, go to the VSCodeExtensions screen (Ctrl+Shift+X) and search forplatformio in the marketplace. It appears in the first place in the match list. Click on theInstall button, and that is it:

Figure 2.10: Installing PlatformIO

After a few minutes, the installation completes and we have PlatformIO installed in the VSCode IDE. PlatformIO has some unique features. The most notable one is probably the declarative development environment. With PlatformIO, we only need to specify what we’re going to use in our project, including the chip type (not limited to Espressif products), which framework and which version of that framework, other libraries with version constraints, and any combination of them. We’ll see what all these things mean and how to configure a project shortly. Apart from that, PlatformIO has all the utilities that you would need when developing an embedded project, such as debugging, unit testing, static code analysis, and firmware memory inspection. When I used PlatformIO for the first time roughly 8 years ago, the debug feature was not available in the free version. It is now a free and open-source project with all features at our disposal. Thank you, guys! Enough talking, let’s develop the same application with PlatformIO.

Hello world with PlatformIO

Now, we’regoing to use PlatformIO. Here are the steps to develop the application:

1. Go to the PlatformIO Home screen.

   

   Figure 2.11: PlatformIO Home

2. Clickon theNew Project button on the right of the same screen.

   

   Figure 2.12: Quick access buttons on the PlatformIO Home screen

3. A pop-up window appears. Set the project name, selectEspressif ESP32-S3-Boxfor the board, and specify the framework asEspressif IoT Development Framework. You can choose a directory for the project or leave it at the PlatformIO default. Click onFinish to let PlatformIO do its job.

   

   Figure 2.13: PlatformIO Project Wizard

4. When the project is created, we have the following directory structure.

   

   Figure 2.14: Project directory structure

5. Renamesrc/main.c tosrc/main.cpp and copy-paste the same code that we havealready developed with the ESP-IDF extension.

   #include<iostream>extern"C"voidapp_main(){    std::cout <<"Hello World!\n";}

6. Edit theplatformio.ini file to have the following configuration settings.

   [env:esp32s3box]platform = espressif32board = esp32s3boxframework = espidfmonitor_speed=115200monitor_rts =0monitor_dtr =0

7. On thePlatformIO tasks list, you will see theUpload and Monitor task under thePROJECT TASKS | esp32s3box | General menu. It will build, flash, and monitor the application.

   

   Figure 2.15: PlatformIO project tasks

8. You can observe the application output in the integrated terminal.

   

   Figure 2.16: Application output in the terminal

As you might have already noticed, we didn’t download or install anything except PlatformIO. It handled all these low-level configuration and installation tasks for us. PlatformIO uses theplatformio.ini file for this purpose. Let’s investigate its content.

The first line defines the environment. The name of the environment isesp32s3box. We can write anything as the environment name:

[env:esp32s3box]

The second line shows the platform –espressif32:

platform = espressif32

As of writing this chapter, PlatformIO supports 48 different platforms.espressif32 is one of them. We can specify the platform version if needed and PlatformIO will find and download it for us. If none is specified, it will assume the latest version of the platform. Then the board that we use in the project is listed:

board = esp32s3box

The board in the project isesp32s3box. There are 11,420 different boards supported by PlatformIO, 162 of which are in theespressif32 platform. Next, we see the framework:

framework = espidf

The framework isespidf. This category contains 24 more frameworks in the PlatformIO registry.

Theplatform,board, andframework settings were added automatically instep 3 with the PlatformIO project wizard. PlatformIO collected them as user inputs at the project definition stage and set the initial content ofplatformio.ini with these values.

Then, we added the next three lines manually to define the serial monitor behavior:

monitor_speed=115200monitor_rts =0monitor_dtr =0

We set the serial baud rate to 115,200bps, and RTS and DTR to 0 in order to reset the chip when the serial monitor connects so that we can see the entire serial output of the application.

Before moving on, let’s include our other board, ESP32-C3-DevKitM-1, in the project and see how easy it is to update the configuration of the project for different boards. To do that, just append the following lines at the end ofplatformio.ini and save the file:

[env:esp32c3kit]platform = espressif32board =esp32-c3-devkitm-1framework = espidfmonitor_speed=115200monitor_rts =0monitor_dtr =0

When you save the file, PlatformIO will detect this and create another entry in the project tasks for the new environment as can be seen in the following figure:

Figure 2.17: New environment under PROJECT TASKS

After pluggingthe new devkit, you can upload and monitor the same application without making any other modifications in the project. Again, we didn’t manually download or install anything for ESP32-C3-DevKitM-1; it was all handled by PlatformIO. If you’re wondering where those downloads go, you can find them in the$HOME/.platformio/platforms/ directory of your development machine. The PlatformIO documentation provides complete information about what can be configured inplatformio.ini with examples:https://docs.platformio.org/en/latest/projectconf/index.html.

PlatformIO Terminal

In addition tothe GUI features, PlatformIO also provides a command-line tool –pio – which is accessible throughPlatformIO Terminal. It can be quite useful in some cases, especially if you enjoy command-line tools in general. To start PlatformIO Terminal, you can click on thePlatformIO: New Terminal button in the bottom toolbar of VSCode.

Figure 2.18: VSCode bottom toolbar

This toolbar also has other quick-access buttons for the frequently used features, such as compilation, upload, monitor, etc. When you click on theTerminal button (the labels appear when you hover the mouse pointer over the buttons), it will redirect you to a command-line terminal where you can enterpio commands. Write pio and press theEnter key to display thepio options.

Figure 2.19: PlatformIO Terminal and the pio command-line tool

We canflash ESP32-C3-DevKitM-1 by usingpio as the following:

$pio run -t upload -e esp32c3kitProcessing esp32c3kit (platform: espressif32; board: esp32-c3-devkitm-1; framework: espidf)----------------------------------------------Verbose mode can be enabled via `-v, --verbose` optionCONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32-c3-devkitm-1.htmlPLATFORM: Espressif 32 (5.1.1) > Espressif ESP32-C3-DevKitM-1HARDWARE: ESP32C3 160MHz, 320KB RAM, 4MB Flash…Leaving...Hard resetting via RTS pin...============ [SUCCESS] Took 24.90 seconds =================Environment    Status    Duration-------------  --------  ------------esp32c3kit     SUCCESS   00:00:24.902=======================================

And we can monitor the serial output with the following command:

$pio device monitor -e esp32c3kit--- forcing DTR inactive--- forcing RTS inactive--- Terminal on /dev/ttyUSB0 | 115200 8-N-1<removed>ESC[0;32mI (324) cpu_start: Starting scheduler.ESC[0mHello World!

Thepio toolhasall the functions that you can do with the GUI. To see how to use any other command, just append the-h option after the command’s name. The online documentation provides more detailed information about the commands:https://docs.platformio.org/en/latest/core/userguide/index.html#commands.

This completes the introduction to PlatformIO. In the next topic, we will discuss FreeRTOS, the officialReal-Time Operating System (RTOS) supported by ESP-IDF.

There aredifferent flavors of FreeRTOS. FreeRTOS was originally designed for single-core architectures. However, ESP32 has two cores, and therefore the Espressif port of FreeRTOS is designed to handle dual-core systems. Most of the differences between vanilla FreeRTOS and ESP-IDF FreeRTOS stem from this. The following list shows some of those differences:

• Creating a new task: There is a new function in ESP-IDF FreeRTOS where we can specify on which core to run a new task; it isxTaskCreatePinnedToCore. This function takes a parameter to set the task affinity to the specified core. If a task is created by the originalxTaskCreate, it doesn’t belong to any core, and any core can choose to run it at the next tick interrupt.
• Scheduler suspension: ThevTaskSuspendAll function call only suspends the scheduler on the core on which it is called. The other core continues its operation. Therefore, it is not the right way to suspend the scheduler and protect shared resources.
• Critical sections: Entering a critical section stops the scheduler and interrupts only on the calling core. The other core continues its operation. However, the critical section is still protected by a mutex, preventing the other core from running the critical section until the first core exits. We can use thetaskENTER_CRITICAL(mux) andtaskEXIT_CRITICAL(mux) macros for this purpose.

Anotherflavor of FreeRTOS is Amazon FreeRTOS, which adds more features. On top of the basic kernel functionality, with Amazon FreeRTOS developers also get common IoT libraries, suchascoreHTTP,coreJSON,coreMQTT, andSecure Sockets, for connectivity. Amazon FreeRTOS aims to allow any embedded devices to be connected to the AWS IoT platform easily and securely. We will talk about Amazon FreeRTOS in more detail later in the book. For now, let’s stick to ESP-IDF FreeRTOS and see a classic example of the producer-consumer pattern.

Creating the producer-consumer project

In this example, wewill simply implement the producer-consumer pattern to show some functionality of Espressif FreeRTOS. There will be a single producer and two consumer FreeRTOS tasks, one on each core of ESP32. As you might guess, the devkit is ESP32-S3-BOX-Lite (ESP32-C3 has a single RISC-V core). The producer task will generate numbers and push them to the tail of a queue. The consumers will pop numbers from the head. The following figure depicts what we will develop in this example:

Figure 2.20: Producer-consumer pattern

Theproducer task will have no affinity, meaning that the FreeRTOS scheduler will assign it to a core at runtime. We will pin a consumer task to each core. There will be a FreeRTOS queue to pass integer values between the producer and the consumers. FreeRTOS queues are thread-safe, so we don’t need to think about protecting the queue against reading/writing by multiple tasks. We will simply push values to the back of the queue and pop from the front (there is a good article here about how FreeRTOS queues work:https://www.freertos.org/Embedded-RTOS-Queues.html).

Let’s prepare the project in steps:

1. Plug the devkit in a USB of your development machine and start a new PlatformIO project with the following parameters:
   • Name:espidf_freertos_ex
   • Board:Espressif ESP32-S3-Box
   • Framework:Espressif IoT Development Framework
2. Editplatformio.ini and append the following lines (the last two lines will provide a nice, colorful output on the serial monitor):

   monitor_speed=115200monitor_rts =0monitor_dtr =0monitor_filters=colorizemonitor_raw=yes

3. Renamesrc/main.c tosrc/main.cpp and edit it by adding the following temporary code:

   #include<iostream>extern"C"voidapp_main(){    std::cout <<"hi\n";}

4. Runmenuconfig by selectingPLATFORMIO | PROJECT TASKS | esp32s3box | Platform | Run Menuconfig.

   

   Figure 2.21: Running menuconfig

5. This is the first time we runmenuconfig to configure ESP-IDF. We need to change a configuration value in order to enable a FreeRTOS function that lists the FreeRTOS tasks in an application. Whenmenuconfig starts, navigate to(Top) Component config FreeRTOS Kernel and check the following options (the latter two are dependent on the first one, and will become visible when the first is enabled):
   • EnableFreeRTOS trace utility
   • EnableFreeRTOS stats formatting functions
   • Enable display ofxCoreID invTaskList

   

   Figure 2.22: Configuring FreeRTOS in menuconfig

6. Buildthe project (PLATFORMIO | PROJECT TASKS | esp32s3box | General | Build).
7. Flash and monitor the application to see thehi text on the serial monitor (PLATFORMIO/PROJECT TASKS | esp32s3box | General | Upload and Monitor).

   

   Figure 2.23: The serial monitor output when the application is configured successfully

Now that we have the project configured, we can develop the application, next.

Coding application

So far, sogood. Now, we can implement the producer-consumer pattern in thesrc/main.cpp file. First, we clear the temporary code inside the file and then add the following headers:

#include<freertos/FreeRTOS.h>#include<freertos/task.h>#include<freertos/queue.h>#include<esp_log.h>

Thefreertos/FreeRTOS.h header file contains the backbone definitions based on the configuration. When we need a FreeRTOS function, we first include this header file, then the specific header where the needed function is declared. In our example, we will create tasks and a queue for the producer-consumer pattern; thus, we includefreertos/task.h andfreertos/queue.h respectively. The last header file,esp_log.h, is for printing log messages on the serial console. Instead of direct access to the serial output viaiostream, we will use the ESP-IDF logging macros in this application. Then we can define the global variables in the file scope:

namespace{    QueueHandle_t m_number_queue{xQueueCreate(5,sizeof(int))};constconstexprint MAX_COUNT{10};constconstexprchar *TAG{"app"};voidproducer(void *p);voidconsumer(void *p);}// end of namespace

In the anonymous namespace, we define a FreeRTOS queue,m_number_queue. This will be the medium in which to exchange data between the producer and consumers. ThexQueueCreate function (in fact, it is a macro) creates a queue to hold 5 integers. The producer will generate integers to push into the queue. TheMAX_COUNT constant shows the maximum number of integers to be generated by the producer.TAG is required by the logging macros. We will use it as a parameter when we want to log something. A logging macro prints the provided tag before any message.producer andconsumer are the functions to be passed to the FreeRTOS tasks. We will see how to do this next:

extern"C"voidapp_main(){ESP_LOGI(TAG,"application started");xTaskCreate(producer,"producer",4096,nullptr,5,nullptr);

Now, we’re implementing theapp_main function. Remember that this is the application entry point. The first statement is theESP_LOGI macro call withTAG and a message.application started will be printed on the serial monitor when the application starts. There are other macros in the logging family, such asESP_LOGE for errors andESP_LOGW for warnings. In the next line after printing the log message, we create our first FreeRTOS task by callingxTaskCreate. It has the following syntax in thefreertos/task.h header file:

xTaskCreate(task_function, task_name, stack_depth,    function_parameters, priority, task_handle_address)

Lookingat this prototype,xTaskCreate will create a FreeRTOS task that runs theproducer function that we declared earlier. The task name will beproducer with a stack size of4096 bytes. We don’t pass any parameters to the task. The task priority is5, and finally, we don’t provide any address for the task handle since we don’t need it in this example. The FreeRTOS scheduler will create the producer task with these parameters.

Then, we need the consumers:

    xTaskCreatePinnedToCore(consumer,"consumer-0",4096, (void *)0,5,nullptr,0);    xTaskCreatePinnedToCore(consumer,"consumer-1",4096, (void *)1,5,nullptr,1);

We will have two consumers. For this, we use thexTaskCreatePinnedToCore function this time. It is very similar toxTaskCreate. Its prototype is:

xTaskCreatePinnedToCore(task_function, task_name, stack_depth,    function_parameters, priority, task_handle_address, task_affinity)

In addition to the parameters thatxCreateTask uses,xTaskCreatePinnedToCore needs a task affinity defined – i.e., on which core to run the task. In our example, the first consumer task will run oncpu-0, and the second one will run oncpu-1. This function is specific to ESP-IDF FreeRTOS in order to support dual-core processors as we mentioned earlier.

We have now created all the tasks. Let’s see the list of the FreeRTOS tasks that we have in this application with the following lines of code:

char buffer[256]{0};vTaskList(buffer);ESP_LOGI(TAG,"\n%s", buffer);}// end of app_main

To list the tasks, we callvTaskList with abuffer parameter. It fills the buffer with the task information and we print the buffer on the serial output.vTaskList has been enabled by amenuconfig entry during the project initialization phase. This completes theapp_main function. Next, we will implement the producer task function in the anonymous namespace:

namespace{voidproducer(void *p){int cnt{0};vTaskDelay(pdMS_TO_TICKS(500));

In theproducer function, we define a variable,cnt, to count the numbers that we push into the queue. Then, we implement a 500 ms delay in the task execution. We add a loop for enqueueing the numbers as follows:

while (++cnt <= MAX_COUNT)        {xQueueSendToBack(m_number_queue, &cnt, portMAX_DELAY);ESP_LOGI(TAG,"p:%d", cnt);        }

In the loop, we use thexQueueSendToBack function of FreeRTOS to send the numbers into the queue. ThexQueueSendToBack function takes the queue reference, a pointer to the value to be pushed into the queue, and the maximum time for which to block the task if the queue is full. The number that is passed to the queue is the value of thecnt variable itself. Therefore, we will see the numbers starting from 1 up to 10 in the queue. We finish the producer task function as follows:

vTaskDelete(nullptr);    }// end of producer

A FreeRTOS task cannot return, else the result would be an application crash. When we are done with a task and we don’t need it anymore, we simply delete it by calling thevTaskDelete function. This function takes the task handle as a parameter, and passingnullptr means that the current task is the one to be deleted. Since there is no task after that point, we can safely return from theproducer function. Then we implement theconsumer function:

voidconsumer(void *p){int num;

Theconsumer function will run on both cores of ESP32-S3. When we defined two consumer tasks in theapp_main function, we passed theconsumer function as the task function and the core number as the parameter to be passed to theconsumer function. Therefore, thep argument of the function shows the core number. In theconsumer function body, we first define a variable,num, to hold the values that come from the queue. Next comes the task loop:

while (true)        {xQueueReceive(m_number_queue, &num, portMAX_DELAY);ESP_LOGI(TAG,"c%d:%d", (int)p, num);vTaskDelay(2);        }    }// end of consumer}// end of namespace

The task loop is an infinite loop, so the function will never return as it should be. ThexQueueReceive function takes the same parameters as with thexQueueSendToBack function that we used in theproducer function. However, thexQueueReceive function pops the value at the front of the queue. When all values in the queue are consumed, it will block the task until a new value arrives. If no value comes, then thexQueueReceive function will block forever since we passedportMAX_DELAY as its third argument. The application is ready to run on the devkit, let’s do it next.

Running the application

We canupload and monitor it by clicking on theUpload and Monitor project task of the PlatformIO IDE. Let’s discuss the output briefly:

<Previous logs are removed ...>I (280) cpu_start: Starting scheduler on PRO CPU.I (0) cpu_start: Starting scheduler on APP CPU.I (301) app: application started

After the start of the FreeRTOS schedulers on both CPUs, our application prints its first log asapplication started. Then we see thevTaskList output as follows:

I (301) app:consumer-1      R   5       3580        9       1main               X   1       1936        4       0IDLE                R   0       892        6       1IDLE                R   0       1012        5       0producer           B   5       3500        7       -1esp_timer           S   22     3432        3       0ipc1                B   24      884         2       1consumer-0      B 5       3412        8       0ipc0                B   24      892         1       0

The columns in this table are:

• Task name
• Task state
• Priority
• Used stack in bytes
• The order in which the tasks are created
• Core ID

We cansee our tasks in the list in addition to other default tasks. They are (in the order of creation) as follows:

• TheInter-Processor Call(IPC) tasks (ipc0 and ipc1) for triggering execution on theother CPU
• esp_timer for RTOS tick period
• Themain task that calls theapp_main function (entry point) of the application
• TheIDLE tasks of FreeRTOS

After the default FreeRTOS tasks, our tasks start. When you look at the last column of the table,consumer-0 has started oncpu0,consumer-1 has started on cpu1, and forproducer, the core ID value is displayed as-1, which means it can run on both CPUs.

The logs from the tasks come next on the serial output:

I (801) app: p:1I (801) app: p:2I (801) app: c1:1I (801) app: p:3I (801) app: c0:2I (801) app: p:4I (801) app: p:5I (801) app: p:6I (801) app: p:7I (821) app: c1:3I (821) app: p:8I (831) app: c0:4I (831) app: p:9I (841) app: c1:5I (841) app: p:10I (851) app: c0:6I (861) app: c1:7I (871) app: c0:8I (881) app: c1:9I (891) app: c0:10

Becauseof the delays in the consumer tasks, the producer fills up the queue faster than the consumers remove numbers and the producer has to wait for the consumers to make some space so it can insert a new number. Whenconsumer-1 removes3 from the queue, then the producer can enqueue8. It stops pushing new numbers when it gets to10 as we coded. The rest of the job is only for the consumers to dequeue all numbers remaining in the queue.

This example demonstrated how to utilize FreeRTOS for a simple producer-consumer problem and the basic usage of the ESP32 cores with different tasks. We will continue to employ FreeRTOS in the examples of the upcoming chapters and learn about more of its features. The official ESP-IDF FreeRTOS API documentation is here:https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/freertos.html.

In the next topic, we will discuss how we can debug our applications.

All families of ESP32 MCUssupportJoint Test Action Group (JTAG) debugging. ESP-IDF makes use ofOpenOCD, an open-source software, to interface with theJTAG probe/adapter, such as anESP-Prog debugger. To debug ourapplications, we use anEspressif version ofgdb (the GNU debugger), depending on the architecture of ESP32 that we have in a particular project. The next figure shows a general ESP32 debug setup:

Figure 2.24: JTAG debugging

When we develop our own custom ESP32 devices, we can connect to the standard JTAG interface of ESP32 to debug the application. With this option, we need to use a JTAG probe between the development machine and the custom ESP32 device. The JTAG pins are listed on the official documentation for each family of ESP32 (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/jtag-debugging/configure-other-jtag.html).

The issue with JTAG debugging is that it requires at least 4 GPIO pins to carry the JTAG signals, which means 4 GPIO pins less available to your application. This might be a problem in some projects where you need more GPIO pins. To address this issue, Espressif introduces direct USB debugging (built-in JTAG) without a JTAG probe. In the preceding figure, the JTAG probe in the middle is not needed for debugging and OpenOCD running on the development machine talks directly to the MCU over USB. The built-in JTAG debugging requires only two pins on ESP32, which saves two pins compared to the ordinary JTAG debugging with a probe. This feature is not available in all ESP32 families but ESP32-C3 and ESP32-S3 do have it; thus, we will prefer this method in this example with our ESP32-S3-BOX-Lite devkit. We don’t need a JTAG probe but we still need a USB cable with the pins exposed outside to be able to connect them to the corresponding pins of the devkit. The connections are:

ESP32-S3 Pin    USB SignalGPIO19            D-GPIO20            D+5V                V_BUSGND            Ground

We canfind a USB cable on many online shopswith all lines exposed but it is perfectly fine to cut a USB cable and solder a 4-pin header to use its pins. You can see my simple setup below:

Figure 2.25: Built-in JTAG

We don’t need a driver for Linux or macOS to use the built-in JTAG debugging. The Windows driver comeswith theESP-IDF Tools Installer (https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/windows-setup.html#esp-idf-tools-installer).

Now, it is time to create the project and upload the firmware to see whether our setup works. In this example, we will see another way of creating an ESP-IDF project. We will work in the ESP-IDF environment from the command line and use theidf.py script to create theproject and for other project tasks. Let’s do this in steps:

1. If you are aWindows user, run theESP-IDF Command Prompt shortcut from the Windows Start menu. It will open a command-line terminal with the ESP-IDF environment. If your development platform is Linux or macOS, start a terminal and run theexport.sh orexport.fish scripts respectively to have the ESP-IDF environment in the terminal:

   $source ~/esp/esp-idf/export.sh Detecting the Python interpreterChecking "python" ...Python 3.10.11"python" has been detectedAdding ESP-IDF tools to PATH…<more logs>Done! You can now compile ESP-IDF projects.Go to the project directory and run:idf.py build

2. Test theidf.pyscript by running it without any arguments. It will print the help message:

   $idf.pyUsage: idf.py [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...ESP-IDF CLI build management tool. For commands that are not known to idf.py an attempt to execute it as a build system target will be made.<rest of the help message>

3. Go to your project directory and runidf.py with a new project name. The script will create an ESP-IDF project with that name in a directory with the same name:

   $idf.py create-project debugging_exExecuting action: create-projectThe project was created in <your project directory>/debugging_ex$lsdebugging_ex$cd debugging_ex/$tree.├── CMakeLists.txt└── main    ├──CMakeLists.txt    └── debugging_ex.c1 directory, 3 files

4. Download thesdkconfig file from the book repository into the project directory. It can be found here:https://github.com/PacktPublishing/Developing-IoT-Projects-with-ESP32-2nd-edition/blob/main/ch2/debugging_ex/sdkconfig.
5. Run VSCode and open thedebugging_ex directory.
6. In the VSCode IDE, renamemain/main.c tomain/main.cpp and edit it to have the followingcode inside:

   #include"freertos/FreeRTOS.h"#include"freertos/task.h"voidmy_func(void){int j =0;    ++j;}extern"C"voidapp_main(){int i =0;while (1)    {vTaskDelay(pdMS_TO_TICKS(1000));        ++i;my_func();    }}

7. Update themain/CMakeLists.txt file with the following content:

   idf_component_register(SRCS "debugging_ex.cpp" INCLUDE_DIRS ".")

8. After these changes, we should have the following directory structure:

   $tree.├── CMakeLists.txt├── main│   ├── CMakeLists.txt│   └── debugging_ex.cpp└── sdkconfig1 directory, 4 files

9. Weneed to enable the debug options inthe rootCMakeLists.txt file. Edit it and set its content as the following:

   cmake_minimum_required(VERSION 3.16.0)include($ENV{IDF_PATH}/tools/cmake/project.cmake)project(debugging_ex)idf_build_set_property(COMPILE_OPTIONS "-O0" "-ggdb3" "-g3" APPEND)

10. In the terminal, build the application by runningidf.py:

    $idf.py buildExecuting action: all (aliases: build)<more logs>Creating esp32s3 image...Merged 2 ELF sectionsSuccessfully created esp32s3 image.<more logs>

11. See that theidf.py script has generated the application binary under the build directory:

    $ls build/*.binbuild/debugging_ex.bin

12. Run the following command to see the basic size information of the application:

    $idf.py size<some logs>Total sizes:Used static IRAM:   87430 bytes ( 274810 remain, 24.1% used)      .text size:   86403 bytes   .vectors size:    1027 bytesUsed stat D/IRAM:   13941 bytes ( 158987 remain, 8.1% used)      .data size:   11389 bytes      .bss  size:    2552 bytesUsed Flash size :  153979 bytes      .text     :  113547 bytes      .rodata   :   40176 bytesTotal image size:  252798 bytes (.bin may be padded larger)

13. Flash theapplication on the devkit (if flashing fails with a port error, just reverse the D+/D- connections of the devkit. Thissimple change will probably solve the problem):

    $idf.py flashExecuting action: flashSerial port /dev/ttyACM0Connecting...Detecting chip type... ESP32-S3<more logs>Leaving...Hard resetting via RTS pin...Done

14. We now have an application running on the devkit, ready for debugging. Run a GDB server with the following command (idf.py will start anOpenOCD process for this):

    $idf.py openocd --openocd-commands"-f board/esp32s3-builtin.cfg"Executing action: openocdOpenOCD started as a background task 477341Executing action: post_debugOpen On-Chip Debugger v0.11.0-esp32-20221026 (2022-10-26-14:47)Licensed under GNU GPL v2<more logs>

15. Start another ESP-IDF command-line terminal as we did instep 1 and change the current directory to the project root directory.
16. Run thefollowing command to start a GDB client. It will open a web-based GUI in your default browser:

    $idf.py gdbguiExecuting action: gdbguigdbgui started as a background task 476131

Theidf.py script isthe single point of contact to manage an ESP-IDF project. We can create, build, flash, and debug an application by only using this script. It has more features and we will have many chances to learn and practice those features throughout the book. The ESP-IDF build system documentation provides detailed information about theidf.py script and how it works withcmake to collect all the project components to compile them into an application. Here is the link for the documentation:https://docs.espressif.com/projects/esp-idf/en/v4.4.4/esp32s3/api-guides/build-system.html.

With a debugging session ready on the web GUI, we can now debug the application. The following screenshot shows this GUI.

Figure 2.26: Web-based debugger

The left panel shows the source code that is being debugged. We can set/remove breakpoints by clicking on the row numbers. The rightpanel shows the current status of the application, including threads, variables, memory, etc. On the top right, the buttons for the debug functions (restart, pause, continue, and step in/out/over) are placed. The debug functions also have keyboard shortcuts to ease the debugging process. Try the following debugging tasks on the GUI:

• Click on line number17 to set a breakpoint.
• PressC (continue) on the keyboard and observe that the local variablei increases every time the execution hits the breakpoint.
• Try pressingN (next) to run each of the lines consecutively.
• When the execution comes tomy_func, pressS(step in) to enter the function. You can exit the function by pressingU (up).

This GUI is enough for an average debugging session and can be used to observe the behavior of the application when necessary. If you need to access othergdb commands, there is also another panel at the bottom where you can type these commands.

Whether youpreferTest-Driven Development(TDD) or justwrite unit tests as a safety net against regression, it is always wise to include them in the plans of any type of software project. Although the adopted testing strategy for a project depends on the project type and company policies, well-designed unit tests are the basic safeguard of any serious product. The time and effort you put into unit tests always pay off in every stage of the product life cycle, from the beginning of the development to the maintenance and upgrades.

For ESP32 projects, we have several unit-testing framework options. ESP-IDF supports theUnity frameworkbut we canalso useGoogleTest in our projects. We can configure PlatformIO to use any of them and run tests on a target device, such as an ESP32 devkit, and/or on the local development machine. Therefore, it is really easy to select different strategies for unit testing. For example, if the library that you are working on doesn’t need to use hardware peripherals, then it can be tested on the local machine and you can instruct PlatformIO to do this by simply adding some definitions to theplatformio.ini file of the project.

We will create a sample project to see how unit testing is done in an ESP32 project next.

Creating a project

Let’s assumethat we want to develop a simple light control class that sets a GPIO pin of ESP32-C3-DevKitM-1 to high/low in order to turn on and off the light that is connected to the GPIO pin. In this example, we will develop that class and write tests for it with the GoogleTest framework. We will also configure PlatformIO to run the tests on the devkit so that we know that the class works as intended on the real hardware. Let’s create the project as in the following steps:

1. Start a new PlatformIO project with the following parameters:
   • Name:unit_testing_ex
   • Board:Espressif ESP32-C3-DevKitM-1
   • Framework:Espressif IoT Development Framework
2. Open theplatformio.ini file and set its content as follows:

   [env:esp32-c3-devkitm-1]platform = espressif32@6.2.0board = esp32-c3-devkitm-1framework = espidfbuild_flags = -std=gnu++11 -Wno-unused-resultmonitor_speed =115200monitor_rts =0monitor_dtr =0monitor_filters = colorizelib_deps = google/googletest@1.12.1test_framework = googletest

3. Build the project (PLATFORMIO | PROJECT TASKS | esp32-c3-devkitm-1 | General | Build).

The project is now configured and ready for development. However, before moving on, I want to briefly discuss the library management mechanism of PlatformIO. We have several options to add external libraries to our projects. The easiest one is probably just by referring to the PlatformIO registry. You can search the registry by navigating toPlatformIO Home/Libraries and typing the name of the library that you are looking for. When the library is listed, you select it and PlatformIO shows its detailed information. At this point, you can click on theAdd to Project button to include the library in the project.

Figure 2.27: PlatformIO Registry

In our example, Ipreferred to specifygoogletest directly as in the following configuration line without using the graphical interface:

lib_deps = google/googletest@1.12.1

This line points to the PlatformIO registry. The format is<provider>/<library>@<version>. In this way, it is possible to add many other libraries consecutively in a project. With thelib_deps configuration parameter, we can also refer to other online repositories by providing their URLs.

The other popular option is to add local directories withlib_extra_dirs inplatformio.ini. Any ESP-IDF-compatible library in these directories can be included in projects. I will talk about whatcompatible means in this context later in the book.

You may have noticed that we can also set the platform version:

platform = espressif32@6.2.0

With this configuration, we set the platform version to a fixed value so that no matter when wecompile the project, we know that it will compile without any compatibility issues with all other versioned libraries and of course with our code.

After this brief overview of library management, we can continue with the application.

Coding the application

Let’s beginwith adding a header file, namedsrc/AppLight.hpp, for the light control class and add the required header for GPIO control:

#pragma once#include"driver/gpio.h"#defineGPIO_SEL_4 (1<<4)

Then we define the class as follows:

namespace app{classAppLight    {private:boolm_initialized;

In theprivate section of the class, we define a member variable,m_initialized, which shows if the class is initialized. Thepublic section comes next:

public:AppLight() :m_initialized(false) {}voidinitialize(){if (!m_initialized)            {gpio_config_tconfig_pin4{GPIO_SEL_4,GPIO_MODE_INPUT_OUTPUT,                  GPIO_PULLUP_DISABLE,                  GPIO_PULLDOWN_DISABLE,                  GPIO_INTR_DISABLE                };gpio_config(&config_pin4);                m_initialized =true;            }off();        }

After theconstructor, we implement theinitialize function. Its job is to configure the GPIO-4 pin of the devkit if it is not initialized yet and set its initial state to off. We use thegpio_config function to configure a GPIO pin as defined in the configuration structure that is provided as input. Here, it isconfig_pin4. Thegpio_config function and thegpio_config_t structure are declared in thedriver/gpio.h header file.

Theoff function is another member function of the class to be implemented next:

voidoff(){gpio_set_level(GPIO_NUM_4,0);        }voidon(){gpio_set_level(GPIO_NUM_4,1);        }    };}// namespace app

In theoff member function, we callgpio_set_level with the parameters ofGPIO_NUM_4 as the pin number and0 as the pin level. Again, thegpio_set_level function is declared in thedriver/gpio.h header file. Similarly, we add another function,on, in order to set the pin level to1, or high.

TheAppLight class is ready and we can write the test code for it next.

Adding unit tests

Wecreateanother source file,test/test_main.cpp, and add the header files that are needed for the unit tests:

#include"gtest/gtest.h"#include"AppLight.hpp"#include"driver/gpio.h"

For theAppLight testing, itwould be a good idea to create a test fixture:

namespace app{classLightTest :public::testing::Test    {protected:static AppLight light;LightTest()        {light.initialize();        }    };    AppLight LightTest::light;

Thename of the test fixture isLightTest and it is derived from the::testing::Test base class. In its protected area, we declare a staticAppLight object and initialize it in the constructor of the fixture. With the fixture ready, we can now write a test as follows:

TEST_F(LightTest, turns_on_light)    {        light.on();ASSERT_GT(gpio_get_level(GPIO_NUM_4),0);    }

TheTEST_F macro defines a test on a test fixture. The first parameter shows the fixture name and the second parameter is the test name. In the test, we turn the light on, and assert whether it is really turned on. TheASSERT_GT macro checks whether the first parameter is greater than the second one.

Another test checks whether theoff function is working properly or not. It is very similar to the previous test:

TEST_F(LightTest, turns_off_light)    {        light.off();ASSERT_EQ(gpio_get_level(GPIO_NUM_4),0);    }}// namespace app

This time, we turn the light off, and check whether it is actually turned off by using theASSERT_EQ macro.

We stillneed anapp_main function as usual. Here it comes:

extern"C"voidapp_main(){::testing::InitGoogleTest();RUN_ALL_TESTS();}

The two lines in theapp_main function initialize and run all of the test cases. This finalizes the test coding. Let’s run it on the devkit and see the test results.

Running unit tests

We can runthe test application on the devkit and see theunit test results as in the following steps:

1. Plug the devkit into one of the USB ports of your development machine.
2. Navigate toPLATFORMIO | PROJECT TASKS | esp32-c3-devkitm-1 | Advanced and click on theTest option there.

   

   Figure 2.28: PlatformIO unit testing

3. PlatformIOwill compile the test application, upload it, and then run the tests. You can see the result on the terminalthat popped up when you clicked on theTest option.

   

   Figure 2.29: Terminal output

Theterminallists the tests and the results. When a test fails, you can go back to the code, debug it, and run the tests again until they all pass.

With this topic, we conclude the chapter. However, I strongly suggest you don’t limit yourself to the explanations here and try other tools from both PlatformIO and ESP-IDF. I will continue to talk about them throughout the book and use them within the examples to help you get familiar with the tools and their features as much as possible.

In this chapter, we have learned about the tools and the basics of ESP32 development.ESP-IDF is the official framework to develop applications on any family of ESP32 series microcontrollers, maintained by Espressif Systems. It comes with the entire set of command-line utilities that you would need in your ESP32 projects.PlatformIO adds more IDE features on top of that. With its strong integration with the VSCode IDE and declarative project configuration approach, it provides a professional environment for embedded developers.

In the next chapter, we’ll discuss the ESP32 peripherals. Although it is impossible to cover all of them in a single chapter, we will learn about the common peripherals using examples so that we can easily carry out the tasks in real projects and the other experiments in the book.

Let’s answer the following questions to test our knowledge of the topics covered in this chapter:

1. What is the name of the script file that the ESP-IDF build system uses to configure an ESP32 project?
   1. CMakeLists.txt
   2. platformio.ini
   3. main.cpp
   4. Makefile
2. What is the name of the project-specific configuration file that is usually edited by runningmenuconfig?
   1. CMakeLists.txt
   2. sdkconfig
   3. pio
   4. platformio.ini
3. Which of the following is the most fundamental tool that comes with ESP-IDF to manage an ESP32 project?
   1. pio
   2. openocd
   3. gdb
   4. idf.py
4. Which of the following methods is the easiest to debug an ESP32-S3 board?
   1. JTAG
   2. SWD
   3. Built-in JTAG/USB
   4. UART
5. Which file defines a PlatformIO project?
   1. CMakeLists.txt
   2. sdkconfig
   3. pio
   4. platformio.ini

• Hands-On RTOS with Microcontrollers, Brian Amos, Packt Publishing (https://www.packtpub.com/product/hands-on-rtos-with-microcontrollers/9781838826734): Although the book uses STM32 in the examples, it is a great resource to learn about FreeRTOS.Chapter 20 discusses FreeRTOS on multi-core systems such as ESP32.
• Embedded Systems Architecture – Second Edition, Daniele Lacamera, Packt Publishing (https://www.packtpub.com/product/embedded-systems-architecture-second-edition/9781803239545): A good reading to learn the embedded systems in general.Part 4 talks about multithreading.

Learn more on Discord

To join the Discord community for this book – where you can share feedback, ask questions to the author, and learn about new releases – follow the QR code below:

https://discord.gg/3Q9egBjWVZ