Thanks for the PR. Unfortunately, this include breaking changes and will require a new major release, however we are currently discussing a v4 release anyway, so probably this is not too much of an issue. If I can just ask you to be patient, once we come up with a game plan I can get back to you with regards to merging this. Hopefully within a week or two we will have a better idea of what our plans are moving forward.

Hi ! Thanks for the quick response.
Yes, I understand. I'll wait for your input as to how to get it merged to v4.

We did some planning this week for the v4 release. I do not think we will start working on this quite yet, probably not for a few weeks still, but in the meantime I will publish one final v3 release on Monday, and then we can at least unblock this PR (and others).

@4rzael Please excuse my ignorance... I do not understand howespflash can write encrypted payload given that there is no encryption key around?

@4rzael Also I'm a bit lost as to what exactly is requested in#70 ?

My understanding as to what@jessebraham demonstrates in#70, is just one way to do encryption, which is - "first flash plaintext images, and then the bootloader will encrypt them on-the-fly" (that is, given that the bootloader is compiled with encryption support in the first place, which I doubt to be the case with the bootloaders provided byespflash itself and embedded inside it withinclude_bytes! or whatever). By the way, this is done by the bootloader by it doing a lot of (dangerous!) e-fuse operations, like generating a random encryption key, burning it, then read+write protecting it and a few others.

This is by the way (and to my understanding - I'm still debugging the bootloader so I might very well be wrong!) aone time operation? If you can confirm my understanding is correct?

Like, once it is done, you can't just burn plaintext images and hope the bootloader would encrypt them? Or am I missing something here? My understanding is, once the initial bootloader-driven encryption is done, that's it end of story. The bootloader does not repeat the process anymore, and will assume all new images you upload to be pre-encrypted. and since you don't know the encryption key, you can't pre-encrypt them in the first place? That is, given that the UART downloader was left enabled (which depends on whether you are in DEVELOPMENT or RELEASE mode w.r.t. encryption).

I'm myself following the "other" process, where the encryption key is "externally" provided, and the payload (bootloader, part-table, app image, nvs keys etc.) are pre-encrypted. Seems to be what Espressif recommends as well?:
https://docs.espressif.com/projects/esp-idf/en/v5.4/esp32s3/security/security-features-enablement-workflows.html

If you have an opinion on this stuff - please share. I'm about to enter production with my approach...

@ivmarkov
Hey. Sorry about the slow answer there.

My understanding as to what@jessebraham demonstrates in#70, is just one way to do encryption, which is - "first flash plaintext images, and then the bootloader will encrypt them on-the-fly"

Yes. IIRC, this used to be the recommended/documented workflow if you didn't want to keep the keys.
The process you linked is equivalent (as long as you destroy the key afterwards), and a is bit faster in the production line (the other approach requires the esp encrypting the flash at first boot, which can easily take like 30s).

The goal of this PR, however, is to replicate esptool's encrypt feature, which is indeed only useful in development mode, where the bootloader's ability to perform encrypted-flashes is not disabled.
You're completely right about it being impossible to re-flash once all security options are enabled in release mode.

Basically, after encryption has been enabled (by whichever of the 2 methods you prefer), you can make the bootloader reflash a plaintext image, and encrypt it on the fly, allowing developers to flash firmwares on devices with encryption enabled (and thus as similar as possible to the end product).

@ivmarkov Hey. Sorry about the slow answer there.

My understanding as to what@jessebraham demonstrates in#70, is just one way to do encryption, which is - "first flash plaintext images, and then the bootloader will encrypt them on-the-fly"

Yes. IIRC, this used to be the recommended/documented workflow if you didn't want to keep the keys. The process you linked is equivalent (as long as you destroy the key afterwards), and a is bit faster in the production line (the other approach requires the esp encrypting the flash at first boot, which can easily take like 30s).

The goal of this PR, however, is to replicate esptool's encrypt feature, which is indeed only useful in development mode, where the bootloader's ability to perform encrypted-flashes is not disabled. You're completely right about it being impossible to re-flash once all security options are enabled in release mode.

Basically, after encryption has been enabled (by whichever of the 2 methods you prefer), you can make the bootloader reflash a plaintext image, and encrypt it on the fly, allowing developers to flash firmwares on devices with encryption enabled (and thus as similar as possible to the end product).

@4rzael Thank you for your response. I completely agree with your statements. Ideally,espflash should not get in the way of supporting either of the two workflows (flashing pre-encrypted images or flashing plain-text images and then leaving it up to the bootloader to encrypt).

The goal of this PR, however, is to replicate esptool's encrypt feature, which is indeed only useful in development mode, where the bootloader's ability to perform encrypted-flashes is not disabled. You're completely right about it being impossible to re-flash once all security options are enabled in release mode.

This ^^^ is the one thing I'm struggling with. I can't see an "encrypt" command onesptool.py?

ivan@ivan-laptop:~/rp$ esptools tool -hTool esptool mounted as`/home/ivan/.cache/esptools/78c195b08bd57bf27167b323600d417a546d3393/esptool`Executing`esptool`usage: esptool [-h]               [--chip {auto,esp8266,esp32,esp32s2,esp32s3beta2,esp32s3,esp32c3,esp32c6beta,esp32h2beta1,esp32h2beta2,esp32c2,esp32c6,esp32c61,esp32c5,esp32c5beta3,esp32h2,esp32p4}]               [--port PORT] [--baud BAUD] [--port-filter PORT_FILTER]               [--before {default_reset,usb_reset,no_reset,no_reset_no_sync}]               [--after {hard_reset,soft_reset,no_reset,no_reset_stub}] [--no-stub] [--trace]               [--override-vddsdio [{1.8V,1.9V,OFF}]] [--connect-attempts CONNECT_ATTEMPTS]               {load_ram,dump_mem,read_mem,write_mem,write_flash,run,image_info,make_image,elf2image,read_mac,chip_id,flash_id,read_flash_status,write_flash_status,read_flash,verify_flash,erase_flash,erase_region,read_flash_sfdp,merge_bin,get_security_info,version}               ...esptool.py v4.8.1 - Espressif chips ROM Bootloader Utilitypositional arguments:  {load_ram,dump_mem,read_mem,write_mem,write_flash,run,image_info,make_image,elf2image,read_mac,chip_id,flash_id,read_flash_status,write_flash_status,read_flash,verify_flash,erase_flash,erase_region,read_flash_sfdp,merge_bin,get_security_info,version}                        Run esptool.py {command}-hfor additionalhelp    load_ram            Download an image to RAM and execute    dump_mem            Dump arbitrary memory to disk    read_mem            Read arbitrary memory location    write_mem           Read-modify-write to arbitrary memory location    write_flash         Write a binary blob to flash    run                 Run application codein flash    image_info          Dump headers from a binary file (bootloader or application)    make_image          Create an application image from binary files    elf2image           Create an application image from ELF file    read_mac            Read MAC address from OTP ROM    chip_id             Read Chip ID from OTP ROM    flash_id            Read SPI flash manufacturer and device ID    read_flash_status   Read SPI flash status register    write_flash_status  Write SPI flash status register    read_flash          Read SPI flash content    verify_flash        Verify a binary blob against flash    erase_flash         Perform Chip Erase on SPI flash    erase_region        Erase a region of the flash    read_flash_sfdp     Read SPI flash SFDP (Serial Flash Discoverable Parameters)    merge_bin           Merge multiple raw binary files into a single filefor later flashing    get_security_info   Get some security-related data    version             Print esptool versionoptional arguments:-h, --help            show thishelp message andexit  --chip {auto,esp8266,esp32,esp32s2,esp32s3beta2,esp32s3,esp32c3,esp32c6beta,esp32h2beta1,esp32h2beta2,esp32c2,esp32c6,esp32c61,esp32c5,esp32c5beta3,esp32h2,esp32p4},-c {auto,esp8266,esp32,esp32s2,esp32s3beta2,esp32s3,esp32c3,esp32c6beta,esp32h2beta1,esp32h2beta2,esp32c2,esp32c6,esp32c61,esp32c5,esp32c5beta3,esp32h2,esp32p4}                        Target chiptype  --port PORT,-p PORT  Serial port device  --baud BAUD,-b BAUD  Serial port baud rate used when flashing/reading  --port-filter PORT_FILTER                        Serial port device filter, can be vid=NUMBER, pid=NUMBER, name=SUBSTRING  --before {default_reset,usb_reset,no_reset,no_reset_no_sync}                        What todo before connecting to the chip  --after {hard_reset,soft_reset,no_reset,no_reset_stub},-a {hard_reset,soft_reset,no_reset,no_reset_stub}                        What todo after esptool.py is finished  --no-stub             Disable launching the flasher stub, only talk to ROM bootloader. Some features will not be available.  --trace,-t           Enable trace-level output of esptool.py interactions.  --override-vddsdio [{1.8V,1.9V,OFF}]                        Override ESP32 VDDSDIO internal voltage regulator (use with care)  --connect-attempts CONNECT_ATTEMPTS                        Number of attempts to connect, negative or 0for infinite. Default: 7.Status Ok(ExitStatus(unix_wait_status(0)))

See? And that's becauseesptool.py does not care whether its flashing an encrypted image or not.

Now, there is a command "encrypt_flash_data" onespsecure.py, if that's what you actually meant, but this command is for pre-encrypting the image on the host machine, which is not your use case anyway.

What am I missing?

@ivmarkov Yeah, I wasn't super clear. What I'm talking about is the--encrypt argument in esptool'swrite_flash command, which tells the bootloader to encrypt the supplied plaintext blob during the transfer.
While I haven't experimented with the workflow where you directly encrypt on the host, I agree with you that, AFAIK, it should be completely transparent to esptool.

Hi! Do you have any updates on your tests? Would you like me to help testing?