Repository files navigation

hot-lib-reloader is a development tool that allows you to reload functions of a running Rust program.This allows to do "live programming" where you modify code and immediately see the effects in your running program.

This is build around thelibloading crate and will require you to put code you want to hot-reload inside a Rust library (dylib). For a detailed discussion about the idea and implementation seethis blog post.

For a demo and explanation see alsothis Rust and Tell presentation.

• Usage

  • Example project setup
    • Executable
    • Library
    • Running it
  • lib-reload events

• Usage tips

  • Know the limitations
    • No signature changes
    • Type changes require some care
    • Hot-reloadable functions cannot be generic
    • Global state in reloadable code
  • Use feature flags to switch between hot-reload and static code
  • Disable#[no-mangle] in release mode
  • Use serialization or generic values for changing types
  • Use a hot-reload friendly app structure
  • Use multiple libraries
  • Adjust the file watch debounce duration
  • Debugging

• Examples

• Known issues

  • tracing crate

To quicky generate a new project supporting hot-reload you can use acargo generate template:cargo generate rksm/rust-hot-reload.

On macOS the reloadable library needs to get codesigned.For this purpose, hot-lib-reloader will try to use thecodesign binary that is part of the XCode command line tools.It is recommended to make surethose are installed.

It should work out of the box.

Assuming you use a workspace project with the following layout:

├── Cargo.toml└── src│   └── main.rs└── lib    ├── Cargo.toml    └── src        └── lib.rs

Setup the workspace with a root project namedbin in./Cargo.toml:

[workspace]resolver ="2"members = ["lib"][package]name ="bin"version ="0.1.0"edition ="2021"[dependencies]hot-lib-reloader ="^0.6"lib = {path ="lib" }

In./src/main.rs define a sub-module using the[hot_lib_reloader_macro::hot_module] attribute macro which wraps the functionsexported by the library:

// The value of `dylib = "..."` should be the library containing the hot-reloadable functions// It should normally be the crate name of your sub-crate.#[hot_lib_reloader::hot_module(dylib ="lib")]mod hot_lib{// Reads public no_mangle functions from lib.rs and  generates hot-reloadable// wrapper functions with the same signature inside this module.// Note that this path relative to the project root (or absolute)hot_functions_from_file!("lib/src/lib.rs");// Because we generate functions with the exact same signatures,// we need to import types usedpubuse lib::State;}fnmain(){letmut state = hot_lib::State{counter:0};// Running in a loop so you can modify the code and see the effectsloop{        hot_lib::step(&mut state);        std::thread::sleep(std::time::Duration::from_secs(1));}}

The library should expose functions. It should set the crate typedylib in./lib/Cargo.toml:

[package]name ="lib"version ="0.1.0"edition ="2021"[lib]crate-type = ["rlib","dylib"]

The functions you want to be reloadable should be public and have the#[no_mangle] attribute. Note that you can define other function that are not supposed to change withoutno_mangle and you will be able to use those alongside the other functions.

pubstructState{pubcounter:usize,}#[no_mangle]pubfnstep(state:&mutState){    state.counter +=1;println!("doing stuff in iteration {}", state.counter);}

1. Start compilation of the library:cargo watch -w lib -x 'build -p lib'
2. In another terminal run the executable:cargo run

Now change for example the print statement inlib/lib.rs and see the effect on the runtime.

In addition, using a tool likecargo runcc is recommended. This allows to run both the lib build and the application in one go.

You can get notified about two kinds of events using the methods provided by [LibReloadObserver]:

• wait_for_about_to_reload the watched library is about to be reloaded (but the old version is still loaded)
• wait_for_reload a new version of the watched library was just reloaded

This is useful to run code before and / or after library updates. One use case is to serialize and then deserialize state another one is driving the application.

To continue with the example above, let's say instead of running the library functionstep every second we only want to re-run it when the library has changed.In order to do that, we first need to get hold of theLibReloadObserver. For that we can expose a functionsubscribe() that is annotated with the#[lib_change_subscription] (that attribute tells thehot_module macro to provide an implementation for it):

#[hot_lib_reloader::hot_module(dylib ="lib")]mod hot_lib{/* code from above */// expose a type to subscribe to lib load events#[lib_change_subscription]pubfnsubscribe() -> hot_lib_reloader::LibReloadObserver{}}

And then the main function just waits for reloaded events:

fnmain(){letmut state = hot_lib::State{counter:0};let lib_observer = hot_lib::subscribe();loop{        hot_lib::step(&mut state);// blocks until lib was reloaded        lib_observer.wait_for_reload();}}

How to block reload to do serialization / deserialization is shown in thereload-events example.

To just figure out if the library has changed, a simple test function can be exposed:

#[hot_lib_reloader::hot_module(dylib ="lib")]mod hot_lib{/* ... */#[lib_updated]pubfnwas_updated() ->bool{}}

hot_lib::was_updated() will returntrue the first time it is called after the library was reloaded.It will then return false until another reload occurred.

Reloading code from dynamic libraries comes with a number of caveats which are discussed in some detailhere.

When the signature of a hot-reloadable function changes, the parameter and result types the executable expects differ from what the library provides. In that case you'll likely see a crash.

Types of structs and enums that are used in both the executable and library cannot be freely changed. If the layout of types differs you run into undefined behavior which will likely result in a crash.

Seeuse serialization for a way around it.

Since#[no_mangle] does not support generics, generic functions can't be named / found in the library.

If your hot-reload library contains global state (or depends on a library that does), you will need to re-initialize it after reload. This can be a problem with libraries that hide the global state from the user. If you need to use global state, keep it inside the executable and pass it into the reloadable functions if possible.

Note also that "global state" is more than just global variables. As noted inthis issue, crates relying on theTypeId of a type (like most ECS systems do) will expect the type/id mapping to be constant. After reloading, types will have different ids, however, which makes (de)serialization more challenging.

See thereload-feature example for a complete project.

Cargo allows to specify optional dependencies and conditional compilation through feature flags.When you define a feature like this

[features]default = []reload = ["dep:hot-lib-reloader"][dependencies]hot-lib-reloader = {version ="^0.6",optional =true }

and then conditionally use either the normal or the hot module in the code calling the reloadable functions you can seamlessly switch between a static and hot-reloadable version of your application:

#[cfg(feature ="reload")]use hot_lib::*;#[cfg(not(feature ="reload"))]use lib::*;#[cfg(feature ="reload")]#[hot_lib_reloader::hot_module(dylib ="lib")]mod hot_lib{/*...*/}

To run the static version just usecargo run the hot reloadable variant withcargo run --features reload.

To not pay a penalty for exposing functions using#[no_mangle] in release mode where everything is statically compiled (see previous tip) and no functions need to be exported, you can use theno-mangle-if-debug attribute macro. It will conditionally disable name mangling, depending on wether you build release or debug mode.

If you want to iterate on state while developing you have the option to serialize it. If you use a generic value representation such asserde_json::Value, you don't need string or binary formats and typically don't even need to clone anything.

Here is an example where we crate a state container that has an innerserde_json::Value:

#[hot_lib_reloader::hot_module(dylib ="lib")]mod hot_lib{pubuse lib::State;hot_functions_from_file!("lib/src/lib.rs");}fnmain(){letmut state = hot_lib::State{inner: serde_json::json!(null),};loop{        state = hot_lib::step(state);        std::thread::sleep(std::time::Duration::from_secs(1));}}

In the library we are now able to change the value and type layout ofInnerState as we wish:

#[derive(Debug)]pubstructState{pubinner: serde_json::Value,}#[derive(serde::Deserialize, serde::Serialize)]structInnerState{}#[no_mangle]pubfnstep(state:State) ->State{let inner:InnerState = serde_json::from_value(state.inner).unwrap_or(InnerState{});// You can modify the InnerState layout freely and state.inner value here freely!State{inner: serde_json::to_value(inner).unwrap(),}}

Alternatively you can also do the serialization just before the lib is to be reloaded and deserialize immediately thereafter. This is shown in thereload-events example.

Whether or not hot-reload is easy to use depends on how you architect your app. In particular, the"functional core, imparative shell" pattern makes it easy to split state and behavior and works well withhot-lib-reloader

For example, for a simple game where you have the main loop in your control, setting up the outer state in the main function and then passing it into afn update(state: &mut State) and afn render(state: &State) is a straightforward way to get two hot-reloadable functions.

But even when using a framework that takes control, chances are that there are ways to have it call hot-reloadable code. Thebevy example where system functions can be made hot-reloadable, shows how this can work.See theegui andtokio examples possible setupts.

Thehot_module macro allows setting thefile_watch_debounce attribute which defines the debounce duration for file changes in milliseconds.This is 500ms by default.If you see multiple updates triggered for one recompile (can happen the library is very large), increase that value.You can try to decrease it for faster reloads. With small libraries / fast hardware 50ms or 20ms should work fine.

#[hot_module(dylib ="lib", file_watch_debounce =50)]/* ... */

By defaulthot-lib-reloader assumes that there will be a dynamic library available in the$CARGO_MANIFEST_DIR/target/debug/ or$CARGO_MANIFEST_DIR/target/release folder, depending on whether the debug or release profile is used.The name of the library is defined by thedylib = "..." portion of the#[hot_module(...)] macro.So by specifying#[hot_module(dylib = "lib")] and building with debug settings,hot-lib-reloader will try to load atarget/debug/liblib.dylib on MacOS, atarget/debug/liblib.so on Linux or atarget/debug/lib.dll on Windows.

If the library should be loaded from a different location you can specify this by setting thelib_dir attribute like:

#[hot_lib_reloader::hot_module(    dylib ="lib",    lib_dir = concat!(env!("CARGO_MANIFEST_DIR"),"/target/debug"))]mod hot_lib{/* ... */}

Thehot_module macro allows setting the shadow file name using theloaded_lib_name_template parameter.This is useful when multiple processes are trying to hot reload the same library and can be used to prevent conflicts.This attribute allows for placeholders that can be dynamically replaced:

If you don't specify theloaded_lib_name_template parameter, a default naming convention is used for the shadow filename.This default pattern is:{lib_name}-hot-{load_counter}.

#[hot_lib_reloader::hot_module(    dylib ="lib",// Might result in the following shadow file lib_hot_2644_0_5e659d6e-b78c-4682-9cdd-b8a0cd3e8fc6.dll// Requires the 'uuid' feature flags for the {uuid} placeholder    loaded_lib_name_template ="{lib_name}_hot_{pid}_{load_counter}_{uuid}")]mod hot_lib{/* ... */}

If yourhot_module gives you a strange compilation error, trycargo expand to see what code is generated.

By default thehot-lib-reloader crate won't write to stdout or stderr but it logs what it does with info, debug, and trace log levels using thelog crate.Depending on what logging framework you use (e.g.env_logger), you can enable those logs by setting aRUST_LOG filter likeRUST_LOG=hot_lib_reloader=trace.

Examples can be found atrksm/hot-lib-reloader-rs/examples.

• minimal: Bare-bones setup.
• reload-feature: Use a feature to switch between dynamic and static version.
• serialized-state: Shows an option to allow to modify types and state freely.
• reload-events: How to block reload to do serialization / deserialization.
• all-options: All options thehot_module macro accepts.
• bevy: Shows how to hot-reload bevy systems.
• nannou: Interactive generative art withnannou.
• egui: How to hot-reload a native egui / eframe app.
• iced: How to hot-reload an iced app.

When used with thetracing crate multiple issues can occur:

• Whentracing is used in the library that is reloaded the app sometimes crashes withAttempted to register a DefaultCallsite that already exists!
• When used in combination with bevy,commands.insert(component) operations stop to work after a reload, likely because of internal state getting messed up.

If you can, don't usehot-lib-reloader in combination withtracing.

MIT

License: MIT