More documentation on hot reloading and multithreading

.github/workflows/ci.yml

@@ -31,7 +31,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v4
-    - uses: dtolnay/rust-toolchain@1.74
+    - uses: dtolnay/rust-toolchain@stable
       with:
         components: clippy
     - uses: Swatinem/rust-cache@v2

book/src/SUMMARY.md

@@ -27,6 +27,8 @@
 - [Closures](./closures.md)
 - [Asynchronous programming](./async.md)
   - [Streams](./streams.md)
+- [Multithreading](./multithreading.md)
+- [Hot reloading](./hot_reloading.md)
 - [Macros](./macros.md)
 - [Advanced](./advanced.md)
   - [Safety](./safety.md)

book/src/hot_reloading.md

@@ -0,0 +1,22 @@
+# Hot reloading
+
+Compiling a [`Unit`] and a [`RuntimeContext`] are expensive operations compared
+to the cost of calling a function. So you should try to do this as little as
+possible. It is appropriate to recompile a script when the source of the script
+changes. This section provides you with details for how this can be done when
+loading scripts from the filesystem.
+
+A typical way to accomplish this is to watch a scripts directory using the
+[`notify` crate]. This allow the application to generate events whenever changes
+to the directory are detected. See the [`hot_reloading` example] and in
+particular the [`PathReloader`] type.
+
+```
+{{#include ../../examples/examples/hot_reloading.rs}}
+```
+
+[`notify` crate]: https://docs.rs/notify
+[`Unit`]: https://docs.rs/rune/latest/rune/runtime/unit/struct.Unit.html
+[`hot_reloading` example]: https://github.com/rune-rs/rune/blob/main/examples/examples/hot_reloading.rs
+[`PathReloader`]: https://github.com/rune-rs/rune/blob/main/examples/examples/hot_reloading/path_reloader.rs
+

book/src/multithreading.md

@@ -0,0 +1,61 @@
+# Multithreading
+
+Rune is thread safe, but the [`Vm`] does not implement `Sync` so cannot directly
+be shared across threads. This section details instead how you are intended to
+use Rune in a multithreaded environment.
+
+Compiling a [`Unit`] and a [`RuntimeContext`] are expensive operations compared
+to the cost of calling a function. So you should try to do this as little as
+possible. It is appropriate to recompile a script when the source of the script
+changes. See the [Hot reloading] section for more information on this.
+
+Once you have a `Unit` and a `RuntimeContext` they are thread safe and can be
+used by multiple threads simultaneously through `Arc<Unit>` and
+`Arc<RuntimeContext>`. Constructing a `Vm` with these through `Vm::new` is a
+very cheap operation.
+
+> `Vm`'s do allocate a stack, to avoid this you'd have to employ even more
+> advanced techniques, such as storing the virtual machine in a thread local and
+> using it by swapping out the `Unit` and `RuntimeContext` associated with it
+> through [`Vm::unit_mut`] and [`Vm::context_mut`] respectively.
+
+```rust
+let unit: Arc<Unit> = /* todo */;
+let context: Arc<RuntimeContext> = /* todo */;
+
+std::thread::spawn(move || {
+    let mut vm = Vm::new(unit, context);
+    let value = vm.call(["function"], (42,))?;
+    Ok(())
+});
+```
+
+Using [`Vm::send_execute`] is a way to assert that a given execution is thread
+safe. And allows you to use Rune in asynchronous multithreaded environments,
+such as Tokio. This is achieved by ensuring that all captured arguments are
+[`ConstValue`]'s, which in contrast to [`Value`]'s are guaranteed to be
+thread-safe:
+
+```
+{{#include ../../examples/examples/tokio_spawn.rs}}
+```
+
+Finally [`Function::into_sync`] exists to coerce a function into a
+[`SyncFunction], which is a thread-safe variant of a regular [`Function`]. This
+is a fallible operation since all values which are captured in the function-type
+in case its a closure has to be coerced to [`ConstValue`]. If this is not the
+case, the conversion will fail.
+
+[`ConstValue`]: https://docs.rs/rune/latest/rune/runtime/enum.ConstValue.html
+[`Function::into_sync`]: https://docs.rs/rune/latest/rune/runtime/struct.Function.html#method.into_sync
+[`Function`]: https://docs.rs/rune/latest/rune/runtime/struct.Function.html
+[`notify`]: https://docs.rs/notify
+[`RuntimeContext`]: https://docs.rs/rune/latest/rune/runtime/struct.RuntimeContext.html
+[`SyncFunction`]: https://docs.rs/rune/latest/rune/runtime/struct.SyncFunction.html
+[`Unit`]: https://docs.rs/rune/latest/rune/runtime/struct.Unit.html
+[`Value`]: https://docs.rs/rune/latest/rune/runtime/enum.Value.html
+[`Vm::context_mut`]: https://docs.rs/rune/latest/rune/runtime/struct.Vm.html#method.context_mut
+[`Vm::send_execute`]: https://docs.rs/rune/latest/rune/runtime/struct.Vm.html#method.send_execute
+[`Vm::unit_mut`]: https://docs.rs/rune/latest/rune/runtime/struct.Vm.html#method.unit_mut
+[`Vm`]: https://docs.rs/rune/latest/rune/runtime/struct.Vm.html
+[Hot reloading]: ./hot_reloading.md

crates/rune-alloc/src/hashbrown/map.rs

@@ -7057,7 +7057,7 @@ mod test_map {
     use crate::iter::TryExtend;
     use crate::testing::*;
 
-    std::thread_local!(static DROP_VECTOR: RefCell<Vec<i32>> = RefCell::new(Vec::new()));
+    std::thread_local!(static DROP_VECTOR: RefCell<Vec<i32>> = const { RefCell::new(Vec::new()) });
 
     #[test]
     fn test_zero_capacities() {

crates/rune-alloc/src/limit/std.rs

@@ -1,6 +1,6 @@
 use core::cell::Cell;
 
-std::thread_local!(static MEMORY: Cell<usize> = Cell::new(usize::MAX));
+std::thread_local!(static MEMORY: Cell<usize> = const { Cell::new(usize::MAX) });
 
 pub(super) fn rune_memory_take(amount: usize) -> bool {
     MEMORY.with(|tls| {

crates/rune/src/any.rs

@@ -56,26 +56,62 @@ use crate::hash::Hash;
 /// ```
 pub use rune_macros::Any;
 
-/// A trait which can be stored inside of an [AnyObj](crate::runtime::AnyObj).
+/// Derive for types which can be used inside of Rune.
 ///
-/// We use our own marker trait that must be explicitly derived to prevent other
-/// VM native types (like strings) which also implement `std::any::Any` from
-/// being stored as an `AnyObj`.
+/// Rune only supports two types, *built-in* types [`String`] and *external*
+/// types which derive `Any`. Before they can be used they must be registered in
+/// [`Context::install`] through a [`Module`].
 ///
-/// This means, that only types which derive `Any` can be used inside of the VM:
+/// This is typically used in combination with declarative macros to register
+/// functions and macros, such as:
+///
+/// * [`#[rune::function]`]
+/// * [`#[rune::macro_]`]
+///
+/// [`AnyObj`]: crate::runtime::AnyObj
+/// [`Context::install`]: crate::Context::install
+/// [`Module`]: crate::Module
+/// [`String`]: std::string::String
+/// [`#[rune::function]`]: crate::function
+/// [`#[rune::macro_]`]: crate::macro_
+///
+/// # Examples
 ///
 /// ```
 /// use rune::Any;
 ///
 /// #[derive(Any)]
 /// struct Npc {
-///     name: String,
+///     #[rune(get)]
+///     health: u32,
+/// }
+///
+/// impl Npc {
+///     /// Construct a new NPC.
+///     #[rune::function(path = Self::new)]
+///     fn new(health: u32) -> Self {
+///         Self {
+///             health
+///         }
+///     }
+///
+///     /// Damage the NPC with the given `amount`.
+///     #[rune::function]
+///     fn damage(&mut self, amount: u32) {
+///         self.health -= amount;
+///     }
+/// }
+///
+/// fn install() -> Result<rune::Module, rune::ContextError> {
+///     let mut module = rune::Module::new();
+///     module.ty::<Npc>()?;
+///     module.function_meta(Npc::new)?;
+///     module.function_meta(Npc::damage)?;
+///     Ok(module)
 /// }
 /// ```
 pub trait Any: Named + any::Any {
     /// The type hash of the type.
-    ///
-    /// TODO: make const field when `TypeId::of` is const.
     fn type_hash() -> Hash;
 }
 

crates/rune/src/ast/attribute.rs

@@ -153,7 +153,7 @@ impl Parse for AttrStyle {
 }
 
 /// Helper struct to only parse inner attributes.
-pub(crate) struct InnerAttribute(pub(crate) Attribute);
+pub(crate) struct InnerAttribute(#[allow(unused)] pub(crate) Attribute);
 
 impl Parse for InnerAttribute {
     fn parse(p: &mut Parser) -> Result<Self> {

crates/rune/src/doc/context.rs

@@ -12,7 +12,7 @@ pub(crate) enum MetaSource<'a> {
     /// Meta came from context.
     Context,
     /// Meta came from source.
-    Source(&'a Item),
+    Source(#[allow(unused)] &'a Item),
 }
 
 #[derive(Debug, Clone, Copy)]
@@ -94,7 +94,7 @@ pub(crate) enum Kind<'a> {
     Enum,
     Macro,
     Function(Function<'a>),
-    Const(&'a ConstValue),
+    Const(#[allow(unused)] &'a ConstValue),
     Module,
 }
 

crates/rune/src/lib.rs

@@ -184,7 +184,7 @@ pub mod alloc;
 #[cfg(test)]
 pub(crate) mod testing;
 
-/// Helper prelude for #[no_std] support.
+/// Helper prelude for `#[no_std]` support.
 pub mod no_std;
 
 #[macro_use]
@@ -200,6 +200,7 @@ pub mod ast;
 pub mod fmt;
 
 cfg_emit! {
+    #[doc(inline)]
     pub use ::codespan_reporting::term::termcolor;
 }
 
@@ -243,15 +244,18 @@ pub mod parse;
 pub mod query;
 
 pub mod runtime;
+#[doc(inline)]
 pub use self::runtime::{from_value, to_value, FromValue, ToValue, Unit, Value, Vm};
 
 mod shared;
 
 pub mod source;
+#[doc(inline)]
 pub use self::source::Source;
 
 #[macro_use]
 mod sources;
+#[doc(inline)]
 pub use self::sources::{SourceId, Sources};
 
 mod worker;

crates/rune/src/runtime/budget/std.rs

@@ -1,6 +1,6 @@
 use core::cell::Cell;
 
-std::thread_local!(static BUDGET: Cell<usize> = Cell::new(usize::MAX));
+std::thread_local!(static BUDGET: Cell<usize> = const { Cell::new(usize::MAX) });
 
 pub(super) fn rune_budget_take() -> bool {
     BUDGET.with(|tls| {

crates/rune/src/runtime/env/std.rs

@@ -2,7 +2,7 @@ use core::cell::Cell;
 
 use super::Env;
 
-std::thread_local!(static ENV: Cell<Env> = Cell::new(Env::null()));
+std::thread_local!(static ENV: Cell<Env> = const { Cell::new(Env::null()) });
 
 pub(super) fn rune_env_get() -> Env {
     ENV.with(|env| env.get())

crates/rune/src/runtime/vm.rs

@@ -106,6 +106,14 @@ pub struct Vm {
 
 impl Vm {
     /// Construct a new virtual machine.
+    ///
+    /// Constructing a virtual machine is a cheap constant-time operation.
+    ///
+    /// See [`unit_mut`] and [`context_mut`] documentation for information on
+    /// how to re-use existing [`Vm`]'s.
+    ///
+    /// [`unit_mut`]: Vm::unit_mut
+    /// [`context_mut`]: Vm::context_mut
     pub const fn new(context: Arc<RuntimeContext>, unit: Arc<Unit>) -> Self {
         Self::with_stack(context, unit, Stack::new())
     }
@@ -169,6 +177,19 @@ impl Vm {
     }
 
     /// Access the context related to the virtual machine mutably.
+    ///
+    /// Note that this can be used to swap out the [`RuntimeContext`] associated
+    /// with the running vm. Note that this is only necessary if the underlying
+    /// [`Context`] is different or has been modified. In contrast to
+    /// constructing a [`new`] vm, this allows for amortised re-use of any
+    /// allocations.
+    ///
+    /// After doing this, it's important to call [`clear`] to clean up any
+    /// residual state.
+    ///
+    /// [`clear`]: Vm::clear
+    /// [`Context`]: crate::Context
+    /// [`new`]: Vm::new
     #[inline]
     pub fn context_mut(&mut self) -> &mut Arc<RuntimeContext> {
         &mut self.context
@@ -180,7 +201,17 @@ impl Vm {
         &self.context
     }
 
-    /// Access the underlying unit of the virtual machine mutablys.
+    /// Access the underlying unit of the virtual machine mutably.
+    ///
+    /// Note that this can be used to swap out the [`Unit`] of execution in the
+    /// running vm. In contrast to constructing a [`new`] vm, this allows for
+    /// amortised re-use of any allocations.
+    ///
+    /// After doing this, it's important to call [`clear`] to clean up any
+    /// residual state.
+    ///
+    /// [`clear`]: Vm::clear
+    /// [`new`]: Vm::new
     #[inline]
     pub fn unit_mut(&mut self) -> &mut Arc<Unit> {
         &mut self.unit

crates/rune/src/tests/vm_test_from_value_derive.rs

@@ -67,6 +67,7 @@ fn test_missing_dynamic_field() {
     );
 
     #[derive(Debug, FromValue)]
+    #[allow(unused)]
     struct ProxyTuple(u32, u32);
 
     assert_vm_error!(

examples/Cargo.toml

@@ -10,7 +10,10 @@ full = ["rune-modules/full"]
 default = ["full"]
 
 [dependencies]
-tokio = { version = "1.28.1", features = ["macros"] }
-
 rune = { path = "../crates/rune" }
 rune-modules = { path = "../crates/rune-modules" }
+
+tokio = { version = "1.28.1", features = ["macros"] }
+notify = "6.1.1"
+anyhow = "1.0.82"
+pin-project = "1.1.5"