From 234559dd19fc4c9b77ef065301626c72f89ebf44 Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 8 Sep 2024 12:42:22 -0700 Subject: [PATCH 01/43] Introduce ESP32 ADC driver supporting ESP-IDF v5.x This driver offers resource based nif APIs using unit and channel resources, as well as a set of convenient gen_server based APIs, which use pin numbers, as an alternative to using the nif resource handles. Unit 2 is currently disabled for the ESP32C3 by default due to known hardware limitations, see: https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/api-reference/peripherals/adc_oneshot.html#hardware-limitations Unit 2 is currently disabled by default for the ESP32 (classic) due to WiFi conflicts. This decision can be revisited after PR #1137 is merged, and the WiFi driver can be successfully stopped after it has been started. Unit 2 is enabled by default for all other chips with more than one ADC unit, as the chips released after the ESP32 classic have a functional ADC arbitrator peripheral that allows ADC unit 2 use while WiFi is enabled. Signed-off-by: Winford --- libs/eavmlib/src/CMakeLists.txt | 1 + libs/eavmlib/src/esp_adc.erl | 541 ++++++++++++ .../components/avm_builtins/CMakeLists.txt | 3 +- .../esp32/components/avm_builtins/Kconfig | 27 + .../components/avm_builtins/adc_driver.c | 781 ++++++++++++++++++ 5 files changed, 1352 insertions(+), 1 deletion(-) create mode 100644 libs/eavmlib/src/esp_adc.erl create mode 100644 src/platforms/esp32/components/avm_builtins/adc_driver.c diff --git a/libs/eavmlib/src/CMakeLists.txt b/libs/eavmlib/src/CMakeLists.txt index d3f2262c8..b237bc9d9 100644 --- a/libs/eavmlib/src/CMakeLists.txt +++ b/libs/eavmlib/src/CMakeLists.txt @@ -29,6 +29,7 @@ set(ERLANG_MODULES console emscripten esp + esp_adc gpio i2c http_server diff --git a/libs/eavmlib/src/esp_adc.erl b/libs/eavmlib/src/esp_adc.erl new file mode 100644 index 000000000..d50948453 --- /dev/null +++ b/libs/eavmlib/src/esp_adc.erl @@ -0,0 +1,541 @@ +%% +%% Copyright (c) 2020-2023 dushin.net +%% Copyright (c) 2022-2024 Winford +%% All rights reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +% +% SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later +% + +%%----------------------------------------------------------------------------- +%% @doc Analog to digital peripheral support. +%% +%% Use this module to take ADC (analog voltage) readings. Currently this driver +%% only supports the esp32 family of chips, but support for other platforms is +%% planned in the future. On an ESP32 device ADC unit 1 allows taking reading from +%% pins 32-39. ADC unit2 is disabled by default for the ESP32 classic, but when +%% enabled in the build configuration allows pins 0, 2, 4, 12-15, and 25-27 to be +%% used as long as WiFi is not required by the application. Unit 2 is disabled for +%% ESP32C3 due to its inaccurate results. ADC unit 2 is enabled for all other ESP32 +%% series with more than one ADC unit; there is an arbitrator peripheral that allows +%% ADC unit 2 to be used while WiFI is active. The pins available for ADC use vary +%% by device, check your datasheet for specific hardware support. +%% +%% There are two sets of APIs for interacting with the ADC hardware, only one set +%% of API may be used by an application. +%% +%% The core functionality is provided by the low level resource based nif functions. +%% To use the resource based nifs `esp_adc:init/0' and `esp_adc:deinit/1' will acquire and +%% release the adc `unit' resource needed for all other functions. A `channel' resource +%% used to take measurements from a pin can be obtained using `esp_adc:acquire/4', and +%% released using `esp_adc:release_channel/1'. ADC measurements are taken using `sample/3'. +%% +%% For convenience a gen_server managed set of APIs using pin numbers are also available. +%% A pin may be configured using `esp_adc:start/1,2', measurements are taken using +%% `esp_adc:read/1,2', pins can be released individually with `esp_adc:stop/1`, or the driver +%% can be stopped completely using `esp_adc:stop/0'. +%% @end +%%----------------------------------------------------------------------------- +-module(esp_adc). + +-behaviour(gen_server). + +%% Low level resource based nif functions +-export([acquire/4, init/0, release_channel/1, deinit/1, sample/3]). +%% Nif convenience functions +-export([acquire/2, sample/2]). + +%% gen_server convenience functions +-export([start/0, start/1, start/2, read/1, read/2, stop/1, stop/0]). +%% gen_server internals +-export([init/1, handle_call/3, handle_cast/2]). +-export([handle_info/2, terminate/2]). + +-type adc_rsrc() :: {'$adc', Resource :: binary(), Ref :: reference()}. +-type adc_pin() :: non_neg_integer(). +%% ADC capable pins vary by chipset. Consult your datasheet. +-type bit_width() :: bit_9 | bit_10 | bit_11 | bit_12 | bit_13 | bit_max. +%% The default `bit_max' will select the highest value supported by the chipset. Some models only support a single +%% fixed bit width. +-type attenuation() :: db_0 | db_2_5 | db_6 | db_11 | db_12. +%% The decibel gain determines the maximum safe voltage to be measured. Default is `db_11'. The specific range of +%% voltages supported by each setting varies by device. Typical voltage ranges are depicted in the table below: +%% +%% +%% +%% +%% +%% +%%
Attenuation Min Millivolts Max Millivolts
`db_0' 0-100 750-950
`db_2_5' 0-100 1050-1250
`db_6' 0-150 1300-1750
`bd_11' | `db_12' 0-150 2450-2500
+%% Consult the datasheet for your device to determine the exact voltage ranges supported by each gain setting. +%% The option `db_11' has been superseded by `db_12'. The option`db_11' and will be deprecated in a future release, +%% applications should be updated to use `db_12' (except for builds with ESP-IDF versions prior to v5.2). To Continue +%% to support older IDF version builds, the default will remain `db_11', which is the maximum tolerated voltage on +%% all builds, as `db_12' supported builds will automatically use `db_12' in place of `db_11', when `db_11' is +%% deprecated in all builds the default will be changed to `db_12'. + +-type pin_options() :: [pin_option()]. +-type pin_option() :: {bitwidth, Width :: bit_width()} | {atten, Decibels :: attenuation()}. +-type read_options() :: [read_option()]. +-type read_option() :: raw | voltage | {samples, 1..100000}. +%% The value of the `samples' key is the number of samples to be taken and averaged when returning a measurement, +%% default is 64. For optimal stable readings use a 100nF ceramic capacitor input filter, for more info consult +%% Espressif's "ADC Calibration Driver" documentation. +%% The keys `raw' and `voltage' determine if these values are included in the results or returned as `undefined'. +-type raw_value() :: 0..511 | 0..1023 | 0..2047 | 0..4095 | 0..8191 | undefined. +%% The maximum analog value is determined by `bit_width()'. +-type voltage_reading() :: 0..3300 | undefined. +%% The maximum safe millivolt value that can be measured is determined by `attenuation()', this value should never +%% exceed the chips maximum input tolerance. +-type reading() :: {raw_value() | undefined, voltage_reading() | undefined}. + +-define(ADC_RSRC, {'$adc', _Resource, _Ref}). +-define(DEFAULT_SAMPLES, 64). +-define(DEFAULT_READ_OPTIONS, [raw, voltage, {samples, ?DEFAULT_SAMPLES}]). +-define(DEFAULT_PIN_OPTIONS, [{bitwidth, bit_max}, {atten, db_11}]). + +%%----------------------------------------------------------------------------- +%% @returns {ok, ADCUnit :: adc_rsrc()} | {error, Reason} +%% @doc Nif to initialize the ADC unit hardware. +%% +%% The returned ADC unit handle resource must be supplied for all future ADC operations. +%% +%% This is a low level nif that cannot be used in an application that uses the +%% convenience functions. +%% @end +%%----------------------------------------------------------------------------- +-spec init() -> {ok, ADCUnit :: adc_rsrc()} | {error, Reason :: term()}. +init() -> + throw(nif_error). + +%%----------------------------------------------------------------------------- +%% @param UnitResource returned from init/0 +%% @returns ok | {error, Reason} +%% @doc Nif to release the ADC unit resource returned from init/0. +%% +%% Stop the ADC driver and free the unit resource. All active ADC channels should +%% be released using `release_channel/1' to free each configured channel before +%% freeing the unit resource. +%% +%% This is a low level nif that cannot be used in an application that uses the +%% convenience functions. +%% @end +%%----------------------------------------------------------------------------- +-spec deinit(UnitResource :: adc_rsrc()) -> ok | {error, Reason :: term()}. +deinit(_UnitResource) -> + throw(nif_error). + +%%----------------------------------------------------------------------------- +%% @param Pin Pin to configure as ADC +%% @param UnitHandle The unit handle returned from `init/0' +%% @equiv acquire(Pin, UnitHandle, bit_max, db_11) +%% @returns {ok, Channel::adc_rsrc()} | {error, Reason} +%% @doc Nif to initialize an ADC pin. +%% +%% Initializes an ADC pin and returns a channel handle resources. +%% +%% This is a low level nif that cannot be used in an application that uses the +%% convenience functions. +%% @end +%%----------------------------------------------------------------------------- +-spec acquire(Pin :: adc_pin(), UnitHandle :: adc_rsrc()) -> + {ok, Channel :: adc_rsrc()} | {error, Reason :: term()}. +acquire(Pin, UnitHandle) -> + ?MODULE:acquire(Pin, UnitHandle, bit_max, db_11). + +%%----------------------------------------------------------------------------- +%% @param Pin Pin to configure as ADC +%% @param UnitHandle The unit handle returned from `init/0' +%% @param BitWidth Resolution in bit to measure +%% @param Attenuation Decibel gain for voltage range +%% @returns {ok, Channel::adc_rsrc()} | {error, Reason} +%% @doc Nif to initialize an ADC pin. +%% +%% The BitWidth value `bit_max' may be used to automatically select the highest +%% sample rate supported by your ESP chip-set, or choose from a bit width supported +%% by the device. +%% +%% The Attenuation value can be used to adust the gain, and therefore safe +%% measurement range on voltage the exact range of voltages supported by each +%% db gain varies by chip, consult the data sheet for exact range of your model. +%% For more information see the `attenuation()' type specification. +%% +%% Use the returned `Channel' reference in subsequent ADC operations on +%% the same pin. +%% +%% This is a low level nif that cannot be used in an application that uses the +%% convenience functions. +%% @end +%%----------------------------------------------------------------------------- +-spec acquire( + Pin :: adc_pin(), + UnitHandle :: adc_rsrc(), + BitWidth :: bit_width(), + Attenuation :: attenuation() +) -> {ok, Channel :: adc_rsrc()} | {error, Reason :: term()}. +acquire(_Pin, _UnitHandle, _BitWidth, _Attenuation) -> + throw(nif_error). + +%%----------------------------------------------------------------------------- +%% @param ChannelResource of the pin returned from acquire/4 +%% @returns ok | {error, Reason} +%% @doc Nif to deinitialize the specified ADC channel. +%% +%% In the case that an error is returned it is safe to "drop" the `ChannelResource' +%% handle from use. After there are no remaining processes with references to +%% the channel resource handle, the calibration profile and any remaining resources +%% associated with the channel will be released as part of the next garbage +%% collection event. +%% +%% This is a low level nif that cannot be used in an application that uses the +%% convenience functions. +%% @end +%%----------------------------------------------------------------------------- +-spec release_channel(ChannelResource :: adc_rsrc()) -> ok | {error, Reason :: term()}. +release_channel(_ChannelResource) -> + throw(nif_error). + +%%----------------------------------------------------------------------------- +%% @param ChannelResource of the pin returned from acquire/4 +%% @param UnitResource of the pin returned from init/0 +%% @returns {ok, {RawValue, MilliVolts}} | {error, Reason} +%% @equiv sample(ChannelResource, UnitResource, [raw, voltage, {samples, 64}]) +%% @doc Nif to take a reading using default values from an ADC channel. +%% +%% This is a low level nif that cannot be used in an application that uses the +%% convenience functions. +%% @end +%%----------------------------------------------------------------------------- +-spec sample(ChannelResource :: adc_rsrc(), UnitResource :: adc_rsrc()) -> + {ok, reading()} | {error, Reason :: term()}. +sample(ChannelResource, UnitResource) -> + ?MODULE:sample(ChannelResource, UnitResource, ?DEFAULT_READ_OPTIONS). + +%%----------------------------------------------------------------------------- +%% @param ChannelResource of the pin returned from acquire/4 +%% @param UnitResource of the pin returned from init/0 +%% @param ReadOptions extra list of options to override defaults. +%% @returns {ok, {RawValue, MilliVolts}} | {error, Reason} +%% @doc Nif to take a reading from an ADC channel. +%% +%% The Options parameter may be used to specify the behavior of the read +%% operation. +%% +%% If the ReadOptions contains the atom `raw', then the raw value will be returned +%% in the first element of the returned tuple. Otherwise, this element will be +%% the atom `undefined'. +%% +%% If the ReadOptions contains the atom `voltage', then the voltage value will be +%% returned in millivolts in the second element of the returned tuple. Otherwise, +%% this element will be the atom `undefined'. +%% +%% You may specify the number of samples to be taken and averaged over using the +%% tuple `{samples, Samples::pos_integer()}'. +%% +%% If the error `Reason' is timeout and the adc channel is on unit 2 then WiFi is +%% likely enabled and adc2 readings may be blocked until there is less network +%% traffic. On and ESP32 classic the results for unit 2 will always be +%% `{error, timeout}' if wifi is enabled. +%% +%% This is a low level nif that cannot be used in an application that uses the +%% convenience functions. +%% @end +%%----------------------------------------------------------------------------- +-spec sample( + ChannelResource :: adc_rsrc(), UnitResource :: adc_rsrc(), ReadOptions :: read_options() +) -> {ok, Result :: reading()} | {error, Reason :: term()}. +sample(_ChannelResource, _UnitResource, _ReadOptions) -> + throw(nif_error). + +%%----------------------------------------------------------------------------- +%% @returns {ok, Pid} +%% @doc Optionally initialize a gen_server managed ADC driver without a pin. +%% +%% Use of this function is optional, but may be desired if the drivers pid is needed, +%% or it is desireable to start the driver without configuring an initial ADC channel. +%% +%% Note: since only one instance of the driver is allowed it is registered with the +%% name `adc_driver', which also may be used to directly call the gen_server. +%% +%% This convenience function cannot be used in an application that uses the low level +%% nif APIs. +%% @end +%%----------------------------------------------------------------------------- +-spec start() -> {ok, Pid :: pid()}. +start() -> + Pid = get_adc_pid(), + {ok, Pid}. + +%%----------------------------------------------------------------------------- +%% @param Pin Pin to configure as ADC +%% @equiv start(Pin, [{bitwidth, bit_max}, {atten, db_11}]) +%% @returns ok | {error, Reason} +%% @doc Initialize a gen_server managed ADC pin with default options. +%% +%% This convenience function configures an ADC pin with the default options for +%% use with the optional `gen_server' APIs. Default options are: +%% `[{bitwidth, bit_max}, {atten, db_11}]' +%% +%% This function cannot be used in an application that uses the low level +%% nif APIs. +%% @end +%%----------------------------------------------------------------------------- +-spec start(Pin :: adc_pin()) -> ok | {error, Reason :: term()}. +start(Pin) -> + start(Pin, ?DEFAULT_PIN_OPTIONS). + +%%----------------------------------------------------------------------------- +%% @param Pin Pin to configure as ADC +%% @param Options List of options to override default settings +%% @returns ok | {error, Reason} +%% @doc Initialize a gen_server managed ADC pin with the supplied options. +%% +%% This convenience function configures an ADC pin with the provided options to +%% override the default configuration: `[{bitwidth, bit_max}, {atten, db_11}]'. +%% +%% For more details about these options see the `attenuation()' and `bit_width()' +%% type specifications. +%% +%% This function cannot be used in an application that uses the low level nif APIs. +%% @end +%%----------------------------------------------------------------------------- +-spec start(Pin :: adc_pin(), Options :: pin_options()) -> ok | {error, Reason :: term()}. +start(Pin, Options) -> + {Bits, Atten} = validate_pin_options(Options), + gen_server:call(get_adc_pid(), {acquire, Pin, Bits, Atten}). + +%%----------------------------------------------------------------------------- +%% @param Pin the pin to be released +%% @returns ok | {error, Reason} +%% @doc De-initialize the specified ADC pin. +%% +%% This convenience function is used to release a pin from the gen_server managed +%% ADC driver. If an error is returned the ADC channel will still be stopped and +%% release internal resources during the next VM garbage collection event, the pin +%% will immediately no longer be useable in any case. +%% +%% This function cannot be used in an application that uses the low level nif APIs. +%% @end +%%----------------------------------------------------------------------------- +-spec stop(Pin :: adc_pin()) -> ok | {error, Reason :: term()}. +stop(Pin) -> + gen_server:call(adc_driver, {stop, Pin}). + +%%----------------------------------------------------------------------------- +%% @returns ok | {error, Reason} +%% @doc Stop the ADC driver and release all resources. +%% +%% This convenience function is used to completely stop the gen_server managed +%% ADC driver and release all resources. +%% +%% Note: if an error is returned, a full shutdown of the ADC peripheral should +%% still occur, and any remaining resources freed with next VM garbage collection +%% event. Regardless the gen_server will exit normally and the adc peripheral +%% will no longer be usable. +%% +%% This function cannot be used in an application that uses the low level nif APIs. +%% @end +%%----------------------------------------------------------------------------- +-spec stop() -> ok | {error, Reason :: term()}. +stop() -> + gen_server:call(adc_driver, stop). + +%%----------------------------------------------------------------------------- +%% @param Pin The pin from which to take ADC measurement +%% @returns {ok, {RawValue, MilliVolts}} | {error, Reason} +%% @equiv read(Pin, [raw, voltage, {samples, 64}]) +%% @doc Take a reading using default values from an ADC pin. +%% +%% This convenience function is used to take a measurement from a previously +%% started adc pin. +%% +%% This function cannot be used in an application that uses the low level +%% nif APIs. +%% @end +%%----------------------------------------------------------------------------- +-spec read(Pin :: adc_pin()) -> {ok, reading()} | {error, Reason :: term()}. +read(Pin) -> + gen_server:call(adc_driver, {read, Pin, ?DEFAULT_READ_OPTIONS}). + +%%----------------------------------------------------------------------------- +%% @param Pin The pin from which to take ADC measurement +%% @param ReadOptions Extra options +%% @returns {ok, {RawValue, MilliVolts}} | {error, Reason} +%% @doc Take a reading from an ADC pin using the supplied options. +%% +%% This convenience function is used to take a measurement from a previously +%% started adc pin, using the supplied read options parameter. +%% +%% The Options parameter may be used to specify the behavior of the read +%% operation. +%% +%% If the ReadOptions contains the atom `raw', then the raw value will be returned +%% in the first element of the returned tuple. Otherwise, this element will be the +%% atom `undefined'. +%% +%% If the ReadOptions contains the atom `voltage', then the voltage value will be returned +%% in millivolts in the second element of the returned tuple. Otherwise, this element will +%% be the atom `undefined'. +%% +%% You may specify the number of samples to be taken and averaged over using the tuple +%% `{samples, Samples::pos_integer()}', the default is `64'. +%% +%% If the error `Reason' is timeout and the adc channel is on unit 2 then WiFi is +%% likely enabled and adc2 readings may be blocked until there is less network +%% traffic. On and ESP32 classic the results for unit 2 will always be +%% `{error, timeout}' if wifi is enabled. +%% +%% This function cannot be used in an application that uses the low level nif APIs. +%% @end +%%----------------------------------------------------------------------------- +-spec read(Pin :: adc_pin(), ReadOptions :: read_options()) -> + {ok, Result :: reading()} | {error, Reason :: term()}. +read(Pin, ReadOptions) -> + gen_server:call(adc_driver, {read, Pin, ReadOptions}). + +%% +%% gen_server +%% + +-record(state, { + handle, + pins = #{} +}). + +%% @hidden +init([]) -> + try ?MODULE:init() of + {ok, UnitHandle} -> + {ok, #state{handle = UnitHandle}}; + Error -> + Error + catch + _E:Reason -> + {error, Reason} + end. + +%% @hidden +handle_call(stop, _From, State) -> + case do_stop_driver(State) of + {ok, NewState} -> + {stop, normal, ok, NewState}; + {Error, NewState} -> + {stop, normal, {error, Error}, NewState} + end; +handle_call({stop, Pin}, _From, State) -> + {Result, NewState} = do_deinit_pin(Pin, State), + {reply, Result, NewState}; +handle_call({acquire, Pin, Bits, Atten}, _From, State) -> + {Result, NewState} = do_config_pin({Pin, Bits, Atten}, State), + {reply, Result, NewState}; +handle_call({read, Pin, Options}, _From, State) -> + {reply, do_take_reading(maps:get(Pin, State#state.pins), State#state.handle, Options), State}; +handle_call(Request, _From, State) -> + {reply, {error, {unknown_request, Request}}, State}. + +%% @hidden +handle_cast(_Msg, State) -> + {noreply, State}. + +%% @hidden +handle_info(_Info, State) -> + {noreply, State}. + +%% @hidden +terminate(_Reason, _State) -> + ok. + +%% +%% private fun +%% + +%% private +get_adc_pid() -> + case erlang:whereis(adc_driver) of + undefined -> + case gen_server:start_link({local, adc_driver}, ?MODULE, [], []) of + {ok, Pid} -> Pid; + Err -> erlang:throw(Err) + end; + Pid when is_pid(Pid) -> + Pid + end. + +% private +validate_pin_options(Options) -> + Bits = proplists:get_value(bitwidth, Options, bit_max), + Atten = proplists:get_value(atten, Options, db_11), + {Bits, Atten}. + +%private +do_config_pin({Pin, Bits, Atten}, State) -> + try ?MODULE:acquire(Pin, State#state.handle, Bits, Atten) of + {ok, ChanRsrc} -> + {ok, State#state{pins = maps:put(Pin, ChanRsrc, State#state.pins)}}; + Error -> + io:format("[~p] adc_driver: failed to acquire pin, error: ~p~n", [ + erlang:monotonic_time(millisecond), Error + ]), + {Error, State} + catch + _E:Reason -> + {{error, Reason}, State} + end. + +% private +do_take_reading(ChannelHandle, UnitHandle, Options) -> + try ?MODULE:sample(ChannelHandle, UnitHandle, Options) of + Result -> Result + catch + _E:Reason -> + {error, Reason} + end. + +% private +do_deinit_pin(Pin, State) -> + try ?MODULE:release_channel(maps:get(Pin, State#state.pins)) of + ok -> + {ok, State#state{pins = maps:remove(Pin, State#state.pins)}}; + Error -> + {Error, State#state{pins = maps:remove(Pin, State#state.pins)}} + catch + _E:Reason -> + {{error, Reason}, State#state{pins = maps:remove(Pin, State#state.pins)}} + end. + +% private +do_stop_driver(State) -> + Pins = maps:keys(State#state.pins), + case Pins of + [] -> + NewState = State; + _ -> + NewState = stop_pins_from_list(Pins, State) + end, + try ?MODULE:deinit(NewState#state.handle) of + ok -> + {ok, NewState#state{handle = undefined}}; + Error -> + {Error, NewState#state{handle = undefined}} + catch + _E:Reason -> + {{error, Reason}, NewState#state{handle = undefined}} + end. + +% private +stop_pins_from_list([], State) -> + {ok, State}; +stop_pins_from_list([Pin | List], State) -> + {ok, NewState} = do_deinit_pin(Pin, State), + stop_pins_from_list(List, NewState). diff --git a/src/platforms/esp32/components/avm_builtins/CMakeLists.txt b/src/platforms/esp32/components/avm_builtins/CMakeLists.txt index ff163591a..d0fd79e7a 100644 --- a/src/platforms/esp32/components/avm_builtins/CMakeLists.txt +++ b/src/platforms/esp32/components/avm_builtins/CMakeLists.txt @@ -35,7 +35,8 @@ set(AVM_BUILTIN_COMPONENT_SRCS ) if (IDF_VERSION_MAJOR GREATER_EQUAL 5) - set(ADDITIONAL_PRIV_REQUIRES "esp_hw_support" "efuse") + set(ADDITIONAL_PRIV_REQUIRES "esp_hw_support" "efuse" "esp_adc") + set(AVM_BUILTIN_COMPONENT_SRCS "adc_driver.c" ${AVM_BUILTIN_COMPONENT_SRCS}) else() set(ADDITIONAL_PRIV_REQUIRES "") endif() diff --git a/src/platforms/esp32/components/avm_builtins/Kconfig b/src/platforms/esp32/components/avm_builtins/Kconfig index 02fb3ed6a..c4196bfbc 100644 --- a/src/platforms/esp32/components/avm_builtins/Kconfig +++ b/src/platforms/esp32/components/avm_builtins/Kconfig @@ -20,6 +20,33 @@ menu "AtomVM Built-In Components" +config AVM_ENABLE_ADC_NIFS + bool "Enable ADC NIFs" + default y + + config AVM_ADC2_ENABLE + depends on AVM_ENABLE_ADC_NIFS + depends on SOC_ADC_PERIPH_NUM >= 2 + bool "Enable ADC Unit 2" + default "y" if ADC_ONESHOT_FORCE_USE_ADC2_ON_C3 && IDF_TARGET_ESP32C3 + default "n" if IDF_TARGET_ESP32C3 + default "n" if IDF_TARGET_ESP32 + default "y" + help + This will allow using both ADC units. + + ADC2 is used by the Wi-Fi driver. The ESP32 classic can only use ADC2 when the + wifi driver is not in use, all other models have an arbitrator peripheral that + allows the simultaneous use of ADC unit 2 with WiFi. With the possibility of occasional + timeout errors when taking a measurements during times of heavy network traffic. + + The results from ADC2 on the ESP32C3 are not at all accurate and should not be used. + See: https://docs.espressif.com/projects/esp-chip-errata/en/latest/esp32c3/index.html + + For the ESP32C3 if you choose to ignore these warnings and enable unit 2 anyway, + "ADC_ONESHOT_FORCE_USE_ADC2_ON_C3" must also be enabled in the "ADC and ADC Calibration" + submenu. + config AVM_ENABLE_GPIO_NIFS bool "Enable GPIO NIFs" default y diff --git a/src/platforms/esp32/components/avm_builtins/adc_driver.c b/src/platforms/esp32/components/avm_builtins/adc_driver.c new file mode 100644 index 000000000..8afd60e40 --- /dev/null +++ b/src/platforms/esp32/components/avm_builtins/adc_driver.c @@ -0,0 +1,781 @@ +/* + * This file is part of AtomVM. + * + * Copyright 2020-2023 dushin.net + * Copyright 2024 Ricardo Lanziano + * Copyright 2022-2024 Winford + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later + */ + +// References +// https://docs.espressif.com/projects/esp-idf/en/v5.0.7/esp32/api-reference/peripherals/adc_oneshot.html +// https://docs.espressif.com/projects/esp-idf/en/v5.0.7/esp32/api-reference/peripherals/adc_calibration.html +// https://docs.espressif.com/projects/esp-idf/en/v5.3/esp32/api-reference/peripherals/adc_oneshot.html +// https://docs.espressif.com/projects/esp-idf/en/v5.3/esp32/api-reference/peripherals/adc_calibration.html + +#include +#ifdef CONFIG_AVM_ENABLE_ADC_NIFS + +#include +#include +#include +#include +#include +#include +#include +#include + +// #define ENABLE_TRACE +#include + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#define TAG "atomvm_adc" +#define DEFAULT_SAMPLES 64 +#define DEFAULT_VREF 1100U +#define ADC_INVALID_PARAM -1 +// 110000 will trigger the watchdog on esp32, other platforms can go higher, but this seems more than enough. +#define MAX_SAMPLES 100000 + +void atomvm_adc_init(GlobalContext *global); +const struct Nif *atomvm_adc_get_nif(const char *nifname); + +typedef enum avm_calibration_method +{ + UNCALIBRATED, + ESTIMATED, + CURVE, + LINE, +} cali_method_t; + +struct ChannelResource +{ + adc_atten_t attenuation; + adc_bitwidth_t width; + adc_unit_t adc_unit; + adc_channel_t channel; + adc_cali_handle_t cali_handle; + cali_method_t calibration; +}; + +struct UnitResource +{ + adc_oneshot_unit_handle_t unit_handle; +#ifdef CONFIG_AVM_ADC2_ENABLE + adc_oneshot_unit_handle_t unit2_handle; +#endif +}; + +static const AtomStringIntPair bit_width_table[] = { + { ATOM_STR("\x7", "bit_max"), ADC_BITWIDTH_DEFAULT }, +#if SOC_ADC_MAX_BITWIDTH == 13 + { ATOM_STR("\x6", "bit_13"), ADC_BITWIDTH_13 }, +#elif SOC_ADC_MAX_BITWIDTH == 12 + { ATOM_STR("\x6", "bit_12"), ADC_BITWIDTH_12 }, +#elif CONFIG_IDF_TARGET_ESP32 + { ATOM_STR("\x6", "bit_11"), ADC_BITWIDTH_11 }, + { ATOM_STR("\x6", "bit_10"), ADC_BITWIDTH_10 }, + { ATOM_STR("\x5", "bit_9"), ADC_BITWIDTH_9 }, +#endif + SELECT_INT_DEFAULT(ADC_INVALID_PARAM) +}; + +static const AtomStringIntPair attenuation_table[] = { + { ATOM_STR("\x4", "db_0"), ADC_ATTEN_DB_0 }, + { ATOM_STR("\x6", "db_2_5"), ADC_ATTEN_DB_2_5 }, + { ATOM_STR("\x4", "db_6"), ADC_ATTEN_DB_6 }, +#if (ESP_IDF_VERSION <= ESP_IDF_VERSION_VAL(5, 1, 0)) + { ATOM_STR("\x5", "db_11"), ADC_ATTEN_DB_11 }, +#else + { ATOM_STR("\x5", "db_12"), ADC_ATTEN_DB_12 }, +#endif + SELECT_INT_DEFAULT(ADC_INVALID_PARAM) +}; + +static ErlNifResourceType *adc_unit_resource; +static ErlNifResourceType *adc_channel_resource; + +// +// internal functions +// + +static bool is_adc_resource(GlobalContext *global, term t) +{ + bool ret = term_is_tuple(t) + && term_get_tuple_arity(t) == 3 + && globalcontext_is_term_equal_to_atom_string(global, term_get_tuple_element(t, 0), ATOM_STR("\x4", "$adc")) + && term_is_binary(term_get_tuple_element(t, 1)) + && term_is_reference(term_get_tuple_element(t, 2)); + + return ret; +} + +static bool to_channel_resource(term chan_resource, struct ChannelResource **rsrc_obj, Context *ctx) +{ + if (!is_adc_resource(ctx->global, chan_resource)) { + return false; + } + void *rsrc_obj_ptr; + if (UNLIKELY(!enif_get_resource(erl_nif_env_from_context(ctx), term_get_tuple_element(chan_resource, 1), adc_channel_resource, &rsrc_obj_ptr))) { + return false; + } + *rsrc_obj = (struct ChannelResource *) rsrc_obj_ptr; + + return true; +} + +static bool to_unit_resource(term unit_resource, struct UnitResource **rsrc_obj, Context *ctx) +{ + if (!is_adc_resource(ctx->global, unit_resource)) { + return false; + } + void *rsrc_obj_ptr; + if (UNLIKELY(!enif_get_resource(erl_nif_env_from_context(ctx), term_get_tuple_element(unit_resource, 1), adc_unit_resource, &rsrc_obj_ptr))) { + return false; + } + *rsrc_obj = (struct UnitResource *) rsrc_obj_ptr; + + return true; +} + +static int approximate_millivolts(int adc_reading, adc_atten_t attenuation, adc_bitwidth_t width) +{ + int digi_max = (int) pow(2, width); // casting double to int here is safe because values are always between 512-8192 + int millivolt_max = 0; + + switch (attenuation) { + case ADC_ATTEN_DB_0: + millivolt_max = 950; + break; + case ADC_ATTEN_DB_2_5: + millivolt_max = 1250; + break; + case ADC_ATTEN_DB_6: + millivolt_max = 1750; + break; +#if (ESP_IDF_VERSION <= ESP_IDF_VERSION_VAL(5, 1, 0)) + case ADC_ATTEN_DB_11: + millivolt_max = 2450; + break; +#else + case ADC_ATTEN_DB_12: + millivolt_max = 2450; + break; +#endif + } + + // Estimate V = RAW * ATTEN_MAX_MV / 2^BITWIDTH + // See: https://docs.espressif.com/projects/esp-idf/en/v5.2.2/esp32/api-reference/peripherals/adc_oneshot.html#read-conversion-result + + return (adc_reading * millivolt_max) / digi_max; +} + +static term adc_err_to_atom_term(GlobalContext *glb, esp_err_t error) +{ + term atom = term_invalid_term(); + switch (error) { + case ESP_ERR_INVALID_ARG: + atom = BADARG_ATOM; + break; + case ESP_ERR_NO_MEM: + atom = OUT_OF_MEMORY_ATOM; + break; + case ESP_ERR_NOT_FOUND: + atom = BADARG_ATOM; + break; + case ESP_FAIL: + atom = globalcontext_make_atom(glb, ATOM_STR("\x8", "no_clock")); + break; + case ESP_ERR_TIMEOUT: + atom = globalcontext_make_atom(glb, ATOM_STR("\x7", "timeout")); + break; + default: + atom = BADARG_ATOM; + } + return atom; +} + +static term create_pair(Context *ctx, term term1, term term2) +{ + term ret = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(ret, 0, term1); + term_put_tuple_element(ret, 1, term2); + + return ret; +} + +static term error_return_tuple(Context *ctx, term term) +{ + if (UNLIKELY(memory_ensure_free(ctx, TUPLE_SIZE(2)) != MEMORY_GC_OK)) { + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + return create_pair(ctx, ERROR_ATOM, term); +} + +static cali_method_t do_adc_calibration(adc_unit_t unit, adc_channel_t chan, adc_atten_t atten, adc_bitwidth_t width, adc_cali_handle_t *cali_handle) +{ + cali_method_t calibration = UNCALIBRATED; + esp_err_t err; + +#if defined ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED + adc_cali_curve_fitting_config_t cali_config = { + .unit_id = unit, + .chan = chan, + .atten = atten, + .bitwidth = width, + }; + err = adc_cali_create_scheme_curve_fitting(&cali_config, cali_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to calibrate using the supported curve fitting scheme: %s", esp_err_to_name(err)); + ESP_LOGW(TAG, "any reading requesting 'voltage' will receive an estimated value"); + } else { + calibration = CURVE; + ESP_LOGD(TAG, "Characterized using curve fitting scheme"); + } +#elif defined ADC_CALI_SCHEME_LINE_FITTING_SUPPORTED +#ifndef CONFIG_IDF_TARGET_ESP32 // other line fitting targets do not use default vref + adc_cali_line_fitting_config_t cali_config = { + .unit_id = unit, + .atten = atten, + .bitwidth = width, + }; + err = adc_cali_create_scheme_line_fitting(&cali_config, cali_handle); +#else // CONFIG_IDF_TARGET_ESP32 is defined + adc_cali_line_fitting_efuse_val_t cali_fuse; + err = adc_cali_scheme_line_fitting_check_efuse(&cali_fuse); + if ((err == ESP_OK) && (cali_fuse == ADC_CALI_LINE_FITTING_EFUSE_VAL_DEFAULT_VREF)) { + ESP_LOGI(TAG, "Unit calibrated with line_fitting scheme, channel uses default vref of %u mV.", DEFAULT_VREF); + ESP_LOGW(TAG, "A stable voltage of 1100 mV should be supplied to the pin during acquisition for accurate calibration."); + adc_cali_line_fitting_config_t cali_config = { + .unit_id = unit, + .atten = atten, + .bitwidth = width, + .default_vref = DEFAULT_VREF, + }; + err = adc_cali_create_scheme_line_fitting(&cali_config, cali_handle); + } else { + adc_cali_line_fitting_config_t cali_config = { + .unit_id = unit, + .atten = atten, + .bitwidth = width, + }; + err = adc_cali_create_scheme_line_fitting(&cali_config, cali_handle); + } +#endif // end CONFIG_IDF_TARGET ifelse + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to calibrate using the supported line fitting scheme: %s", esp_err_to_name(err)); + ESP_LOGW(TAG, "any reading requesting 'voltage' will receive an estimated value"); + } else { + calibration = LINE; + ESP_LOGD(TAG, "Characterized using line fitting scheme"); + } +#else // no calibration support defined + calibration = ESTIMATED; + ESP_LOGD(TAG, "No supported calibration method, readings requesting 'voltage' will receive an estimated value"); +#endif + + return calibration; +} + +// +// Nif functions +// + +static term nif_adc_init(Context *ctx, int argc, term argv[]) +{ + UNUSED(argc); + UNUSED(argv); + + struct UnitResource *unit_rsrc = enif_alloc_resource(adc_unit_resource, sizeof(struct UnitResource)); + if (IS_NULL_PTR(unit_rsrc)) { + ESP_LOGE(TAG, "failed to allocate resource: %s:%i.", __FILE__, __LINE__); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + unit_rsrc->unit_handle = NULL; + + adc_unit_t adc_unit = ADC_UNIT_1; + + adc_oneshot_unit_init_cfg_t init_config = { + .unit_id = adc_unit, + .ulp_mode = ADC_ULP_MODE_DISABLE, + }; + + esp_err_t err = adc_oneshot_new_unit(&init_config, &unit_rsrc->unit_handle); + if (UNLIKELY(err != ESP_OK)) { + return error_return_tuple(ctx, adc_err_to_atom_term(ctx->global, err)); + } + +#ifdef CONFIG_AVM_ADC2_ENABLE + unit_rsrc->unit2_handle = NULL; + + adc_unit = ADC_UNIT_2; + + adc_oneshot_unit_init_cfg_t init_config2 = { + .unit_id = adc_unit, + .ulp_mode = ADC_ULP_MODE_DISABLE, + }; + + err = adc_oneshot_new_unit(&init_config2, &unit_rsrc->unit2_handle); + if (UNLIKELY(err != ESP_OK)) { + return error_return_tuple(ctx, adc_err_to_atom_term(ctx->global, err)); + } +#endif + + if (UNLIKELY(memory_ensure_free(ctx, TERM_BOXED_RESOURCE_SIZE) != MEMORY_GC_OK)) { + enif_release_resource(unit_rsrc); + ESP_LOGE(TAG, "failed to allocate memory for resource: %s:%i.", __FILE__, __LINE__); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + ERL_NIF_TERM unit_obj = enif_make_resource(erl_nif_env_from_context(ctx), unit_rsrc); + enif_release_resource(unit_rsrc); + + // {ok, {'$adc', Unit :: resource(), ref()}} + size_t requested_size = TUPLE_SIZE(2) + TUPLE_SIZE(3) + REF_SIZE; + ESP_LOGD(TAG, "Requesting memory size %u for return message", requested_size); + if (UNLIKELY(memory_ensure_free_with_roots(ctx, requested_size, 1, &unit_obj, MEMORY_CAN_SHRINK) != MEMORY_GC_OK)) { + ESP_LOGE(TAG, "failed to allocate tuple memory size %u: %s:%i.", requested_size, __FILE__, __LINE__); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + term unit_resource = term_alloc_tuple(3, &ctx->heap); + term_put_tuple_element(unit_resource, 0, globalcontext_make_atom(ctx->global, ATOM_STR("\x4", "$adc"))); + term_put_tuple_element(unit_resource, 1, unit_obj); + uint64_t ref_ticks = globalcontext_get_ref_ticks(ctx->global); + term ref = term_from_ref_ticks(ref_ticks, &ctx->heap); + term_put_tuple_element(unit_resource, 2, ref); + + term ret = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(ret, 0, OK_ATOM); + term_put_tuple_element(ret, 1, unit_resource); + + return ret; +} + +static term nif_adc_deinit(Context *ctx, int argc, term argv[]) +{ + term unit_term = argv[0]; + if (UNLIKELY(!is_adc_resource(ctx->global, unit_term))) { + ESP_LOGE(TAG, "handle supplied is not a valid adc resource"); + RAISE_ERROR(BADARG_ATOM); + } + struct UnitResource *unit_rsrc = NULL; + if (UNLIKELY(!to_unit_resource(unit_term, &unit_rsrc, ctx))) { + ESP_LOGE(TAG, "resource supplied is not a valid adc unit resource"); + RAISE_ERROR(BADARG_ATOM); + } + + esp_err_t err = adc_oneshot_del_unit(unit_rsrc->unit_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release ADC Unit 1"); + return error_return_tuple(ctx, adc_err_to_atom_term(ctx->global, err)); + } + unit_rsrc->unit_handle = NULL; + ESP_LOGD(TAG, "ADC unit 1 released"); +#ifdef CONFIG_AVM_ADC2_ENABLE + err = adc_oneshot_del_unit(unit_rsrc->unit2_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release ADC Unit 2"); + return error_return_tuple(ctx, adc_err_to_atom_term(ctx->global, err)); + } + unit_rsrc->unit2_handle = NULL; + ESP_LOGD(TAG, "ADC unit 2 released"); +#endif + + return OK_ATOM; +} + +static term nif_adc_acquire(Context *ctx, int argc, term argv[]) +{ + UNUSED(argc); + term pin = argv[0]; + VALIDATE_VALUE(pin, term_is_integer); + int pin_num = term_to_int(pin); + + adc_unit_t adc_unit; + adc_channel_t adc_channel; + + esp_err_t err = adc_oneshot_io_to_channel(pin_num, &adc_unit, &adc_channel); + if (UNLIKELY(err != ESP_OK)) { + ESP_LOGE(TAG, "pin %i does not support ADC peripheral", pin_num); + RAISE_ERROR(BADARG_ATOM); + } + + term unit_term = argv[1]; + struct UnitResource *unit_rsrc = NULL; + if (UNLIKELY(!is_adc_resource(ctx->global, unit_term))) { + ESP_LOGE(TAG, "handle supplied is not a valid adc resource"); + RAISE_ERROR(BADARG_ATOM); + } + if (UNLIKELY(!to_unit_resource(unit_term, &unit_rsrc, ctx))) { + ESP_LOGE(TAG, "resource supplied is not a valid adc unit resource"); + RAISE_ERROR(BADARG_ATOM); + } + + adc_oneshot_unit_handle_t unit_handle = NULL; + if (adc_unit == ADC_UNIT_1) { + unit_handle = unit_rsrc->unit_handle; + } +#ifdef CONFIG_AVM_ADC2_ENABLE + else if (adc_unit == ADC_UNIT_2) { + unit_handle = unit_rsrc->unit2_handle; + } +#endif + else { + ESP_LOGE(TAG, "no enabled ADC unit matching pin"); + RAISE_ERROR(BADARG_ATOM); + } + + term width = argv[2]; + VALIDATE_VALUE(width, term_is_atom); + int bits = interop_atom_term_select_int(bit_width_table, width, ctx->global); + if (UNLIKELY(bits == ADC_INVALID_PARAM)) { + ESP_LOGE(TAG, "invalid bitwidth"); + RAISE_ERROR(BADARG_ATOM); + } + adc_bitwidth_t bit_width = (adc_bitwidth_t) bits; + + term attenuation = argv[3]; + VALIDATE_VALUE(attenuation, term_is_atom); + // TODO: remove macro and update log to ESP_LOGW after ESP-IDf v5.1 is EOL; then all current will deprecate db_11. +#if (ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 2, 0)) + if (attenuation == globalcontext_make_atom(ctx->global, ATOM_STR("\x5", "db_11"))) { + ESP_LOGI(TAG, "attenuation 'db_11' replaced by 'db_12' and will be deprecated in a future\nrelease, applications should be updated to use 'db_12' instead."); + attenuation = globalcontext_make_atom(ctx->global, ATOM_STR("\x5", "db_12")); + } +#endif + adc_atten_t atten = interop_atom_term_select_int(attenuation_table, attenuation, ctx->global); + if (UNLIKELY(atten == ADC_INVALID_PARAM)) { + ESP_LOGE(TAG, "invalid attenuation"); + RAISE_ERROR(BADARG_ATOM); + } + + adc_oneshot_chan_cfg_t config = { + .bitwidth = bit_width, + .atten = atten, + }; + + err = adc_oneshot_config_channel(unit_handle, adc_channel, &config); + if (UNLIKELY(err != ESP_OK)) { + return error_return_tuple(ctx, (adc_err_to_atom_term(ctx->global, err))); + } + + struct ChannelResource *chan_rsrc = enif_alloc_resource(adc_channel_resource, sizeof(struct ChannelResource)); + if (IS_NULL_PTR(chan_rsrc)) { + ESP_LOGE(TAG, "failed to allocate resource: %s:%i.", __FILE__, __LINE__); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + chan_rsrc->cali_handle = NULL; + cali_method_t calibration = do_adc_calibration(adc_unit, adc_channel, atten, bit_width, &chan_rsrc->cali_handle); + + chan_rsrc->attenuation = atten; + chan_rsrc->width = bit_width; + chan_rsrc->adc_unit = adc_unit; + chan_rsrc->channel = adc_channel; + chan_rsrc->calibration = calibration; + + if (UNLIKELY(memory_ensure_free(ctx, TERM_BOXED_RESOURCE_SIZE != MEMORY_GC_OK))) { + enif_release_resource(chan_rsrc); + ESP_LOGE(TAG, "failed to allocate memory for resource: %s:%i.", __FILE__, __LINE__); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + + term chan_obj = enif_make_resource(erl_nif_env_from_context(ctx), chan_rsrc); + enif_release_resource(chan_rsrc); + + // {ok, {'$adc', resource(), ref()}} + size_t requested_size = TUPLE_SIZE(2) + TUPLE_SIZE(3) + REF_SIZE; + ESP_LOGD(TAG, "Requesting memory size %u for return message", requested_size); + if (UNLIKELY(memory_ensure_free_with_roots(ctx, requested_size, 1, &chan_obj, MEMORY_CAN_SHRINK) != MEMORY_GC_OK)) { + ESP_LOGE(TAG, "failed to allocate tuple memory size %u: %s:%i.", requested_size, __FILE__, __LINE__); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + + term chan_resource = term_alloc_tuple(3, &ctx->heap); + term_put_tuple_element(chan_resource, 0, globalcontext_make_atom(ctx->global, ATOM_STR("\x4", "$adc"))); + term_put_tuple_element(chan_resource, 1, chan_obj); + uint64_t ref_ticks = globalcontext_get_ref_ticks(ctx->global); + term ref = term_from_ref_ticks(ref_ticks, &ctx->heap); + term_put_tuple_element(chan_resource, 2, ref); + + term ret = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(ret, 0, OK_ATOM); + term_put_tuple_element(ret, 1, chan_resource); + + return ret; +} + +static term nif_adc_release_channel(Context *ctx, int argc, term argv[]) +{ + if (UNLIKELY(!is_adc_resource(ctx->global, argv[0]))) { + ESP_LOGE(TAG, "no valid adc channel resource"); + RAISE_ERROR(BADARG_ATOM); + } + term channel_resource = argv[0]; + struct ChannelResource *chan_rsrc; + if (UNLIKELY(!to_channel_resource(channel_resource, &chan_rsrc, ctx))) { + ESP_LOGE(TAG, "resource is not a valid adc channel resource"); + RAISE_ERROR(BADARG_ATOM); + } + + if (chan_rsrc->calibration > ESTIMATED) { +#if ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED + ESP_LOGD(TAG, "deregister curve fitting calibration scheme"); + esp_err_t err = adc_cali_delete_scheme_curve_fitting(chan_rsrc->cali_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release calibration profile"); + return error_return_tuple(ctx, (adc_err_to_atom_term(ctx->global, err))); + } + +#elif ADC_CALI_SCHEME_LINE_FITTING_SUPPORTED + ESP_LOGD(TAG, "deregister line fitting calibration scheme"); + esp_err_t err = adc_cali_delete_scheme_line_fitting(chan_rsrc->cali_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release calibration profile"); + return error_return_tuple(ctx, (adc_err_to_atom_term(ctx->global, err))); + } +#endif + } + chan_rsrc->cali_handle = NULL; + chan_rsrc->calibration = UNCALIBRATED; + + return OK_ATOM; +} + +static term nif_adc_sample(Context *ctx, int argc, term argv[]) +{ + UNUSED(argc); + + term chan_term = argv[0]; + if (UNLIKELY(!is_adc_resource(ctx->global, chan_term))) { + ESP_LOGE(TAG, "Invalid channel resource"); + RAISE_ERROR(BADARG_ATOM); + } + struct ChannelResource *chan_rsrc; + if (UNLIKELY(!to_channel_resource(chan_term, &chan_rsrc, ctx))) { + ESP_LOGE(TAG, "failed to convert adc channel resource"); + RAISE_ERROR(BADARG_ATOM); + return BADARG_ATOM; + } + + term unit_term = argv[1]; + if (UNLIKELY(!is_adc_resource(ctx->global, unit_term))) { + ESP_LOGE(TAG, "handle supplied is not a valid adc resource"); + RAISE_ERROR(BADARG_ATOM); + } + struct UnitResource *unit_rsrc = NULL; + if (UNLIKELY(!to_unit_resource(unit_term, &unit_rsrc, ctx))) { + ESP_LOGE(TAG, "resource supplied is not a valid adc unit resource"); + RAISE_ERROR(BADARG_ATOM); + } + + adc_oneshot_unit_handle_t unit_handle = NULL; + if (chan_rsrc->adc_unit == ADC_UNIT_1) { + unit_handle = unit_rsrc->unit_handle; +#ifdef CONFIG_AVM_ADC2_ENABLE + } else if (chan_rsrc->adc_unit == ADC_UNIT_2) { + unit_handle = unit_rsrc->unit2_handle; +#endif + } else { + ESP_LOGE(TAG, "no valid unit handle found in resource"); + RAISE_ERROR(BADARG_ATOM); + } + + term read_options = argv[2]; + VALIDATE_VALUE(read_options, term_is_list); + term samples = interop_kv_get_value_default(read_options, ATOM_STR("\x7", "samples"), term_from_int32(DEFAULT_SAMPLES), ctx->global); + if (UNLIKELY(!term_is_integer(samples))) { + ESP_LOGE(TAG, "samples value must be an integer from 1 to 1024."); + RAISE_ERROR(BADARG_ATOM); + } + int samples_val = term_to_int32(samples); + if (UNLIKELY((samples_val < 1) || (samples_val > MAX_SAMPLES))) { + ESP_LOGE(TAG, "invalid samples value: %i, out of range (1..1024)", samples_val); + RAISE_ERROR(BADARG_ATOM); + } + ESP_LOGD(TAG, "read samples: %i", samples_val); + term raw = interop_kv_get_value_default(read_options, ATOM_STR("\x3", "raw"), FALSE_ATOM, ctx->global); + term voltage = interop_kv_get_value_default(read_options, ATOM_STR("\x7", "voltage"), FALSE_ATOM, ctx->global); + + int adc_reading = 0; + int adc_raw = 0; + + esp_err_t err = ESP_FAIL; + for (int i = 0; i < samples_val; ++i) { + err = adc_oneshot_read(unit_handle, chan_rsrc->channel, &adc_reading); + if (UNLIKELY(err != ESP_OK)) { + ESP_LOGE(TAG, "adc_oneshot_read read failed for unit: %i channel: %i", (int) chan_rsrc->adc_unit, (int) chan_rsrc->channel); + return adc_err_to_atom_term(ctx->global, err); + } + adc_raw += adc_reading; + } + + adc_raw /= samples_val; + ESP_LOGD(TAG, "read adc raw reading: %i", adc_raw); + + raw = raw == TRUE_ATOM ? term_from_int32(adc_raw) : UNDEFINED_ATOM; + if (voltage == TRUE_ATOM) { + int millivolts = 0; + if (chan_rsrc->calibration > ESTIMATED) { + err = adc_cali_raw_to_voltage(chan_rsrc->cali_handle, adc_raw, &millivolts); + if (UNLIKELY(err != ESP_OK)) { + ESP_LOGW(TAG, "Failed to get calibrated voltage, returning estimated voltage"); + voltage = term_from_int32(approximate_millivolts(adc_raw, chan_rsrc->attenuation, chan_rsrc->width)); + } else { + voltage = term_from_int32(millivolts); + } + } else { + ESP_LOGD(TAG, "ADC channel not calibrated, using estimated voltage"); + voltage = term_from_int32(approximate_millivolts(adc_raw, chan_rsrc->attenuation, chan_rsrc->width)); + } + } else { + voltage = UNDEFINED_ATOM; + }; + + size_t request_size = TUPLE_SIZE(2) + TUPLE_SIZE(2); + if (UNLIKELY(memory_ensure_free_opt(ctx, request_size, MEMORY_NO_SHRINK) != MEMORY_GC_OK)) { + return OUT_OF_MEMORY_ATOM; + } + term values = create_pair(ctx, raw, voltage); + term ret = create_pair(ctx, OK_ATOM, values); + + return ret; +} + +// +// Nif Entry/Exit +// + +static void nif_adc_chan_resource_dtor(ErlNifEnv *caller_env, void *obj) +{ + UNUSED(caller_env); + + struct ChannelResource *chan_rsrc = (struct ChannelResource *) obj; + + if (LIKELY((chan_rsrc->cali_handle != NULL) && (chan_rsrc->calibration > ESTIMATED))) { +#if ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED + ESP_LOGD(TAG, "deregister curve fitting calibration scheme"); + esp_err_t err = adc_cali_delete_scheme_curve_fitting(chan_rsrc->cali_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release curve fitting calibration profile"); + } + +#elif ADC_CALI_SCHEME_LINE_FITTING_SUPPORTED + ESP_LOGD(TAG, "deregister line fitting calibration scheme"); + esp_err_t err = adc_cali_delete_scheme_line_fitting(chan_rsrc->cali_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release line fitting calibration profile"); + } +#endif + } +} + +static void nif_adc_unit_resource_dtor(ErlNifEnv *caller_env, void *obj) +{ + UNUSED(caller_env); + + struct UnitResource *unit_rsrc = (struct UnitResource *) obj; + + if (LIKELY(unit_rsrc->unit_handle != NULL)) { + esp_err_t err = adc_oneshot_del_unit(unit_rsrc->unit_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release adc"); + } + } +#ifdef CONFIG_AVM_ADC2_ENABLE + if (LIKELY(unit_rsrc->unit2_handle != NULL)) { + esp_err_t err = adc_oneshot_del_unit(unit_rsrc->unit2_handle); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to release adc"); + } + } +#endif +} + +static const ErlNifResourceTypeInit ChannelResourceTypeInit = { + .members = 1, + .dtor = nif_adc_chan_resource_dtor, +}; + +static const ErlNifResourceTypeInit UnitResourceTypeInit = { + .members = 1, + .dtor = nif_adc_unit_resource_dtor, +}; + +static const struct Nif adc_init_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_adc_init +}; +static const struct Nif adc_deinit_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_adc_deinit +}; +static const struct Nif adc_acquire_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_adc_acquire +}; +static const struct Nif adc_release_channel_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_adc_release_channel +}; +static const struct Nif adc_sample_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_adc_sample +}; + +void atomvm_adc_init(GlobalContext *global) +{ + ErlNifEnv env; + erl_nif_env_partial_init_from_globalcontext(&env, global); + adc_channel_resource = enif_init_resource_type(&env, "adc_channel_resource", &ChannelResourceTypeInit, ERL_NIF_RT_CREATE, NULL); + adc_unit_resource = enif_init_resource_type(&env, "adc_unit_resource", &UnitResourceTypeInit, ERL_NIF_RT_CREATE, NULL); +} + +const struct Nif *atomvm_adc_get_nif(const char *nifname) +{ + TRACE("Locating nif %s ...", nifname); + if (strcmp("esp_adc:sample/3", nifname) == 0 || strcmp("Elixir.Esp.ADC:sample/3", nifname) == 0) { + TRACE("Resolved platform nif %s ...", nifname); + return &adc_sample_nif; + } + if (strcmp("esp_adc:acquire/4", nifname) == 0 || strcmp("Elixir.Esp.ADC:acquire/4", nifname) == 0) { + TRACE("Resolved platform nif %s ...", nifname); + return &adc_acquire_nif; + } + if (strcmp("esp_adc:release_channel/1", nifname) == 0 || strcmp("Elixir.Esp.ADC:release_channel/1", nifname) == 0) { + TRACE("Resolved platform nif %s ...", nifname); + return &adc_release_channel_nif; + } + if (strcmp("esp_adc:init/0", nifname) == 0 || strcmp("Elixir.Esp.ADC:init/0", nifname) == 0) { + TRACE("Resolved platform nif %s ...", nifname); + return &adc_init_nif; + } + if (strcmp("esp_adc:deinit/1", nifname) == 0 || strcmp("Elixir.Esp.ADC:deinit/1", nifname) == 0) { + TRACE("Resolved platform nif %s ...", nifname); + return &adc_deinit_nif; + } + return NULL; +} + +REGISTER_NIF_COLLECTION(atomvm_adc, atomvm_adc_init, NULL, atomvm_adc_get_nif) +#endif From da1bf5e6c49ee214c7f38e429c45ef69bc15e467 Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 8 Sep 2024 13:07:55 -0700 Subject: [PATCH 02/43] Add examples and documentation for ESP32 ADC driver Adds Erlang and Elixir examples for the ESP32 ADC driver, and documentation to the Programmers Guide. Signed-off-by: Winford --- CHANGELOG.md | 1 + doc/src/programmers-guide.md | 57 +++++++++++++++++++++++ examples/elixir/esp32/Adc_nifs.ex | 45 ++++++++++++++++++ examples/erlang/esp32/adc_example.erl | 40 ++++++++++++++++ examples/erlang/esp32/adc_nif_example.erl | 40 ++++++++++++++++ 5 files changed, 183 insertions(+) create mode 100644 examples/elixir/esp32/Adc_nifs.ex create mode 100644 examples/erlang/esp32/adc_example.erl create mode 100644 examples/erlang/esp32/adc_nif_example.erl diff --git a/CHANGELOG.md b/CHANGELOG.md index 4617f75e1..f6fa97927 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - Support for Elixir `List.Chars` protocol - Support for `gen_server:start_monitor/3,4` - Support for `code:ensure_loaded/1` +- ESP32: add support for `esp_adc` ADC driver, with Erlang and Elixir examples ### Changed diff --git a/doc/src/programmers-guide.md b/doc/src/programmers-guide.md index 9f5b62bfd..0599d3f31 100644 --- a/doc/src/programmers-guide.md +++ b/doc/src/programmers-guide.md @@ -1381,6 +1381,63 @@ Since only one instance of the GPIO driver is allowed, you may also simply use [ ok = gpio:stop(). ``` +### ESP32 ADC + +The [`esp_adc` module](./apidocs/erlang/eavmlib/esp_adc.md) provides the functionality to use the ESP32 family [SAR ADC](https://en.wikipedia.org/wiki/Successive-approximation_ADC) peripheral to measure (analog) voltages from a pin and obtain both raw bit values as well as calibrated voltage values in millivolts. + +The module provides two sets of APIs for using the ADC peripheral; there is a set of low level resource based nifs, and a gen_server managed set of convenience functions. The nifs rely on unit and channel handle resources for configuring and taking measurements. The convenience functions use the gen_server to maintain these resources and use pin numbers to interact with the driver. Examples for both APIs can be found the [AtomVM repository atomvm/examples/erlang/esp32](https://github.com/atomvm/AtomVM/tree/main/examples/erlang/esp32) directory. A demonstration of the simple APIs is as follows: + +```erlang +... + Pin = 33, + ok = esp_adc:start(Pin, [{bitwidth, bit_12}, {atten, db_2_5}]), + {ok, {Raw, Mv}} = esp_adc:read(Pin, [raw, voltage, {samples, 48}]), + io:format("ADC pin ~p raw value=~p millivolts=~p~n", [Pin, Raw, Mv]), + ok = esp_adc:stop(), +... +``` + +#### ESP32 ADC configuration options + +Some newer ESP32 family devices only use a single fixed bit width, this is typically 12 bits, but some provide 13 bit resolution. The ESP32 classic supports 9 bit up to 12 bit resolutions. The `bitwidth` option `bit_max` will use the highest supported resolution for the device. + +The `attenuation` option determines the range of voltage to be measured, the specific voltage range for each setting varies by chip, so as always consult your devices datasheet before connecting an ADC pin to a voltage supply to be measured. The chart below depicts the approximate safe voltage ranges for each attenuation level: + +| Attenuation | Min Millivolts | Max Millivolts | +|------------------|----------------|----------------| +| `db_0` | 0-100 | 750-950 | +| `db_2_5` | 0-100 | 1050-1250 | +| `db_6` | 0-150 | 1300-1750 | +| `db_11 \| db_12` | 0-150 | 2450-2500 | + +Consult the datasheet of your device for the exact voltage ranges supported by each attenuation level. + +```{warning} +The option `db_11` has been superseded by `db_12`. The option `db_11` and will be deprecated in a future release, applications should be updated to use `db_12` (except for builds with ESP-IDF versions prior to v5.2). To Continue to support older IDF version builds, the default will remain `db_11`, which is the maximum tolerated voltage on all builds, as `db_12` supported builds will automatically use `db_12` in place of `db_11`. After `db_11` is deprecated in all builds (with the sunset of ESP-IDF v5.1 support) the default will be changed to `db_12`. +``` + +```{note} +For a higher degree of accuracy increase the number of sample taken, the default is 64. If highly stable and accurate ADC measurements are required for an application you may need to connect a bypass capacitor (e.g., a 100 nF ceramic capacitor) to the ADC input pad in use, to minimize noise. This chart from the [Espressif ADC Calibration Driver documentation](https://docs.espressif.com/projects/esp-idf/en/v5.3/esp32/api-reference/peripherals/adc_calibration.html) shows the difference between the use of a capacitor and without, as well as with a capacitor and multisampling of 64 samples. + +![ADC Noise Comparison](https://docs.espressif.com/projects/esp-idf/en/v5.3/esp32/_images/adc-noise-graph.jpg) + +You can clearly see the noisy results without a capacitor. This is mitigated by the use of multisampling but without a decoupling capacitor results will likely still contain some noise. +``` + +When an ADC channel is configured by the use of `esp_adc:acquire/2,4` or `esp_adc:start/1,2` the driver will select the optimal calibration mechanism supported by the device and channel configuration. If neither the line fitting or curve fitting mechanisms are supported by the device using the provided configuration options an estimated result will be used to provide `voltage` values, based on the [formula suggested by Espressif](https://docs.espressif.com/projects/esp-idf/en/v5.3/esp32/api-reference/peripherals/adc_oneshot.html#read-conversion-result). For chips using the line fitting calibration scheme that do not have the default vref efuse set, a default vref of 1100 mV is used, this is not currently settable. + +#### ESP32 ADC read options + +The read options take the form of a proplist, if the key `raw` is true (`{raw, true}` or simply appears in the list as the atom `raw`), then the raw value will be returned in the first element of the returned tuple. Otherwise, this element will be the atom `undefined`. + +If the key `voltage` is true (or simply appears in the list as an atom), then a calibrated voltage value will be returned in millivolts in the second element of the returned tuple. Otherwise, this element will be the atom `undefined`. + +You may specify the number of samples (1 - 100000) to be taken and averaged over using the tuple `{samples, Samples :: 1..100000}`, the default is `64`. + +```{warning} +Using a large number of samples can significantly increase the amount of time before a response, up to several seconds. +``` + ### I2C The [`i2c` module](./apidocs/erlang/eavmlib/i2c.md) encapsulates functionality associated with the 2-wire Inter-Integrated Circuit (I2C) interface. diff --git a/examples/elixir/esp32/Adc_nifs.ex b/examples/elixir/esp32/Adc_nifs.ex new file mode 100644 index 000000000..243d79e8e --- /dev/null +++ b/examples/elixir/esp32/Adc_nifs.ex @@ -0,0 +1,45 @@ +# +# This file is part of AtomVM. +# +# Copyright 2024 Winford +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later +# + + +defmodule ADCnifs do + # suppress warnings when compiling the VM + @compile {:no_warn_undefined, [Esp.ADC]} + @pin 33 + + def start() do + IO.puts("Testing ADC on pin #{@pin}") + {:ok, unit} = Esp.ADC.init() + {:ok, chan} = Esp.ADC.acquire(@pin, unit, :bit_max, :db_12) + loop(@pin, unit, chan) + end + + defp loop(pin, unit, chan) do + case Esp.ADC.sample(chan, unit, [:raw, :voltage, {:samples, 64}]) do + {:ok, {raw, mv}} -> + IO.puts("Pin #{pin} value = #{raw}, millivolts = #{mv}") + error -> + IO.puts("Error taking ADC sample from pin #{pin}: #{error}") + end + Process.sleep(500) + loop(pin, unit, chan) + end + +end diff --git a/examples/erlang/esp32/adc_example.erl b/examples/erlang/esp32/adc_example.erl new file mode 100644 index 000000000..b4cef5713 --- /dev/null +++ b/examples/erlang/esp32/adc_example.erl @@ -0,0 +1,40 @@ +%% Copyright (c) 2020 dushin.net +%% Copyright (c) 2024 Winford +%% All rights reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +% +% SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later +% + +-module(adc_example). + +-export([start/0]). + +-define(Pin, 34). + +start() -> + io:format("Testing ADC on pin ~p~n", [?Pin]), + ok = esp_adc:start(?Pin), + loop(?Pin). + +loop(Pin) -> + case esp_adc:read(Pin) of + {ok, {Raw, MilliVolts}} -> + io:format("Raw: ~p Voltage: ~pmV~n", [Raw, MilliVolts]); + Error -> + io:format("Error taking reading: ~p~n", [Error]) + end, + timer:sleep(1000), + loop(Pin). diff --git a/examples/erlang/esp32/adc_nif_example.erl b/examples/erlang/esp32/adc_nif_example.erl new file mode 100644 index 000000000..3cedf2c13 --- /dev/null +++ b/examples/erlang/esp32/adc_nif_example.erl @@ -0,0 +1,40 @@ +%% +%% Copyright (c) 2024 Winford +%% All rights reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +% +% SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later +% + +-module(adc_nif_example). + +-export([start/0]). +-define(Pin, 34). + +start() -> + io:format("Testing ADC resource NIFs on pin ~p~n", [?Pin]), + {ok, Unit} = esp_adc:init(), + {ok, Chan} = esp_adc:acquire(?Pin, Unit), + loop(Chan, Unit). + +loop(Chan, Unit) -> + case esp_adc:sample(Chan, Unit) of + {ok, {Raw, MilliVolts}} -> + io:format("Raw: ~p Voltage: ~pmV~n", [Raw, MilliVolts]); + Error -> + io:format("Error taking reading: ~p~n", [Error]) + end, + timer:sleep(1000), + loop(Chan, Unit). From e8af06a42970617eb6952ccc3899fdbe9adf9b94 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Mon, 30 Sep 2024 01:23:08 +0200 Subject: [PATCH 03/43] Add POSIX functions to list directory entries on supported platforms Add `atomvm:posix_opendir/1`, `atomvm:posix_closedir/1` and `atomvm:posix_readdir/1` functions. Signed-off-by: Davide Bettio --- .github/workflows/build-and-test-other.yaml | 4 +- CHANGELOG.md | 2 + libs/eavmlib/src/atomvm.erl | 44 ++++- src/libAtomVM/CMakeLists.txt | 5 +- src/libAtomVM/globalcontext.c | 15 ++ src/libAtomVM/globalcontext.h | 4 + src/libAtomVM/nifs.c | 5 + src/libAtomVM/nifs.gperf | 3 + src/libAtomVM/posix_nifs.c | 187 +++++++++++++++++++- src/libAtomVM/posix_nifs.h | 6 + src/platforms/esp32/CMakeLists.txt | 6 + tests/libs/eavmlib/CMakeLists.txt | 1 + tests/libs/eavmlib/test_dir.erl | 38 ++++ tests/libs/eavmlib/tests.erl | 1 + 14 files changed, 316 insertions(+), 5 deletions(-) create mode 100644 tests/libs/eavmlib/test_dir.erl diff --git a/.github/workflows/build-and-test-other.yaml b/.github/workflows/build-and-test-other.yaml index 49b18c44c..98502a79d 100644 --- a/.github/workflows/build-and-test-other.yaml +++ b/.github/workflows/build-and-test-other.yaml @@ -93,7 +93,9 @@ jobs: - arch: "arm32v7" platform: "arm/v7" tag: "bullseye" - cflags: "-mcpu=cortex-a7 -mfloat-abi=hard -O2 -mthumb -mthumb-interwork" + # -D_FILE_OFFSET_BITS=64 is required for making atomvm:posix_readdir/1 test work + # otherwise readdir will fail due to 64 bits inode numbers with 32 bit ino_t + cflags: "-mcpu=cortex-a7 -mfloat-abi=hard -O2 -mthumb -mthumb-interwork -D_FILE_OFFSET_BITS=64" cmake_opts: "-DAVM_WARNINGS_ARE_ERRORS=ON" - arch: "arm64v8" diff --git a/CHANGELOG.md b/CHANGELOG.md index d0ec7173d..3bafb392f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,8 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - Support for Elixir `IO.chardata_to_string/1` - Support for Elixir `List.duplicate/2` - Support for `binary:copy/1,2` +- Support for directory listing using POSIX APIs: (`atomvm:posix_opendir/1`, +`atomvm:posix_readdir/1`, `atomvm:posix_closedir/1`). ### Changed diff --git a/libs/eavmlib/src/atomvm.erl b/libs/eavmlib/src/atomvm.erl index e9cd15e1c..a5dd82645 100644 --- a/libs/eavmlib/src/atomvm.erl +++ b/libs/eavmlib/src/atomvm.erl @@ -40,12 +40,16 @@ posix_close/1, posix_read/2, posix_write/2, - posix_clock_settime/2 + posix_clock_settime/2, + posix_opendir/1, + posix_closedir/1, + posix_readdir/1 ]). -export_type([ posix_fd/0, - posix_open_flag/0 + posix_open_flag/0, + posix_dir/0 ]). -deprecated([ @@ -84,6 +88,8 @@ atom() | integer(). +-opaque posix_dir() :: binary(). + %%----------------------------------------------------------------------------- %% @returns The platform name. %% @doc Return the platform moniker. @@ -295,3 +301,37 @@ posix_write(_File, _Data) -> ok | {error, Reason :: posix_error()}. posix_clock_settime(_ClockId, _Time) -> erlang:nif_error(undefined). + +%%----------------------------------------------------------------------------- +%% @param Path Path to the directory to open +%% @returns A tuple with a directory descriptor or an error tuple. +%% @doc Open a file (on platforms that have `opendir(3)'). +%% @end +%%----------------------------------------------------------------------------- +-spec posix_opendir(Path :: iodata()) -> + {ok, posix_dir()} | {error, posix_error()}. +posix_opendir(_Path) -> + erlang:nif_error(undefined). + +%%----------------------------------------------------------------------------- +%% @param Dir Descriptor to a directory to close +%% @returns `ok' or an error tuple +%% @doc Close a directory that was opened with `posix_opendir/1' +%% @end +%%----------------------------------------------------------------------------- +-spec posix_closedir(Dir :: posix_dir()) -> ok | {error, posix_error()}. +posix_closedir(_Dir) -> + erlang:nif_error(undefined). + +%%----------------------------------------------------------------------------- +%% @param Dir Descriptor to an open directory +%% @returns a `{dirent, InodeNo, Name}' tuple, `eof' or an error tuple +%% @doc Read a directory entry +%% `eof' is returned if no more data can be read because the directory cursor +%% reached the end. +%% @end +%%----------------------------------------------------------------------------- +-spec posix_readdir(Dir :: posix_dir()) -> + {ok, {dirent, Inode :: integer(), Name :: binary()}} | eof | {error, posix_error()}. +posix_readdir(_Dir) -> + erlang:nif_error(undefined). diff --git a/src/libAtomVM/CMakeLists.txt b/src/libAtomVM/CMakeLists.txt index c4273607e..d96503a04 100644 --- a/src/libAtomVM/CMakeLists.txt +++ b/src/libAtomVM/CMakeLists.txt @@ -187,10 +187,13 @@ if (NOT ATOMIC_POINTER_LOCK_FREE_IS_TWO AND NOT (HAVE_PLATFORM_ATOMIC_H OR (AVM_ endif() include(DefineIfExists) -# HAVE_OPEN & HAVE_CLOSE are used in globalcontext.h +# HAVE_OPEN, HAVE_OPENDIR, HAVE_CLOSE HAVE_CLOSEDIR, HAVE_READDIR are used in globalcontext.h define_if_function_exists(libAtomVM open "fcntl.h" PUBLIC HAVE_OPEN) +define_if_function_exists(libAtomVM opendir "dirent.h" PUBLIC HAVE_OPENDIR) define_if_function_exists(libAtomVM close "unistd.h" PUBLIC HAVE_CLOSE) +define_if_function_exists(libAtomVM closedir "dirent.h" PUBLIC HAVE_CLOSEDIR) define_if_function_exists(libAtomVM mkfifo "sys/stat.h" PRIVATE HAVE_MKFIFO) +define_if_function_exists(libAtomVM readdir "dirent.h" PUBLIC HAVE_READDIR) define_if_function_exists(libAtomVM unlink "unistd.h" PRIVATE HAVE_UNLINK) define_if_symbol_exists(libAtomVM O_CLOEXEC "fcntl.h" PRIVATE HAVE_O_CLOEXEC) define_if_symbol_exists(libAtomVM O_DIRECTORY "fcntl.h" PRIVATE HAVE_O_DIRECTORY) diff --git a/src/libAtomVM/globalcontext.c b/src/libAtomVM/globalcontext.c index fc28c4ead..54a8d291e 100644 --- a/src/libAtomVM/globalcontext.c +++ b/src/libAtomVM/globalcontext.c @@ -135,6 +135,21 @@ GlobalContext *globalcontext_new() } #endif +#if HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR + ErlNifEnv dir_env; + erl_nif_env_partial_init_from_globalcontext(&dir_env, glb); + glb->posix_dir_resource_type = enif_init_resource_type(&env, "posix_dir", &posix_dir_resource_type_init, ERL_NIF_RT_CREATE, NULL); + if (IS_NULL_PTR(glb->posix_dir_resource_type)) { +#ifndef AVM_NO_SMP + smp_rwlock_destroy(glb->modules_lock); +#endif + free(glb->modules_table); + atom_table_destroy(glb->atom_table); + free(glb); + return NULL; + } +#endif + sys_init_platform(glb); #ifndef AVM_NO_SMP diff --git a/src/libAtomVM/globalcontext.h b/src/libAtomVM/globalcontext.h index 605600618..71f88b633 100644 --- a/src/libAtomVM/globalcontext.h +++ b/src/libAtomVM/globalcontext.h @@ -152,6 +152,10 @@ struct GlobalContext ErlNifResourceType *posix_fd_resource_type; #endif +#if HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR + ErlNifResourceType *posix_dir_resource_type; +#endif + void *platform_data; }; diff --git a/src/libAtomVM/nifs.c b/src/libAtomVM/nifs.c index 03daba1a7..798ba07b7 100644 --- a/src/libAtomVM/nifs.c +++ b/src/libAtomVM/nifs.c @@ -794,6 +794,11 @@ DEFINE_MATH_NIF(tanh) #else #define IF_HAVE_CLOCK_SETTIME_OR_SETTIMEOFDAY(expr) NULL #endif +#if HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR +#define IF_HAVE_OPENDIR_READDIR_CLOSEDIR(expr) (expr) +#else +#define IF_HAVE_OPENDIR_READDIR_CLOSEDIR(expr) NULL +#endif //Ignore warning caused by gperf generated code #pragma GCC diagnostic push diff --git a/src/libAtomVM/nifs.gperf b/src/libAtomVM/nifs.gperf index f23644630..e3a9e8246 100644 --- a/src/libAtomVM/nifs.gperf +++ b/src/libAtomVM/nifs.gperf @@ -141,6 +141,9 @@ atomvm:posix_select_stop/1, IF_HAVE_OPEN_CLOSE(&atomvm_posix_select_stop_nif) atomvm:posix_mkfifo/2, IF_HAVE_MKFIFO(&atomvm_posix_mkfifo_nif) atomvm:posix_unlink/1, IF_HAVE_UNLINK(&atomvm_posix_unlink_nif) atomvm:posix_clock_settime/2, IF_HAVE_CLOCK_SETTIME_OR_SETTIMEOFDAY(&atomvm_posix_clock_settime_nif) +atomvm:posix_opendir/1, IF_HAVE_OPENDIR_READDIR_CLOSEDIR(&atomvm_posix_opendir_nif) +atomvm:posix_closedir/1, IF_HAVE_OPENDIR_READDIR_CLOSEDIR(&atomvm_posix_closedir_nif) +atomvm:posix_readdir/1, IF_HAVE_OPENDIR_READDIR_CLOSEDIR(&atomvm_posix_readdir_nif) code:load_abs/1, &code_load_abs_nif code:load_binary/3, &code_load_binary_nif code:ensure_loaded/1, &code_ensure_loaded_nif diff --git a/src/libAtomVM/posix_nifs.c b/src/libAtomVM/posix_nifs.c index 07881e0ff..ceeb9f854 100644 --- a/src/libAtomVM/posix_nifs.c +++ b/src/libAtomVM/posix_nifs.c @@ -38,10 +38,15 @@ #include #endif -#if HAVE_OPEN && HAVE_CLOSE || defined(HAVE_CLOCK_SETTIME) || defined(HAVE_SETTIMEOFDAY) +#if HAVE_OPEN && HAVE_CLOSE || defined(HAVE_CLOCK_SETTIME) || defined(HAVE_SETTIMEOFDAY) \ + || HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR #include #endif +#if HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR +#include +#endif + #include "defaultatoms.h" #include "erl_nif_priv.h" #include "globalcontext.h" @@ -602,6 +607,172 @@ static term nif_atomvm_posix_clock_settime(Context *ctx, int argc, term argv[]) } #endif +#if HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR +struct PosixDir +{ + DIR *dir; +}; + +static void posix_dir_dtor(ErlNifEnv *caller_env, void *obj) +{ + UNUSED(caller_env); + + struct PosixDir *dir_obj = (struct PosixDir *) obj; + if (dir_obj->dir) { + closedir(dir_obj->dir); + dir_obj->dir = NULL; + } +} + +const ErlNifResourceTypeInit posix_dir_resource_type_init = { + .members = 1, + .dtor = posix_dir_dtor +}; + +static term errno_to_error_tuple_maybe_gc(Context *ctx) +{ + if (UNLIKELY(memory_ensure_free_opt(ctx, TUPLE_SIZE(2), MEMORY_CAN_SHRINK) != MEMORY_GC_OK)) { + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + + term result = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(result, 0, ERROR_ATOM); + term_put_tuple_element(result, 1, posix_errno_to_term(errno, ctx->global)); + + return result; +} + +static term nif_atomvm_posix_opendir(Context *ctx, int argc, term argv[]) +{ + UNUSED(argc); + + GlobalContext *glb = ctx->global; + + term path_term = argv[0]; + + int ok; + char *path = interop_term_to_string(path_term, &ok); + if (UNLIKELY(!ok)) { + RAISE_ERROR(BADARG_ATOM); + } + + term result; + DIR *dir = opendir(path); + free(path); + + if (IS_NULL_PTR(dir)) { + return errno_to_error_tuple_maybe_gc(ctx); + } else { + // Return a resource object + struct PosixDir *dir_obj + = enif_alloc_resource(glb->posix_dir_resource_type, sizeof(struct PosixDir)); + if (IS_NULL_PTR(dir_obj)) { + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + dir_obj->dir = dir; + if (UNLIKELY(memory_ensure_free_opt( + ctx, TUPLE_SIZE(2) + TERM_BOXED_RESOURCE_SIZE, MEMORY_CAN_SHRINK) + != MEMORY_GC_OK)) { + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + term obj = term_from_resource(dir_obj, &ctx->heap); + result = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(result, 0, OK_ATOM); + term_put_tuple_element(result, 1, obj); + } + + return result; +} + +static term nif_atomvm_posix_closedir(Context *ctx, int argc, term argv[]) +{ + UNUSED(argc); + + term result = OK_ATOM; + + void *dir_obj_ptr; + if (UNLIKELY(!enif_get_resource(erl_nif_env_from_context(ctx), argv[0], + ctx->global->posix_dir_resource_type, &dir_obj_ptr))) { + RAISE_ERROR(BADARG_ATOM); + } + struct PosixDir *dir_obj = (struct PosixDir *) dir_obj_ptr; + if (dir_obj->dir != NULL) { + if (UNLIKELY(closedir(dir_obj->dir) < 0)) { + dir_obj->dir = NULL; // even if bad things happen, do not close twice. + return errno_to_error_tuple_maybe_gc(ctx); + } + dir_obj->dir = NULL; + } + + return result; +} + +// This function main purpose is to avoid warnings, such as: +// warning: comparison is always true due to limited range of data type [-Wtype-limits] +static inline term to_boxed_safe(uint64_t value, Context *ctx) +{ + if (value <= INT64_MAX) { + return term_make_maybe_boxed_int64(value, &ctx->heap); + } else { + return UNDEFINED_ATOM; + } +} + +static term nif_atomvm_posix_readdir(Context *ctx, int argc, term argv[]) +{ + UNUSED(argc); + + GlobalContext *glb = ctx->global; + + void *dir_obj_ptr; + if (UNLIKELY(!enif_get_resource( + erl_nif_env_from_context(ctx), argv[0], glb->posix_dir_resource_type, &dir_obj_ptr))) { + RAISE_ERROR(BADARG_ATOM); + } + struct PosixDir *dir_obj = (struct PosixDir *) dir_obj_ptr; + + errno = 0; + struct dirent *dir_result = readdir(dir_obj->dir); + if (dir_result == NULL) { + if (UNLIKELY(errno != 0)) { + return errno_to_error_tuple_maybe_gc(ctx); + } + + return globalcontext_make_atom(glb, ATOM_STR("\x3", "eof")); + } + + size_t name_len = strlen(dir_result->d_name); + if (UNLIKELY( + memory_ensure_free_opt(ctx, + BOXED_INT64_SIZE + term_binary_heap_size(name_len) + TUPLE_SIZE(3) + TUPLE_SIZE(2), + MEMORY_CAN_SHRINK) + != MEMORY_GC_OK)) { + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + + term ino_no = to_boxed_safe(dir_result->d_ino, ctx); + + term name_term = term_create_uninitialized_binary(name_len, &ctx->heap, glb); + memcpy((void *) term_binary_data(name_term), dir_result->d_name, name_len); + + term dirent_atom = globalcontext_make_atom(glb, ATOM_STR("\x6", "dirent")); + + // {dirent, Inode, Name} + term dirent_term = term_alloc_tuple(3, &ctx->heap); + term_put_tuple_element(dirent_term, 0, dirent_atom); + term_put_tuple_element(dirent_term, 1, ino_no); + term_put_tuple_element(dirent_term, 2, name_term); + + // {ok, DirentTuple} + term result = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(result, 0, OK_ATOM); + term_put_tuple_element(result, 1, dirent_term); + + return result; +} + +#endif + #if HAVE_OPEN && HAVE_CLOSE const struct Nif atomvm_posix_open_nif = { .base.type = NIFFunctionType, @@ -650,3 +821,17 @@ const struct Nif atomvm_posix_clock_settime_nif = { .nif_ptr = nif_atomvm_posix_clock_settime }; #endif +#if HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR +const struct Nif atomvm_posix_opendir_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_atomvm_posix_opendir +}; +const struct Nif atomvm_posix_closedir_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_atomvm_posix_closedir +}; +const struct Nif atomvm_posix_readdir_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_atomvm_posix_readdir +}; +#endif diff --git a/src/libAtomVM/posix_nifs.h b/src/libAtomVM/posix_nifs.h index 061a155c8..4b425529e 100644 --- a/src/libAtomVM/posix_nifs.h +++ b/src/libAtomVM/posix_nifs.h @@ -53,6 +53,12 @@ extern const struct Nif atomvm_posix_unlink_nif; #if defined(HAVE_CLOCK_SETTIME) || defined(HAVE_SETTIMEOFDAY) extern const struct Nif atomvm_posix_clock_settime_nif; #endif +#if HAVE_OPENDIR && HAVE_READDIR && HAVE_CLOSEDIR +extern const ErlNifResourceTypeInit posix_dir_resource_type_init; +extern const struct Nif atomvm_posix_opendir_nif; +extern const struct Nif atomvm_posix_readdir_nif; +extern const struct Nif atomvm_posix_closedir_nif; +#endif /** * @brief Convenient function to return posix errors as atom. diff --git a/src/platforms/esp32/CMakeLists.txt b/src/platforms/esp32/CMakeLists.txt index 55407bc33..08fbfabd6 100644 --- a/src/platforms/esp32/CMakeLists.txt +++ b/src/platforms/esp32/CMakeLists.txt @@ -31,6 +31,12 @@ set(HAVE_MKFIFO "" CACHE INTERNAL "Have symbol mkfifo" FORCE) # in CMAKE_REQUIRED_INCLUDES as lwip includes freetos and many esp system components set(HAVE_SOCKET 1 CACHE INTERNAL "Have symbol socket" FORCE) +# opendir, closedir and readdir functions are not detected +# but they are available +set(HAVE_OPENDIR 1 CACHE INTERNAL "Have symbol opendir" FORCE) +set(HAVE_CLOSEDIR 1 CACHE INTERNAL "Have symbol closedir" FORCE) +set(HAVE_READDIR 1 CACHE INTERNAL "Have symbol readdir" FORCE) + list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../CMakeModules") # Disable SMP with esp32 socs that have only one core diff --git a/tests/libs/eavmlib/CMakeLists.txt b/tests/libs/eavmlib/CMakeLists.txt index df2f30129..758764111 100644 --- a/tests/libs/eavmlib/CMakeLists.txt +++ b/tests/libs/eavmlib/CMakeLists.txt @@ -23,6 +23,7 @@ project(test_eavmlib) include(BuildErlang) set(ERLANG_MODULES + test_dir test_file test_ahttp_client test_port diff --git a/tests/libs/eavmlib/test_dir.erl b/tests/libs/eavmlib/test_dir.erl new file mode 100644 index 000000000..40e66126e --- /dev/null +++ b/tests/libs/eavmlib/test_dir.erl @@ -0,0 +1,38 @@ +% +% This file is part of AtomVM. +% +% Copyright 2024 Davide Bettio +% +% Licensed under the Apache License, Version 2.0 (the "License"); +% you may not use this file except in compliance with the License. +% You may obtain a copy of the License at +% +% http://www.apache.org/licenses/LICENSE-2.0 +% +% Unless required by applicable law or agreed to in writing, software +% distributed under the License is distributed on an "AS IS" BASIS, +% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +% See the License for the specific language governing permissions and +% limitations under the License. +% +% SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later +% + +-module(test_dir). + +-export([test/0]). + +-include("etest.hrl"). + +test() -> + {ok, Dir} = atomvm:posix_opendir("."), + [eof | _Entries] = all_dir_entries(Dir, []), + ok = atomvm:posix_closedir(Dir). + +all_dir_entries(Dir, Acc) -> + case atomvm:posix_readdir(Dir) of + eof -> + [eof | Acc]; + {ok, {dirent, Inode, Name} = Dirent} when is_integer(Inode) and is_binary(Name) -> + all_dir_entries(Dir, [Dirent | Acc]) + end. diff --git a/tests/libs/eavmlib/tests.erl b/tests/libs/eavmlib/tests.erl index 914cd8bde..c8ceac567 100644 --- a/tests/libs/eavmlib/tests.erl +++ b/tests/libs/eavmlib/tests.erl @@ -24,6 +24,7 @@ start() -> etest:test([ + test_dir, test_file, test_port, test_timer_manager, From 1984f76eb8301d21a9d051ac55e7eca4d18c9f7b Mon Sep 17 00:00:00 2001 From: Winford Date: Tue, 30 Apr 2024 19:26:45 -0700 Subject: [PATCH 04/43] Fix several possible double free() in ESP32 network_driver.c Removes several possible uses of free() on memory that had been previously free'd. This would happen under specific error conditions in the `start_network` function in network_driver.c that `network:start/1` uses to configure and start the network. Signed-off-by: Winford --- CHANGELOG.md | 2 ++ .../esp32/components/avm_builtins/network_driver.c | 7 ++++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4884eedb..ff4d071f4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -55,6 +55,8 @@ instead - `unicode:characters_to_list`: fixed bogus out_of_memory error on some platforms such as ESP32 - Fix crash in Elixir library when doing `inspect(:atom)` - General inspect() compliance with Elixir behavior (but there are still some minor differences) +- Fix several uses of free on prevously released memory on ESP32, under certain error condition using +`network:start/1`, that would lead to a hard crash of the VM. ## [0.6.4] - 2024-08-18 diff --git a/src/platforms/esp32/components/avm_builtins/network_driver.c b/src/platforms/esp32/components/avm_builtins/network_driver.c index 6252e6196..78a33449e 100644 --- a/src/platforms/esp32/components/avm_builtins/network_driver.c +++ b/src/platforms/esp32/components/avm_builtins/network_driver.c @@ -744,7 +744,6 @@ static void start_network(Context *ctx, term pid, term ref, term config) if ((err = esp_wifi_set_config(ESP_IF_WIFI_AP, ap_wifi_config)) != ESP_OK) { ESP_LOGE(TAG, "Error setting AP mode config %d", err); free(ap_wifi_config); - free(sta_wifi_config); term error = port_create_error_tuple(ctx, term_from_int(err)); port_send_reply(ctx, pid, ref, error); return; @@ -753,12 +752,14 @@ static void start_network(Context *ctx, term pid, term ref, term config) free(ap_wifi_config); } } + + // + // Start the configured interface(s) + // if ((err = esp_wifi_start()) != ESP_OK) { ESP_LOGE(TAG, "Error in esp_wifi_start %d", err); term error = port_create_error_tuple(ctx, term_from_int(err)); port_send_reply(ctx, pid, ref, error); - free(ap_wifi_config); - free(sta_wifi_config); return; } else { ESP_LOGI(TAG, "WIFI started"); From ad3c796845aaa0114af55f65da3c9cfd5a9c942b Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 21 Apr 2024 15:54:40 -0700 Subject: [PATCH 05/43] Completly stop driver and free all resources with network:stop/0 on ESP32 Adds a destroy callback to the ESP32 network driver to completely stop the driver and free all network resources when network:stop/0 is used. Previosly the driver was not being stopped internally and resources were not freed when the gen_server was stopped, causing instability, and possible crashes when event callbacks were triggered, but there was no process alive to handle them. Closes #643 Signed-off-by: Winford --- CHANGELOG.md | 2 + libs/eavmlib/src/network.erl | 2 + .../components/avm_builtins/network_driver.c | 51 +++++++++++++++++-- 3 files changed, 52 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ff4d071f4..b79865094 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,8 @@ instead - General inspect() compliance with Elixir behavior (but there are still some minor differences) - Fix several uses of free on prevously released memory on ESP32, under certain error condition using `network:start/1`, that would lead to a hard crash of the VM. +- Fix a bug in ESP32 network driver where the low level driver was not being stopped and resoureces were not freed +when `network:stop/0` was used, see issue [#643](https://github.com/atomvm/AtomVM/issues/643) ## [0.6.4] - 2024-08-18 diff --git a/libs/eavmlib/src/network.erl b/libs/eavmlib/src/network.erl index 9943e12f2..cb4f3961f 100644 --- a/libs/eavmlib/src/network.erl +++ b/libs/eavmlib/src/network.erl @@ -386,6 +386,8 @@ handle_info(Msg, State) -> %% @hidden terminate(_Reason, _State) -> + Ref = make_ref(), + network_port ! {?SERVER, Ref, stop}, ok. %% diff --git a/src/platforms/esp32/components/avm_builtins/network_driver.c b/src/platforms/esp32/components/avm_builtins/network_driver.c index 78a33449e..93ba60bad 100644 --- a/src/platforms/esp32/components/avm_builtins/network_driver.c +++ b/src/platforms/esp32/components/avm_builtins/network_driver.c @@ -92,12 +92,14 @@ enum network_cmd NetworkInvalidCmd = 0, // TODO add support for scan, ifconfig NetworkStartCmd, - NetworkRssiCmd + NetworkRssiCmd, + NetworkStopCmd }; static const AtomStringIntPair cmd_table[] = { { ATOM_STR("\x5", "start"), NetworkStartCmd }, { ATOM_STR("\x4", "rssi"), NetworkRssiCmd }, + { ATOM_STR("\x4", "stop"), NetworkStopCmd }, SELECT_INT_DEFAULT(NetworkInvalidCmd) }; @@ -779,12 +781,51 @@ static void start_network(Context *ctx, term pid, term ref, term config) if (!IS_NULL_PTR(ap_wifi_config)) { set_dhcp_hostname(ap_wifi_interface, "AP", interop_kv_get_value(ap_config, dhcp_hostname_atom, ctx->global)); } + // // Done -- send an ok so the FSM can proceed // port_send_reply(ctx, pid, ref, OK_ATOM); } +static void stop_network(Context *ctx) +{ + // Stop unregister event callbacks so they dont trigger during shutdown. + esp_event_handler_unregister(WIFI_EVENT, ESP_EVENT_ANY_ID, &event_handler); + + esp_netif_t *sta_wifi_interface = esp_netif_get_handle_from_ifkey("WIFI_STA_DEF"); + esp_netif_t *ap_wifi_interface = esp_netif_get_handle_from_ifkey("WIFI_AP_DEF"); + + // Disconnect STA if connected to access point + if ((sta_wifi_interface != NULL) && (esp_netif_is_netif_up(sta_wifi_interface))) { + esp_err_t err = esp_wifi_disconnect(); + if (UNLIKELY(err == ESP_FAIL)) { + ESP_LOGE(TAG, "ESP FAIL error while disconnecting from AP, continuing network shutdown..."); + } + } + + // Stop and deinit the WiFi driver, these only return OK, or not init error (fine to ignore). + esp_wifi_stop(); + esp_wifi_deinit(); + + // Stop sntp (ignore OK, or not configured error) + esp_sntp_stop(); + + // Delete network event loop + esp_err_t err = esp_event_loop_delete_default(); + if (err != ESP_OK) { + ESP_LOGE(TAG, "Invalid state error while deleting event loop, continuing network shutdown..."); + } + + // Destroy existing netif interfaces + if (ap_wifi_interface != NULL) { + esp_netif_destroy_default_wifi(ap_wifi_interface); + } + if (sta_wifi_interface != NULL) { + esp_netif_destroy_default_wifi(sta_wifi_interface); + } +} + static void get_sta_rssi(Context *ctx, term pid, term ref) { size_t tuple_reply_size = PORT_REPLY_SIZE + TUPLE_SIZE(2); @@ -805,11 +846,11 @@ static void get_sta_rssi(Context *ctx, term pid, term ref) port_ensure_available(ctx, tuple_reply_size); term reply = port_create_tuple2(ctx, make_atom(ctx->global, ATOM_STR("\x4", "rssi")), rssi); port_send_reply(ctx, pid, ref, reply); - } static NativeHandlerResult consume_mailbox(Context *ctx) { + bool cmd_terminate = false; Message *message = mailbox_first(&ctx->mailbox); term msg = message->message; @@ -842,6 +883,10 @@ static NativeHandlerResult consume_mailbox(Context *ctx) case NetworkRssiCmd: get_sta_rssi(ctx, pid, ref); break; + case NetworkStopCmd: + cmd_terminate = true; + stop_network(ctx); + break; default: { ESP_LOGE(TAG, "Unrecognized command: %x", cmd); @@ -868,7 +913,7 @@ static NativeHandlerResult consume_mailbox(Context *ctx) mailbox_remove_message(&ctx->mailbox, &ctx->heap); - return NativeContinue; + return cmd_terminate ? NativeTerminate : NativeContinue; } // From ec7602c6c440eb1a799ae02bcee6d384b4f6347a Mon Sep 17 00:00:00 2001 From: Winford Date: Fri, 19 Apr 2024 19:02:41 -0700 Subject: [PATCH 06/43] Add ESP32 beacon timeout event handler Adds an event handler for `event 21` the `WIFI_EVENT_STA_BEACON_TIMEOUT` event and an option to add an Erlang callback handler for the event. The event will be logged with an info level message that includes a suggestion about the two most likely causes, poor rssi and network congestion. A callback config option `{beacon_timeout, fun()}` may be added to the `sta` config. Closes #1100 Signed-off-by: Winford --- CHANGELOG.md | 2 ++ doc/src/network-programming-guide.md | 1 + libs/eavmlib/src/network.erl | 9 +++++++++ .../components/avm_builtins/network_driver.c | 19 +++++++++++++++++++ 4 files changed, 31 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b79865094..8b85864c5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,8 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - Support for directory listing using POSIX APIs: (`atomvm:posix_opendir/1`, `atomvm:posix_readdir/1`, `atomvm:posix_closedir/1`). - ESP32: add support for `esp_adc` ADC driver, with Erlang and Elixir examples +- Add handler for ESP32 network driver STA mode `beacon_timeout` (event: 21), see issue +[#1100](https://github.com/atomvm/AtomVM/issues/1100) ### Changed diff --git a/doc/src/network-programming-guide.md b/doc/src/network-programming-guide.md index 55912489f..c46346b46 100644 --- a/doc/src/network-programming-guide.md +++ b/doc/src/network-programming-guide.md @@ -46,6 +46,7 @@ Callback functions are optional, but are highly recommended for building robust In addition, the following optional parameters can be specified to configure the AP network (ESP32 only): * `{dhcp_hostname, string()|binary()}` The DHCP hostname as which the device should register (`<<"atomvm-">>`, where `` is the hexadecimal representation of the factory-assigned MAC address of the device). +* `{beacon_timeout, fun(() -> term())}` A callback function which will be called when the device does not receive a beacon frame from the connected access point during the "inactive time" (6 second default, currently not configurable). The following example illustrates initialization of the WiFi network in STA mode. The example program will configure the network to connect to a specified network. Events that occur during the lifecycle of the network will trigger invocations of the specified callback functions. diff --git a/libs/eavmlib/src/network.erl b/libs/eavmlib/src/network.erl index cb4f3961f..3f83ca82c 100644 --- a/libs/eavmlib/src/network.erl +++ b/libs/eavmlib/src/network.erl @@ -50,6 +50,7 @@ -type dhcp_hostname_config() :: {dhcp_hostname, string() | binary()}. -type sta_connected_config() :: {connected, fun(() -> term())}. +-type sta_beacon_timeout_config() :: {beacon_timeout, fun(() -> term())}. -type sta_disconnected_config() :: {disconnected, fun(() -> term())}. -type sta_got_ip_config() :: {got_ip, fun((ip_info()) -> term())}. -type sta_config_property() :: @@ -57,6 +58,7 @@ | psk_config() | dhcp_hostname_config() | sta_connected_config() + | sta_beacon_timeout_config() | sta_disconnected_config() | sta_got_ip_config(). -type sta_config() :: {sta, [sta_config_property()]}. @@ -357,6 +359,9 @@ handle_cast(_Msg, State) -> handle_info({Ref, sta_connected} = _Msg, #state{ref = Ref, config = Config} = State) -> maybe_sta_connected_callback(Config), {noreply, State}; +handle_info({Ref, sta_beacon_timeout} = _Msg, #state{ref = Ref, config = Config} = State) -> + maybe_sta_beacon_timeout_callback(Config), + {noreply, State}; handle_info({Ref, sta_disconnected} = _Msg, #state{ref = Ref, config = Config} = State) -> maybe_sta_disconnected_callback(Config), {noreply, State}; @@ -398,6 +403,10 @@ terminate(_Reason, _State) -> maybe_sta_connected_callback(Config) -> maybe_callback0(connected, proplists:get_value(sta, Config)). +%% @private +maybe_sta_beacon_timeout_callback(Config) -> + maybe_callback0(beacon_timeout, proplists:get_value(sta, Config)). + %% @private maybe_sta_disconnected_callback(Config) -> maybe_callback0(disconnected, proplists:get_value(sta, Config)). diff --git a/src/platforms/esp32/components/avm_builtins/network_driver.c b/src/platforms/esp32/components/avm_builtins/network_driver.c index 93ba60bad..46ee0dc48 100644 --- a/src/platforms/esp32/components/avm_builtins/network_driver.c +++ b/src/platforms/esp32/components/avm_builtins/network_driver.c @@ -76,6 +76,7 @@ static const char *const ssid_atom = ATOM_STR("\x4", "ssid"); static const char *const ssid_hidden_atom = ATOM_STR("\xB", "ssid_hidden"); static const char *const sta_atom = ATOM_STR("\x3", "sta"); static const char *const sta_connected_atom = ATOM_STR("\xD", "sta_connected"); +static const char *const sta_beacon_timeout_atom = ATOM_STR("\x12", "sta_beacon_timeout"); static const char *const sta_disconnected_atom = ATOM_STR("\x10", "sta_disconnected"); static const char *const sta_got_ip_atom = ATOM_STR("\xA", "sta_got_ip"); @@ -168,6 +169,18 @@ static void send_sta_connected(struct ClientData *data) END_WITH_STACK_HEAP(heap, data->global); } +static void send_sta_beacon_timeout(struct ClientData *data) +{ + TRACE("Sending sta_beacon_timeout back to AtomVM\n"); + + // {Ref, sta_beacon_timeout} + BEGIN_WITH_STACK_HEAP(PORT_REPLY_SIZE, heap); + { + send_term(&heap, data, make_atom(data->global, sta_beacon_timeout_atom)); + } + END_WITH_STACK_HEAP(heap, data->global); +} + static void send_sta_disconnected(struct ClientData *data) { TRACE("Sending sta_disconnected back to AtomVM\n"); @@ -304,6 +317,12 @@ static void event_handler(void *arg, esp_event_base_t event_base, int32_t event_ } #endif + case WIFI_EVENT_STA_BEACON_TIMEOUT: { + ESP_LOGI(TAG, "WIFI_EVENT_STA_BEACON_TIMEOUT received. Maybe poor signal, or network congestion?"); + send_sta_beacon_timeout(data); + break; + } + default: ESP_LOGI(TAG, "Unhandled wifi event: %" PRIi32 ".", event_id); break; From f3aedceef694aa348101881b5e4c0e9110ea4a20 Mon Sep 17 00:00:00 2001 From: Winford Date: Thu, 3 Oct 2024 19:12:45 -0700 Subject: [PATCH 07/43] Fix broken docs workflows `edown` is only tested up to OTP-25. It seems to work fine with OTP-26, but fails to parse many of our modules using OTP-27. This change reverts to OTP-26 until an update or alternative makes it possible to build with OTP-27. Signed-off-by: Winford --- .github/workflows/build-docs.yaml | 4 ++-- .github/workflows/publish-docs.yaml | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build-docs.yaml b/.github/workflows/build-docs.yaml index ecaf3e5ff..48eecc957 100644 --- a/.github/workflows/build-docs.yaml +++ b/.github/workflows/build-docs.yaml @@ -45,8 +45,8 @@ jobs: # This workflow contains a single job called "build" build: # The type of runner that the job will run on - runs-on: ubuntu-latest - container: erlang:27 + runs-on: ubuntu-24.04 + container: erlang:26 # Steps represent a sequence of tasks that will be executed as part of the job steps: diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index cde1c5242..7ad443d2e 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -36,7 +36,7 @@ jobs: # This workflow contains a single job called "build" build: # The type of runner that the job will run on - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 env: AVM_DOCS_NAME: ${{ github.ref_name }} @@ -72,7 +72,7 @@ jobs: - uses: erlef/setup-beam@v1 with: - otp-version: "27" + otp-version: "26" elixir-version: "1.17" hexpm-mirrors: | https://builds.hex.pm From e80174ef672fa3c414344448a1f3c494d1a5c6cc Mon Sep 17 00:00:00 2001 From: Winford Date: Fri, 4 Oct 2024 08:16:41 -0700 Subject: [PATCH 08/43] Fix documentation workflow caches The Doxygen install cache needs to be kept in sync with the runner image used for the workflow. This will prevent future updates to the runner image from using an out of sync Doygen install cache. Signed-off-by: Winford --- .github/workflows/build-docs.yaml | 13 ++++++++++--- .github/workflows/publish-docs.yaml | 14 +++++++++----- 2 files changed, 19 insertions(+), 8 deletions(-) diff --git a/.github/workflows/build-docs.yaml b/.github/workflows/build-docs.yaml index 48eecc957..499eaa72c 100644 --- a/.github/workflows/build-docs.yaml +++ b/.github/workflows/build-docs.yaml @@ -44,14 +44,20 @@ concurrency: jobs: # This workflow contains a single job called "build" build: + + strategy: + fail-fast: false + ## don't add more than one OS to matrix, this is only to retrieve the full os-name for keeping cache in sync + matrix: + os: [ ubuntu-24.04 ] # The type of runner that the job will run on - runs-on: ubuntu-24.04 + runs-on: ${{ matrix.os }} + # Documentation currently fails to build with OTP-27 container: erlang:26 # Steps represent a sequence of tasks that will be executed as part of the job steps: # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it - - name: Install Deps run: | apt update -y @@ -61,7 +67,7 @@ jobs: id: sphinx-cache with: path: /home/runner/python-env/sphinx - key: ${{ runner.os }}-sphinx-install + key: ${{ matrix.os }}-${{ job.container.id }}-sphinx-install - name: Install Sphinx if: ${{ steps.sphinx-cache.outputs.cache-hit != 'true' }} @@ -100,6 +106,7 @@ jobs: done - name: Build Site + id: build shell: bash run: | . /home/runner/python-env/sphinx/bin/activate diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index 7ad443d2e..e64a7293a 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -31,15 +31,19 @@ concurrency: group: ${{ github.workflow }}-${{ github.ref_name }} cancel-in-progress: true +env: + AVM_DOCS_NAME: ${{ github.ref_name }} + # A workflow run is made up of one or more jobs that can run sequentially or in parallel jobs: # This workflow contains a single job called "build" build: + strategy: + ## don't add more than one OS to matrix, this is only to retrieve the full os-name for keeping cache in sync + matrix: + os: [ ubuntu-24.04 ] # The type of runner that the job will run on - runs-on: ubuntu-24.04 - - env: - AVM_DOCS_NAME: ${{ github.ref_name }} + runs-on: ${{ matrix.os }} # Steps represent a sequence of tasks that will be executed as part of the job steps: @@ -54,7 +58,7 @@ jobs: id: sphinx-cache with: path: /home/runner/python-env/sphinx - key: ${{ runner.os }}-sphinx-install + key: ${{ matrix.os }}-sphinx-install - name: Install Sphinx if: ${{ steps.sphinx-cache.outputs.cache-hit != 'true' }} From 006d97fefd1b5873cc02918eaa533f3c508b4d79 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Thu, 3 Oct 2024 23:48:41 +0200 Subject: [PATCH 09/43] proplists: make `proplists:proplist/0` and `proplists:property/0` avail Add missing `export_type([...])` and missing `proplists` type Signed-off-by: Davide Bettio --- libs/estdlib/src/proplists.erl | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libs/estdlib/src/proplists.erl b/libs/estdlib/src/proplists.erl index d7f8a9caf..fb098ad0e 100644 --- a/libs/estdlib/src/proplists.erl +++ b/libs/estdlib/src/proplists.erl @@ -43,7 +43,10 @@ to_map/1 ]). +-export_type([property/0, proplist/0]). + -type property() :: atom() | {term(), term()}. +-type proplist() :: [property()]. % Taken from `otp/blob/master/lib/stdlib/src/proplists.erl` %%----------------------------------------------------------------------------- From 5f3a78fba6afd91a10f5b664be91152ce221cc89 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Thu, 3 Oct 2024 23:48:46 +0200 Subject: [PATCH 10/43] ESP32 tests: make room for additional tests Increase factory app partition size. Signed-off-by: Davide Bettio --- .../esp32/test/main/test_erl_sources/test_esp_partition.erl | 6 +++--- src/platforms/esp32/test/partitions.csv | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/src/platforms/esp32/test/main/test_erl_sources/test_esp_partition.erl b/src/platforms/esp32/test/main/test_erl_sources/test_esp_partition.erl index e0810839e..316552ddd 100644 --- a/src/platforms/esp32/test/main/test_erl_sources/test_esp_partition.erl +++ b/src/platforms/esp32/test/main/test_erl_sources/test_esp_partition.erl @@ -25,8 +25,8 @@ start() -> [ {<<"nvs">>, 1, 2, 16#9000, 16#6000, []}, {<<"phy_init">>, 1, 1, 16#f000, 16#1000, []}, - {<<"factory">>, 0, 0, 16#10000, 16#1C0000, []}, - {<<"lib.avm">>, 1, 1, 16#1D0000, 16#40000, []}, - {<<"main.avm">>, 1, 1, 16#210000, 16#100000, []} + {<<"factory">>, 0, 0, 16#10000, 16#200000, []}, + {<<"lib.avm">>, 1, 1, 16#210000, 16#40000, []}, + {<<"main.avm">>, 1, 1, 16#250000, 16#100000, []} ] = esp:partition_list(), 0. diff --git a/src/platforms/esp32/test/partitions.csv b/src/platforms/esp32/test/partitions.csv index aa2ff459f..6a45f33d3 100644 --- a/src/platforms/esp32/test/partitions.csv +++ b/src/platforms/esp32/test/partitions.csv @@ -10,6 +10,6 @@ # Note: if you change the phy_init or app partition offset, make sure to change the offset in Kconfig.projbuild nvs, data, nvs, 0x9000, 0x6000, phy_init, data, phy, 0xf000, 0x1000, -factory, app, factory, 0x10000, 0x1C0000, -lib.avm, data, phy, 0x1D0000, 0x40000, -main.avm, data, phy, 0x210000, 0x100000 +factory, app, factory, 0x10000, 0x200000, +lib.avm, data, phy, 0x210000, 0x40000, +main.avm, data, phy, 0x250000, 0x100000 From e9c080206e9fd59e6180cfbc06e14c20eac6f64a Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Thu, 3 Oct 2024 23:48:49 +0200 Subject: [PATCH 11/43] ESP32: add support for mounting/umounting storage Allow to mount and umount external storage such as SD/MMC or internal flash using `esp:mount/4` and `esp:umount/1`. Right now only `fat` filesystem is supported. Their semantic and parameters resembles unix mount and umount syscalls. Signed-off-by: Davide Bettio --- CHANGELOG.md | 2 + libs/eavmlib/src/esp.erl | 33 ++ .../components/avm_builtins/CMakeLists.txt | 3 +- .../esp32/components/avm_builtins/Kconfig | 4 + .../components/avm_builtins/storage_nif.c | 385 ++++++++++++++++++ .../components/avm_sys/include/esp32_sys.h | 4 + .../test/main/test_erl_sources/CMakeLists.txt | 3 + .../test/main/test_erl_sources/test_mount.erl | 47 +++ src/platforms/esp32/test/main/test_main.c | 11 + 9 files changed, 491 insertions(+), 1 deletion(-) create mode 100644 src/platforms/esp32/components/avm_builtins/storage_nif.c create mode 100644 src/platforms/esp32/test/main/test_erl_sources/test_mount.erl diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b85864c5..6ae49730b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,6 +39,8 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - ESP32: add support for `esp_adc` ADC driver, with Erlang and Elixir examples - Add handler for ESP32 network driver STA mode `beacon_timeout` (event: 21), see issue [#1100](https://github.com/atomvm/AtomVM/issues/1100) +- Support for mounting/unmounting storage on ESP32 (such as SD or internal flash) using +`esp:mount/4` and `esp:umount/1` ### Changed diff --git a/libs/eavmlib/src/esp.erl b/libs/eavmlib/src/esp.erl index 1b87dc122..f9d8ac187 100644 --- a/libs/eavmlib/src/esp.erl +++ b/libs/eavmlib/src/esp.erl @@ -38,6 +38,8 @@ sleep_enable_ulp_wakeup/0, deep_sleep/0, deep_sleep/1, + mount/4, + umount/1, nvs_fetch_binary/2, nvs_get_binary/1, nvs_get_binary/2, nvs_get_binary/3, nvs_set_binary/2, nvs_set_binary/3, @@ -118,6 +120,8 @@ }. -opaque task_wdt_user_handle() :: binary(). +-opaque mounted_fs() :: binary(). + -export_type( [ esp_reset_reason/0, @@ -128,6 +132,7 @@ esp_partition_address/0, esp_partition_size/0, esp_partition_props/0, + mounted_fs/0, task_wdt_config/0, task_wdt_user_handle/0 ] @@ -279,6 +284,34 @@ deep_sleep() -> deep_sleep(_SleepMS) -> erlang:nif_error(undefined). +%%----------------------------------------------------------------------------- +%% @param Source the device that will be mounted +%% @param Target the path where the filesystem will be mounted +%% @param FS the filesystem, only fat is supported now +%% @param Opts +%% @returns either a tuple having `ok' and the mounted fs resource, or an error tuple +%% @doc Mount a filesystem, and return a resource that can be used later for unmounting it +%% @end +%%----------------------------------------------------------------------------- +-spec mount( + Source :: unicode:chardata(), + Target :: unicode:chardata(), + FS :: fat, + Opts :: proplists:proplist() | #{atom() => term()} +) -> {ok, mounted_fs()} | {error, term()}. +mount(_Source, _Target, _FS, _Opts) -> + erlang:nif_error(undefined). + +%%----------------------------------------------------------------------------- +%% @param The mounted filesystem resource that should be unmounted +%% @returns either `ok' or an error tuple +%% @doc Unmounts filesystem located at given path +%% @end +%%----------------------------------------------------------------------------- +-spec umount(mounted_fs()) -> ok | {error, term()}. +umount(_Target) -> + erlang:nif_error(undefined). + %%----------------------------------------------------------------------------- %% @param Namespace NVS namespace %% @param Key NVS key diff --git a/src/platforms/esp32/components/avm_builtins/CMakeLists.txt b/src/platforms/esp32/components/avm_builtins/CMakeLists.txt index d0fd79e7a..f5d1ad290 100644 --- a/src/platforms/esp32/components/avm_builtins/CMakeLists.txt +++ b/src/platforms/esp32/components/avm_builtins/CMakeLists.txt @@ -27,6 +27,7 @@ set(AVM_BUILTIN_COMPONENT_SRCS "rtc_slow_nif.c" "socket_driver.c" "spi_driver.c" + "storage_nif.c" "uart_driver.c" "otp_crypto_platform.c" "otp_net_platform.c" @@ -56,7 +57,7 @@ endif() idf_component_register( SRCS ${AVM_BUILTIN_COMPONENT_SRCS} INCLUDE_DIRS "include" - PRIV_REQUIRES "libatomvm" "avm_sys" "nvs_flash" "driver" "esp_event" "esp_wifi" ${ADDITIONAL_PRIV_REQUIRES} + PRIV_REQUIRES "libatomvm" "avm_sys" "nvs_flash" "driver" "esp_event" "esp_wifi" "fatfs" ${ADDITIONAL_PRIV_REQUIRES} ${OPTIONAL_WHOLE_ARCHIVE} ) diff --git a/src/platforms/esp32/components/avm_builtins/Kconfig b/src/platforms/esp32/components/avm_builtins/Kconfig index c4196bfbc..1ac3a818c 100644 --- a/src/platforms/esp32/components/avm_builtins/Kconfig +++ b/src/platforms/esp32/components/avm_builtins/Kconfig @@ -72,6 +72,10 @@ config AVM_RTC_SLOW_MAX_SIZE # 4KB is a reasonable default default 4096 +config AVM_ENABLE_STORAGE_NIFS + bool "Enable Storage NIFs" + default y + config AVM_ENABLE_GPIO_PORT_DRIVER bool "Enable GPIO port driver" default y diff --git a/src/platforms/esp32/components/avm_builtins/storage_nif.c b/src/platforms/esp32/components/avm_builtins/storage_nif.c new file mode 100644 index 000000000..ad0137c15 --- /dev/null +++ b/src/platforms/esp32/components/avm_builtins/storage_nif.c @@ -0,0 +1,385 @@ +/* + * This file is part of AtomVM. + * + * Copyright 2024 Davide Bettio + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later + */ + +#include +#ifdef CONFIG_AVM_ENABLE_STORAGE_NIFS + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "esp32_sys.h" + +#include + +#include +#include +#include + +#include + +#include "spi_driver.h" + +#define TAG "storage_nif" + +#ifndef AVM_NO_SMP +#define SMP_LOCK_INIT(mounted_fs) smp_spinlock_init(&mounted_fs->lock) +#define SMP_LOCK(mounted_fs) smp_spinlock_lock(&mounted_fs->lock) +#define SMP_UNLOCK(mounted_fs) smp_spinlock_unlock(&mounted_fs->lock) +#else +#define SMP_LOCK_INIT(mounted_fs) +#define SMP_LOCK(mounted_fs) +#define SMP_UNLOCK(mounted_fs) +#endif + +// TODO: allow ro option +enum mount_type +{ + Unmounted, + FATSPIFlash, + FATSDSPI, + FATSDMMC +}; + +struct MountedFS +{ +#ifndef AVM_NO_SMP + SpinLock lock; +#endif + char *base_path; + enum mount_type mount_type; + union + { + sdmmc_card_t *card; + wl_handle_t wl; + } handle; +}; + +static void mounted_fs_dtor(ErlNifEnv *caller_env, void *obj); + +const ErlNifResourceTypeInit mounted_fs_resource_type_init = { + .members = 1, + .dtor = mounted_fs_dtor +}; + +static term make_esp_error_tuple(esp_err_t err, Context *ctx) +{ + if (UNLIKELY(memory_ensure_free(ctx, TUPLE_SIZE(2)) != MEMORY_GC_OK)) { + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + term result = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(result, 0, ERROR_ATOM); + term_put_tuple_element(result, 1, esp_err_to_term(ctx->global, err)); + return result; +} + +static void opts_to_fatfs_mount_config(term opts_term, esp_vfs_fat_mount_config_t *mount_config) +{ + mount_config->format_if_mount_failed = true; + mount_config->max_files = 8; + mount_config->allocation_unit_size = 512; + // TODO: make it configurable: disk_status_check_enable = false +} + +static term nif_esp_mount(Context *ctx, int argc, term argv[]) +{ + GlobalContext *glb = ctx->global; + struct ESP32PlatformData *platform = glb->platform_data; + + term source_term = argv[0]; + term target_term = argv[1]; + term filesystem_type_term = argv[2]; + term opts_term = argv[3]; + + int str_ok; + char *source = interop_term_to_string(source_term, &str_ok); + if (!str_ok) { + RAISE_ERROR(BADARG_ATOM); + } + + char *target = interop_term_to_string(target_term, &str_ok); + if (!str_ok) { + free(source); + RAISE_ERROR(BADARG_ATOM); + } + if (strlen(target) > 8) { + free(source); + free(target); + RAISE_ERROR(BADARG_ATOM); + } + + term fat_term + = globalcontext_existing_term_from_atom_string(ctx->global, ATOM_STR("\x3", "fat")); + if (term_is_invalid_term(fat_term) || filesystem_type_term != fat_term) { + free(source); + free(target); + RAISE_ERROR(BADARG_ATOM); + } + + if (!term_is_list(opts_term) && !term_is_map(opts_term)) { + free(source); + free(target); + RAISE_ERROR(BADARG_ATOM); + } + + esp_vfs_fat_mount_config_t mount_config = {}; + opts_to_fatfs_mount_config(opts_term, &mount_config); + + esp_err_t ret = -1; + struct MountedFS *mount = NULL; + + const char *part_by_name_prefix = "/dev/partition/by-name/"; + int part_by_name_len = strlen(part_by_name_prefix); + if (!strncmp(part_by_name_prefix, source, part_by_name_len)) { + mount_config.allocation_unit_size = CONFIG_WL_SECTOR_SIZE; + + mount = enif_alloc_resource(platform->mounted_fs_resource_type, sizeof(struct MountedFS)); + if (IS_NULL_PTR(mount)) { + free(source); + free(target); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + SMP_LOCK_INIT(mount); + mount->base_path = target; + target = NULL; + mount->mount_type = FATSPIFlash; + +#if ESP_IDF_VERSION_MAJOR >= 5 + ret = esp_vfs_fat_spiflash_mount_rw_wl( + mount->base_path, source + part_by_name_len, &mount_config, &mount->handle.wl); +#else + ret = esp_vfs_fat_spiflash_mount( + mount->base_path, source + part_by_name_len, &mount_config, &mount->handle.wl); +#endif + +// C3 doesn't support this +#ifdef SDMMC_SLOT_CONFIG_DEFAULT + } else if (!strcmp(source, "sdmmc")) { + mount_config.allocation_unit_size = 512; + + sdmmc_host_t host_config = SDMMC_HOST_DEFAULT(); + sdmmc_slot_config_t slot_config = SDMMC_SLOT_CONFIG_DEFAULT(); + + mount = enif_alloc_resource(platform->mounted_fs_resource_type, sizeof(struct MountedFS)); + if (IS_NULL_PTR(mount)) { + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + SMP_LOCK_INIT(mount); + mount->base_path = target; + target = NULL; + mount->mount_type = FATSDMMC; + + ret = esp_vfs_fat_sdmmc_mount( + mount->base_path, &host_config, &slot_config, &mount_config, &mount->handle.card); +#endif + + } else if (!strcmp(source, "sdspi")) { + mount_config.allocation_unit_size = 512; + + sdmmc_host_t host_config = SDSPI_HOST_DEFAULT(); + sdspi_device_config_t spi_slot_config = SDSPI_DEVICE_CONFIG_DEFAULT(); + + term spi_port = interop_kv_get_value_default( + opts_term, ATOM_STR("\x8", "spi_host"), term_invalid_term(), ctx->global); + spi_host_device_t host_dev; + // spi_driver_get_peripheral already checks if spi_port is valid + bool ok = spi_driver_get_peripheral(spi_port, &host_dev, ctx->global); + if (!ok) { + free(source); + free(target); + RAISE_ERROR(BADARG_ATOM); + } + spi_slot_config.host_id = host_dev; + + term cs_term = interop_kv_get_value_default( + opts_term, ATOM_STR("\x2", "cs"), term_invalid_term(), ctx->global); + if (UNLIKELY(!term_is_integer(cs_term))) { + free(source); + free(target); + RAISE_ERROR(BADARG_ATOM); + } + spi_slot_config.gpio_cs = term_to_int(cs_term); + + term cd_term = interop_kv_get_value_default( + opts_term, ATOM_STR("\x2", "cd"), UNDEFINED_ATOM, ctx->global); + if (cd_term != UNDEFINED_ATOM) { + if (UNLIKELY(!term_is_integer(cd_term))) { + free(source); + free(target); + RAISE_ERROR(BADARG_ATOM); + } + spi_slot_config.gpio_cd = term_to_int(cd_term); + } + + mount = enif_alloc_resource(platform->mounted_fs_resource_type, sizeof(struct MountedFS)); + if (IS_NULL_PTR(mount)) { + free(source); + free(target); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + SMP_LOCK_INIT(mount); + mount->base_path = target; + target = NULL; + mount->mount_type = FATSDSPI; + + ret = esp_vfs_fat_sdspi_mount( + mount->base_path, &host_config, &spi_slot_config, &mount_config, &mount->handle.card); + } else { + free(source); + free(target); + RAISE_ERROR(BADARG_ATOM); + } + + free(source); + + term return_term = term_invalid_term(); + if (UNLIKELY(ret != ESP_OK)) { + mount->mount_type = Unmounted; + return_term = make_esp_error_tuple(ret, ctx); + } else { + if (UNLIKELY(memory_ensure_free(ctx, TUPLE_SIZE(2) + TERM_BOXED_RESOURCE_SIZE) + != MEMORY_GC_OK)) { + enif_release_resource(mount); + RAISE_ERROR(OUT_OF_MEMORY_ATOM); + } + term mount_term = enif_make_resource(erl_nif_env_from_context(ctx), mount); + return_term = term_alloc_tuple(2, &ctx->heap); + term_put_tuple_element(return_term, 0, OK_ATOM); + term_put_tuple_element(return_term, 1, mount_term); + } + enif_release_resource(mount); + + return return_term; +} + +static esp_err_t do_umount(struct MountedFS *mount) +{ + SMP_LOCK(mount); + esp_err_t ret = ESP_FAIL; + + switch (mount->mount_type) { + case Unmounted: + ret = ESP_OK; + break; + + case FATSPIFlash: +#if ESP_IDF_VERSION_MAJOR >= 5 + ret = esp_vfs_fat_spiflash_unmount_rw_wl(mount->base_path, mount->handle.wl); +#else + ret = esp_vfs_fat_spiflash_unmount(mount->base_path, mount->handle.wl); +#endif + break; + + case FATSDSPI: + case FATSDMMC: + ret = esp_vfs_fat_sdcard_unmount(mount->base_path, mount->handle.card); + break; + } + + if (ret == ESP_OK) { + mount->mount_type = Unmounted; + } + + SMP_UNLOCK(mount); + return ret; +} + +static term nif_esp_umount(Context *ctx, int argc, term argv[]) +{ + GlobalContext *glb = ctx->global; + struct ESP32PlatformData *platform = glb->platform_data; + + void *mount_obj_ptr; + if (UNLIKELY(!enif_get_resource(erl_nif_env_from_context(ctx), argv[0], + platform->mounted_fs_resource_type, &mount_obj_ptr))) { + RAISE_ERROR(BADARG_ATOM); + } + struct MountedFS *mounted_fs = (struct MountedFS *) mount_obj_ptr; + + if (UNLIKELY(mounted_fs->mount_type == Unmounted)) { + RAISE_ERROR(BADARG_ATOM); + } + + esp_err_t ret = do_umount(mounted_fs); + + if (UNLIKELY(ret != ESP_OK)) { + return make_esp_error_tuple(ret, ctx); + } + + return OK_ATOM; +} + +static void mounted_fs_dtor(ErlNifEnv *caller_env, void *obj) +{ + UNUSED(caller_env); + + struct MountedFS *mounted_fs = (struct MountedFS *) obj; + esp_err_t ret = do_umount(mounted_fs); + + if (UNLIKELY(ret != ESP_OK)) { + ESP_LOGW(TAG, "Failed umount for %s in resource dtor. Please use esp:umount/1.", + mounted_fs->base_path); + } + + free(mounted_fs->base_path); +} + +void storage_nif_init(GlobalContext *global) +{ + struct ESP32PlatformData *platform = global->platform_data; + + ErlNifEnv env; + erl_nif_env_partial_init_from_globalcontext(&env, global); + platform->mounted_fs_resource_type = enif_init_resource_type( + &env, "mounted_fs", &mounted_fs_resource_type_init, ERL_NIF_RT_CREATE, NULL); +} + +static const struct Nif esp_mount_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_esp_mount +}; + +static const struct Nif esp_umount_nif = { + .base.type = NIFFunctionType, + .nif_ptr = nif_esp_umount +}; + +const struct Nif *storage_nif_get_nif(const char *nifname) +{ + if (strcmp("esp:mount/4", nifname) == 0) { + TRACE("Resolved platform nif %s ...\n", nifname); + return &esp_mount_nif; + } else if (strcmp("esp:umount/1", nifname) == 0) { + TRACE("Resolved platform nif %s ...\n", nifname); + return &esp_umount_nif; + } + + return NULL; +} + +REGISTER_NIF_COLLECTION(storage, storage_nif_init, NULL, storage_nif_get_nif) + +#endif diff --git a/src/platforms/esp32/components/avm_sys/include/esp32_sys.h b/src/platforms/esp32/components/avm_sys/include/esp32_sys.h index 30a3ab5b5..b09165405 100644 --- a/src/platforms/esp32/components/avm_sys/include/esp32_sys.h +++ b/src/platforms/esp32/components/avm_sys/include/esp32_sys.h @@ -114,6 +114,10 @@ struct ESP32PlatformData #endif mbedtls_ctr_drbg_context random_ctx; bool random_is_initialized; + +#ifdef CONFIG_AVM_ENABLE_STORAGE_NIFS + ErlNifResourceType *mounted_fs_resource_type; +#endif }; typedef void (*port_driver_init_t)(GlobalContext *global); diff --git a/src/platforms/esp32/test/main/test_erl_sources/CMakeLists.txt b/src/platforms/esp32/test/main/test_erl_sources/CMakeLists.txt index 4cc13d62b..72de3ddac 100644 --- a/src/platforms/esp32/test/main/test_erl_sources/CMakeLists.txt +++ b/src/platforms/esp32/test/main/test_erl_sources/CMakeLists.txt @@ -43,6 +43,7 @@ compile_erlang(test_list_to_binary) compile_erlang(test_md5) compile_erlang(test_crypto) compile_erlang(test_monotonic_time) +compile_erlang(test_mount) compile_erlang(test_net) compile_erlang(test_rtc_slow) compile_erlang(test_select) @@ -62,6 +63,7 @@ add_custom_command( test_md5.beam test_crypto.beam test_monotonic_time.beam + test_mount.beam test_net.beam test_rtc_slow.beam test_select.beam @@ -78,6 +80,7 @@ add_custom_command( "${CMAKE_CURRENT_BINARY_DIR}/test_md5.beam" "${CMAKE_CURRENT_BINARY_DIR}/test_crypto.beam" "${CMAKE_CURRENT_BINARY_DIR}/test_monotonic_time.beam" + "${CMAKE_CURRENT_BINARY_DIR}/test_mount.beam" "${CMAKE_CURRENT_BINARY_DIR}/test_net.beam" "${CMAKE_CURRENT_BINARY_DIR}/test_rtc_slow.beam" "${CMAKE_CURRENT_BINARY_DIR}/test_select.beam" diff --git a/src/platforms/esp32/test/main/test_erl_sources/test_mount.erl b/src/platforms/esp32/test/main/test_erl_sources/test_mount.erl new file mode 100644 index 000000000..e61dac4f9 --- /dev/null +++ b/src/platforms/esp32/test/main/test_erl_sources/test_mount.erl @@ -0,0 +1,47 @@ +% +% This file is part of AtomVM. +% +% Copyright 2024 Davide Bettio +% +% Licensed under the Apache License, Version 2.0 (the "License"); +% you may not use this file except in compliance with the License. +% You may obtain a copy of the License at +% +% http://www.apache.org/licenses/LICENSE-2.0 +% +% Unless required by applicable law or agreed to in writing, software +% distributed under the License is distributed on an "AS IS" BASIS, +% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +% See the License for the specific language governing permissions and +% limitations under the License. +% +% SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later +% + +-module(test_mount). +-export([start/0]). + +start() -> + {ok, Ref} = mount_working_sdmmc(), + ok = umount_prev(Ref), + ok = mount_missing_fat_partition(), + ok = umount_prev(Ref). + +mount_working_sdmmc() -> + {ok, Ref} = esp:mount("sdmmc", "/test", fat, []), + {ok, Fd} = atomvm:posix_open("/test/test.txt", [o_rdwr, o_creat], 8#644), + ok = atomvm:posix_close(Fd), + ok = esp:umount(Ref), + {ok, Ref}. + +mount_missing_fat_partition() -> + {error, esp_err_not_found} = esp:mount("/dev/partition/by-name/missingpart", "/test", fat, []), + ok. + +umount_prev(Ref) -> + try esp:umount(Ref) of + ok -> error + catch + error:badarg -> ok; + _:_ -> not_badarg + end. diff --git a/src/platforms/esp32/test/main/test_main.c b/src/platforms/esp32/test/main/test_main.c index 83076c348..509f948c7 100644 --- a/src/platforms/esp32/test/main/test_main.c +++ b/src/platforms/esp32/test/main/test_main.c @@ -242,6 +242,17 @@ TEST_CASE("test_monotonic_time", "[test_run]") TEST_ASSERT(ret_value == OK_ATOM); } +#if !CONFIG_IDF_TARGET_ESP32C3 +// this test is failing on v5.0.7 due to some kind of problem with atomvm:posix_open +#if ESP_IDF_VERSION_MAJOR >= 5 && ESP_IDF_VERSION_MINOR >= 1 +TEST_CASE("test_mount", "[test_run]") +{ + term ret_value = avm_test_case("test_mount.beam"); + TEST_ASSERT(ret_value == OK_ATOM); +} +#endif +#endif + struct pipefs_global_ctx { int max_fd; From 3f6cb6b1dd45782912424263f808512fa7a87bd9 Mon Sep 17 00:00:00 2001 From: Yuto Oguchi Date: Thu, 10 Oct 2024 09:42:29 +0900 Subject: [PATCH 12/43] Refactor `uart_driver` on ESP32 platform Use newer APIs to make it easier to read Signed-off-by: Yuto Oguchi --- .../components/avm_builtins/uart_driver.c | 67 ++++++------------- 1 file changed, 22 insertions(+), 45 deletions(-) diff --git a/src/platforms/esp32/components/avm_builtins/uart_driver.c b/src/platforms/esp32/components/avm_builtins/uart_driver.c index 8713f9bd7..ea6a1ebc9 100644 --- a/src/platforms/esp32/components/avm_builtins/uart_driver.c +++ b/src/platforms/esp32/components/avm_builtins/uart_driver.c @@ -298,13 +298,12 @@ Context *uart_driver_create_port(GlobalContext *global, term opts) return ctx; } -static void uart_driver_do_read(Context *ctx, term msg) +static void uart_driver_do_read(Context *ctx, GenMessage gen_message) { GlobalContext *glb = ctx->global; struct UARTData *uart_data = ctx->platform_data; - - term pid = term_get_tuple_element(msg, 0); - term ref = term_get_tuple_element(msg, 1); + term pid = gen_message.pid; + term ref = gen_message.ref; uint64_t ref_ticks = term_to_ref_ticks(ref); int local_pid = term_to_local_process_id(pid); @@ -313,20 +312,12 @@ static void uart_driver_do_read(Context *ctx, term msg) if (UNLIKELY(memory_ensure_free(ctx, TUPLE_SIZE(2) * 2 + REF_SIZE) != MEMORY_GC_OK)) { ESP_LOGE(TAG, "[uart_driver_do_read] Failed to allocate space for error tuple"); globalcontext_send_message(glb, local_pid, MEMORY_ATOM); + return; } term ealready = globalcontext_make_atom(glb, ealready_atom); - - term error_tuple = term_alloc_tuple(2, &ctx->heap); - term_put_tuple_element(error_tuple, 0, ERROR_ATOM); - term_put_tuple_element(error_tuple, 1, ealready); - - term result_tuple = term_alloc_tuple(2, &ctx->heap); - term_put_tuple_element(result_tuple, 0, term_from_ref_ticks(ref_ticks, &ctx->heap)); - term_put_tuple_element(result_tuple, 1, error_tuple); - - globalcontext_send_message(glb, local_pid, result_tuple); - + term error_tuple = port_create_error_tuple(ctx, ealready); + port_send_reply(ctx, pid, ref, error_tuple); return; } @@ -348,11 +339,7 @@ static void uart_driver_do_read(Context *ctx, term msg) term_put_tuple_element(ok_tuple, 0, OK_ATOM); term_put_tuple_element(ok_tuple, 1, bin); - term result_tuple = term_alloc_tuple(2, &ctx->heap); - term_put_tuple_element(result_tuple, 0, term_from_ref_ticks(ref_ticks, &ctx->heap)); - term_put_tuple_element(result_tuple, 1, ok_tuple); - - globalcontext_send_message(glb, local_pid, result_tuple); + port_send_reply(ctx, pid, ref, ok_tuple); } else { uart_data->reader_process_pid = pid; @@ -360,18 +347,16 @@ static void uart_driver_do_read(Context *ctx, term msg) } } -static void uart_driver_do_write(Context *ctx, term msg) +static void uart_driver_do_write(Context *ctx, GenMessage gen_message) { GlobalContext *glb = ctx->global; struct UARTData *uart_data = ctx->platform_data; + term msg = gen_message.req; + term pid = gen_message.pid; + term ref = gen_message.ref; - term pid = term_get_tuple_element(msg, 0); - term ref = term_get_tuple_element(msg, 1); - uint64_t ref_ticks = term_to_ref_ticks(ref); - - term cmd = term_get_tuple_element(msg, 2); - - term data = term_get_tuple_element(cmd, 1); + term cmd = term_get_tuple_element(msg, 0); + term data = term_get_tuple_element(msg, 1); size_t buffer_size; switch (interop_iolist_size(data, &buffer_size)) { @@ -408,21 +393,16 @@ static void uart_driver_do_write(Context *ctx, term msg) globalcontext_send_message(glb, local_pid, MEMORY_ATOM); } - term result_tuple = term_alloc_tuple(2, &ctx->heap); - term_put_tuple_element(result_tuple, 0, term_from_ref_ticks(ref_ticks, &ctx->heap)); - term_put_tuple_element(result_tuple, 1, OK_ATOM); - - globalcontext_send_message(glb, local_pid, result_tuple); + port_send_reply(ctx, pid, ref, OK_ATOM); } -static void uart_driver_do_close(Context *ctx, term msg) +static void uart_driver_do_close(Context *ctx, GenMessage gen_message) { GlobalContext *glb = ctx->global; struct UARTData *uart_data = ctx->platform_data; - - term pid = term_get_tuple_element(msg, 0); - term ref = term_get_tuple_element(msg, 1); - uint64_t ref_ticks = term_to_ref_ticks(ref); + term msg = gen_message.req; + term pid = gen_message.pid; + term ref = gen_message.ref; int local_pid = term_to_local_process_id(pid); @@ -433,10 +413,7 @@ static void uart_driver_do_close(Context *ctx, term msg) globalcontext_send_message(glb, local_pid, MEMORY_ATOM); } - term result_tuple = term_alloc_tuple(2, &ctx->heap); - term_put_tuple_element(result_tuple, 0, term_from_ref_ticks(ref_ticks, &ctx->heap)); - term_put_tuple_element(result_tuple, 1, OK_ATOM); - globalcontext_send_message(glb, local_pid, result_tuple); + port_send_reply(ctx, pid, ref, OK_ATOM); esp_err_t err = uart_driver_delete(uart_data->uart_num); if (UNLIKELY(err != ESP_OK)) { @@ -492,17 +469,17 @@ static NativeHandlerResult uart_driver_consume_mailbox(Context *ctx) switch (cmd) { case UARTReadCmd: TRACE("read\n"); - uart_driver_do_read(ctx, msg); + uart_driver_do_read(ctx, gen_message); break; case UARTWriteCmd: TRACE("write\n"); - uart_driver_do_write(ctx, msg); + uart_driver_do_write(ctx, gen_message); break; case UARTCloseCmd: TRACE("close\n"); - uart_driver_do_close(ctx, msg); + uart_driver_do_close(ctx, gen_message); is_closed = true; break; From afe8e0229e0ba98b36f7ef357186306edbd05adf Mon Sep 17 00:00:00 2001 From: Yuto Oguchi Date: Thu, 10 Oct 2024 09:52:58 +0900 Subject: [PATCH 13/43] Fix uart:open/1,2 to take uppercased peripheral name Nif `uart_driver` can only parse uppercased peripheral name Signed-off-by: Yuto Oguchi --- CHANGELOG.md | 1 + libs/eavmlib/src/uart.erl | 10 ++++------ 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b85864c5..338140f24 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,6 +61,7 @@ instead `network:start/1`, that would lead to a hard crash of the VM. - Fix a bug in ESP32 network driver where the low level driver was not being stopped and resoureces were not freed when `network:stop/0` was used, see issue [#643](https://github.com/atomvm/AtomVM/issues/643) +- `uart:open/1,2` now works with uppercase peripheral names ## [0.6.4] - 2024-08-18 diff --git a/libs/eavmlib/src/uart.erl b/libs/eavmlib/src/uart.erl index 601df3bd9..027f82023 100644 --- a/libs/eavmlib/src/uart.erl +++ b/libs/eavmlib/src/uart.erl @@ -88,17 +88,15 @@ warn_deprecated(OldKey, NewKey) -> validate_peripheral(I) when is_integer(I) -> io:format("UART: deprecated integer peripheral is used.~n"), I; -validate_peripheral([$u, $a, $r, $t | N] = Value) -> +validate_peripheral([$U, $A, $R, $T | N] = Value) -> try list_to_integer(N) of - % Internally integers are still used - % TODO: change this as soon as ESP32 code is reworked - I -> I + _ -> Value catch error:_ -> {bardarg, {peripheral, Value}} end; -validate_peripheral(<<"uart", N/binary>> = Value) -> +validate_peripheral(<<"UART", N/binary>> = Value) -> try binary_to_integer(N) of - I -> I + _ -> Value catch error:_ -> {bardarg, {peripheral, Value}} end; From d53fab56b6ae770cfe02be093e236d97e8162265 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Fri, 11 Oct 2024 00:07:59 +0200 Subject: [PATCH 14/43] externalterm: make serialize to buffer functions accessible Make functions for writing a term using external term format to a buffer accessible, so there is no need to use a binary. externalterm_serialize_term(_raw) and externalterm_compute_external_size(_raw) functions can be useful when using external term format in NIFs or port drivers. Before this change `externalterm_to_binary` was the only available function, but was forcing using a binary term as output. Signed-off-by: Davide Bettio --- CHANGELOG.md | 2 + src/libAtomVM/externalterm.c | 16 +++++++- src/libAtomVM/externalterm.h | 73 ++++++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2fd0e31a5..721368f61 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -41,6 +41,8 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` [#1100](https://github.com/atomvm/AtomVM/issues/1100) - Support for mounting/unmounting storage on ESP32 (such as SD or internal flash) using `esp:mount/4` and `esp:umount/1` +- Make external term serialize functions available without using `externalterm_to_binary` so terms +can be written directly to a buffer. ### Changed diff --git a/src/libAtomVM/externalterm.c b/src/libAtomVM/externalterm.c index 9d5cfb8ad..0936ad6b0 100644 --- a/src/libAtomVM/externalterm.c +++ b/src/libAtomVM/externalterm.c @@ -32,7 +32,6 @@ #include "unicode.h" #include "utils.h" -#define EXTERNAL_TERM_TAG 131 #define NEW_FLOAT_EXT 70 #define SMALL_INTEGER_EXT 97 #define INTEGER_EXT 98 @@ -947,3 +946,18 @@ static int calculate_heap_usage(const uint8_t *external_term_buf, size_t remaini return INVALID_TERM_SIZE; } } + +enum ExternalTermResult externalterm_compute_external_size_raw( + term t, size_t *size, GlobalContext *glb) +{ + *size = compute_external_size(t, glb); + + return EXTERNAL_TERM_OK; +} + +enum ExternalTermResult externalterm_serialize_term_raw(void *buf, term t, GlobalContext *glb) +{ + serialize_term(buf, t, glb); + + return EXTERNAL_TERM_OK; +} diff --git a/src/libAtomVM/externalterm.h b/src/libAtomVM/externalterm.h index a43c3813a..6a4b363fb 100644 --- a/src/libAtomVM/externalterm.h +++ b/src/libAtomVM/externalterm.h @@ -30,6 +30,8 @@ #include "term.h" +#define EXTERNAL_TERM_TAG 131 + #ifdef __cplusplus extern "C" { #endif @@ -94,6 +96,77 @@ enum ExternalTermResult externalterm_from_binary(Context *ctx, term *dst, term b */ term externalterm_to_binary(Context *ctx, term t); +/** + * @brief Computes the size required for a external term (tag excluded) + * + * @details This function should be called in order to calculate the required buffer size to store + * a serialized term in external term format. This function doesn't prepend the external term 1 byte + * tag. + * + * @param t the term for which size is calculated + * @param size the required buffer size (tag excluded) + * @param glb the global context + * @returns EXTERNAL_TERM_OK in case of success + */ +enum ExternalTermResult externalterm_compute_external_size_raw( + term t, size_t *size, GlobalContext *glb); + +/** + * @brief Serialize a term (tag excluded) + * + * @details This function serializes in external term format given term, and writes it to the given + * buffer. This function doesn't prepend the external term 1 byte tag. + * + * @param buf the buffer where the external term is written + * @param t the term that will be serialized + * @param glb the global context + * @returns EXTERNAL_TERM_OK in case of success + */ +enum ExternalTermResult externalterm_serialize_term_raw(void *buf, term t, GlobalContext *glb); + +/** + * @brief Computes the size required for a external term + * + * @details This function should be called in order to calculate the required buffer size to store + * a serialized term in external term format. + * + * @param t the term for which size is calculated + * @param size the required buffer size (tag excluded) + * @param glb the global context + * @returns EXTERNAL_TERM_OK in case of success + */ +static inline enum ExternalTermResult externalterm_compute_external_size( + term t, size_t *size, GlobalContext *glb) +{ + size_t raw_size; + enum ExternalTermResult result = externalterm_compute_external_size_raw(t, &raw_size, glb); + if (LIKELY(result == EXTERNAL_TERM_OK)) { + *size = raw_size + 1; + } + return result; +} + +/** + * @brief Serialize a term + * + * @details This function serializes in external term format given term, and writes it to the given + * buffer. + * + * @param buf the buffer where the external term is written + * @param t the term that will be serialized + * @param glb the global context + * @returns EXTERNAL_TERM_OK in case of success + */ +static inline enum ExternalTermResult externalterm_serialize_term( + void *buf, term t, GlobalContext *glb) +{ + enum ExternalTermResult result = externalterm_serialize_term_raw((uint8_t *) buf + 1, t, glb); + if (LIKELY(result == EXTERNAL_TERM_OK)) { + ((uint8_t *) buf)[0] = EXTERNAL_TERM_TAG; + } + return result; +} + #ifdef __cplusplus } #endif From 041b7b3e5b6d3489ce5beb9d61a9bcfd22132280 Mon Sep 17 00:00:00 2001 From: Yuto Oguchi Date: Wed, 2 Oct 2024 18:14:35 +0900 Subject: [PATCH 15/43] Add support for `binary_to_integer/2` Signed-off-by: Yuto Oguchi --- CHANGELOG.md | 1 + libs/estdlib/src/erlang.erl | 11 +++++ src/libAtomVM/nifs.c | 26 +++++++---- src/libAtomVM/nifs.gperf | 1 + tests/erlang_tests/CMakeLists.txt | 2 + .../erlang_tests/test_binary_to_integer_2.erl | 43 +++++++++++++++++++ tests/test.c | 1 + 7 files changed, 77 insertions(+), 8 deletions(-) create mode 100644 tests/erlang_tests/test_binary_to_integer_2.erl diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b85864c5..9c051e899 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,6 +39,7 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - ESP32: add support for `esp_adc` ADC driver, with Erlang and Elixir examples - Add handler for ESP32 network driver STA mode `beacon_timeout` (event: 21), see issue [#1100](https://github.com/atomvm/AtomVM/issues/1100) +- Support for `binary_to_integer/2` ### Changed diff --git a/libs/estdlib/src/erlang.erl b/libs/estdlib/src/erlang.erl index 8d41689b6..5a1164c95 100644 --- a/libs/estdlib/src/erlang.erl +++ b/libs/estdlib/src/erlang.erl @@ -57,6 +57,7 @@ binary_to_atom/1, binary_to_atom/2, binary_to_integer/1, + binary_to_integer/2, binary_to_list/1, atom_to_binary/1, atom_to_binary/2, @@ -651,6 +652,16 @@ binary_to_atom(_Binary, _Encoding) -> binary_to_integer(_Binary) -> erlang:nif_error(undefined). +%%----------------------------------------------------------------------------- +%% @param Binary Binary to parse for integer +%% @returns the integer represented by the binary +%% @doc Parse the text in a given binary as an integer. +%% @end +%%----------------------------------------------------------------------------- +-spec binary_to_integer(Binary :: binary(), Base :: 2..36) -> integer(). +binary_to_integer(_Binary, Base) -> + erlang:nif_error(undefined). + %%----------------------------------------------------------------------------- %% @param Binary Binary to convert to list %% @returns a list of bytes from the binary diff --git a/src/libAtomVM/nifs.c b/src/libAtomVM/nifs.c index 798ba07b7..7549e1249 100644 --- a/src/libAtomVM/nifs.c +++ b/src/libAtomVM/nifs.c @@ -99,7 +99,7 @@ static term nif_erlang_atom_to_binary(Context *ctx, int argc, term argv[]); static term nif_erlang_atom_to_list_1(Context *ctx, int argc, term argv[]); static term nif_erlang_binary_to_atom_2(Context *ctx, int argc, term argv[]); static term nif_erlang_binary_to_float_1(Context *ctx, int argc, term argv[]); -static term nif_erlang_binary_to_integer_1(Context *ctx, int argc, term argv[]); +static term nif_erlang_binary_to_integer(Context *ctx, int argc, term argv[]); static term nif_erlang_binary_to_list_1(Context *ctx, int argc, term argv[]); static term nif_erlang_binary_to_existing_atom_2(Context *ctx, int argc, term argv[]); static term nif_erlang_concat_2(Context *ctx, int argc, term argv[]); @@ -275,7 +275,7 @@ static const struct Nif binary_to_float_nif = static const struct Nif binary_to_integer_nif = { .base.type = NIFFunctionType, - .nif_ptr = nif_erlang_binary_to_integer_1 + .nif_ptr = nif_erlang_binary_to_integer }; static const struct Nif binary_to_list_nif = @@ -1819,10 +1819,8 @@ static term nif_erlang_binary_to_atom_2(Context *ctx, int argc, term argv[]) return binary_to_atom(ctx, argc, argv, 1); } -static term nif_erlang_binary_to_integer_1(Context *ctx, int argc, term argv[]) +static term nif_erlang_binary_to_integer(Context *ctx, int argc, term argv[]) { - UNUSED(argc); - term bin_term = argv[0]; VALIDATE_VALUE(bin_term, term_is_binary); @@ -1833,14 +1831,26 @@ static term nif_erlang_binary_to_integer_1(Context *ctx, int argc, term argv[]) RAISE_ERROR(BADARG_ATOM); } - char null_terminated_buf[24]; + uint8_t base = 10; + + if (argc == 2) { + term int_term = argv[1]; + VALIDATE_VALUE(int_term, term_is_uint8); + base = term_to_uint8(int_term); + } + + if (UNLIKELY((base < 2) || (base > 36))) { + RAISE_ERROR(BADARG_ATOM); + } + + char null_terminated_buf[65]; memcpy(null_terminated_buf, bin_data, bin_data_size); null_terminated_buf[bin_data_size] = '\0'; - //TODO: handle 64 bits numbers //TODO: handle errors + //TODO: do not copy buffer, implement a custom strotoll char *endptr; - uint64_t value = strtoll(null_terminated_buf, &endptr, 10); + uint64_t value = strtoll(null_terminated_buf, &endptr, base); if (*endptr != '\0') { RAISE_ERROR(BADARG_ATOM); } diff --git a/src/libAtomVM/nifs.gperf b/src/libAtomVM/nifs.gperf index e3a9e8246..0ca5f61c6 100644 --- a/src/libAtomVM/nifs.gperf +++ b/src/libAtomVM/nifs.gperf @@ -47,6 +47,7 @@ erlang:binary_to_atom/1, &binary_to_atom_nif erlang:binary_to_atom/2, &binary_to_atom_nif erlang:binary_to_float/1, &binary_to_float_nif erlang:binary_to_integer/1, &binary_to_integer_nif +erlang:binary_to_integer/2, &binary_to_integer_nif erlang:binary_to_list/1, &binary_to_list_nif erlang:binary_to_existing_atom/1, &binary_to_existing_atom_nif erlang:binary_to_existing_atom/2, &binary_to_existing_atom_nif diff --git a/tests/erlang_tests/CMakeLists.txt b/tests/erlang_tests/CMakeLists.txt index 979d9bcf6..07062c4f3 100644 --- a/tests/erlang_tests/CMakeLists.txt +++ b/tests/erlang_tests/CMakeLists.txt @@ -431,6 +431,7 @@ compile_erlang(fail_apply_last) compile_erlang(pid_to_list_test) compile_erlang(ref_to_list_test) compile_erlang(test_binary_to_integer) +compile_erlang(test_binary_to_integer_2) compile_erlang(count_char_bs) compile_erlang(count_char2_bs) @@ -902,6 +903,7 @@ add_custom_target(erlang_test_modules DEPENDS pid_to_list_test.beam ref_to_list_test.beam test_binary_to_integer.beam + test_binary_to_integer_2.beam count_char_bs.beam count_char2_bs.beam diff --git a/tests/erlang_tests/test_binary_to_integer_2.erl b/tests/erlang_tests/test_binary_to_integer_2.erl new file mode 100644 index 000000000..b3cb18e8f --- /dev/null +++ b/tests/erlang_tests/test_binary_to_integer_2.erl @@ -0,0 +1,43 @@ +% +% This file is part of AtomVM. +% +% Copyright 2024 Yuto Oguchi , Realglobe Inc. +% +% Licensed under the Apache License, Version 2.0 (the "License"); +% you may not use this file except in compliance with the License. +% You may obtain a copy of the License at +% +% http://www.apache.org/licenses/LICENSE-2.0 +% +% Unless required by applicable law or agreed to in writing, software +% distributed under the License is distributed on an "AS IS" BASIS, +% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +% See the License for the specific language governing permissions and +% limitations under the License. +% +% SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later +% + +-module(test_binary_to_integer_2). + +-export([start/0]). + +start() -> + ok = assert_badarg(fun() -> binary_to_integer(<<"10">>, -1) end), + ok = assert_badarg(fun() -> binary_to_integer(<<"10">>, 0) end), + ok = assert_badarg(fun() -> binary_to_integer(<<"10">>, 1) end), + 2 = binary_to_integer(<<"10">>, 2), + 36 = binary_to_integer(<<"10">>, 36), + ok = assert_badarg(fun() -> binary_to_integer(<<"10">>, 37) end), + ok = assert_badarg(fun() -> binary_to_integer(<<"">>, 10) end), + 10 = binary_to_integer(<<"0A">>, 16), + 10 = binary_to_integer(<<"0a">>, 16), + 0. + +assert_badarg(F) -> + try + R = F(), + {fail_no_ex, R} + catch + error:badarg -> ok + end. diff --git a/tests/test.c b/tests/test.c index 637343448..29bc258db 100644 --- a/tests/test.c +++ b/tests/test.c @@ -482,6 +482,7 @@ struct Test tests[] = { TEST_CASE_EXPECTED(pid_to_list_test, 63), TEST_CASE_EXPECTED(ref_to_list_test, 386), TEST_CASE_EXPECTED(test_binary_to_integer, 99), + TEST_CASE(test_binary_to_integer_2), TEST_CASE_EXPECTED(count_char_bs, 2), TEST_CASE_EXPECTED(count_char2_bs, 1002), From 545d315f219831719734a76ae047fb13d8cb435c Mon Sep 17 00:00:00 2001 From: Yuto Oguchi Date: Wed, 2 Oct 2024 18:15:10 +0900 Subject: [PATCH 16/43] Add support for `binary:decode_hex/1` and `binary:encode_hex/1,2` Signed-off-by: Yuto Oguchi --- CHANGELOG.md | 1 + libs/estdlib/src/binary.erl | 39 +++++++++++++++++++++++++++++- tests/libs/estdlib/test_binary.erl | 10 ++++++++ 3 files changed, 49 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c051e899..7af3af56e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,6 +40,7 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - Add handler for ESP32 network driver STA mode `beacon_timeout` (event: 21), see issue [#1100](https://github.com/atomvm/AtomVM/issues/1100) - Support for `binary_to_integer/2` +- Support for `binary:decode_hex/1` and `binary:encode_hex/1,2` ### Changed diff --git a/libs/estdlib/src/binary.erl b/libs/estdlib/src/binary.erl index 2254232c4..9b01258f8 100644 --- a/libs/estdlib/src/binary.erl +++ b/libs/estdlib/src/binary.erl @@ -2,6 +2,7 @@ % This file is part of AtomVM. % % Copyright 2023 Paul Guyot +% Copyright 2024 Yuto Oguchi , Realglobe Inc. % % Licensed under the Apache License, Version 2.0 (the "License"); % you may not use this file except in compliance with the License. @@ -24,7 +25,7 @@ %%----------------------------------------------------------------------------- -module(binary). --export([at/2, part/3, split/2, split/3]). +-export([at/2, decode_hex/1, encode_hex/1, encode_hex/2, part/3, split/2, split/3]). %%----------------------------------------------------------------------------- %% @param Binary binary to get a byte from @@ -37,6 +38,42 @@ at(_Binary, _Index) -> erlang:nif_error(undefined). +%%----------------------------------------------------------------------------- +%% @param Data hex encoded binary to decode +%% @returns decoded binary +%% @doc Decodes a hex encoded binary into a binary. +%% @end +%%----------------------------------------------------------------------------- +-spec decode_hex(Data :: <<_:_*16>>) -> binary(). +decode_hex(Data) -> + case byte_size(Data) rem 2 of + 0 -> <<<<(binary_to_integer(B, 16))>> || <> <= Data>>; + _ -> erlang:error(badarg) + end. + +%%----------------------------------------------------------------------------- +%% @param Data binary data to convert into hex encoded binary +%% @returns hex encoded binary +%% @doc Encodes a binary into a hex encoded binary using the specified case for the hexadecimal digits "a" to "f". +%% @end +%%----------------------------------------------------------------------------- +-spec encode_hex(Data :: binary()) -> binary(). +encode_hex(Data) -> + encode_hex(Data, uppercase). + +%%----------------------------------------------------------------------------- +%% @param Data binary data to convert into hex encoded binary +%% @param Case which case to encode into +%% @returns hex encoded binary +%% @doc Encodes a binary into a hex encoded binary using the specified case for the hexadecimal digits "a" to "f". +%% @end +%%----------------------------------------------------------------------------- +-spec encode_hex(Data :: binary(), Case :: lowercase | uppercase) -> binary(). +encode_hex(Data, uppercase) -> + <<(integer_to_binary(B, 16)) || <> <= Data>>; +encode_hex(Data, lowercase) -> + <<<<(hd(string:to_lower(integer_to_list(B, 16)))):8>> || <> <= Data>>. + %%----------------------------------------------------------------------------- %% @param Binary binary to extract a subbinary from %% @param Pos 0-based index of the subbinary to extract diff --git a/tests/libs/estdlib/test_binary.erl b/tests/libs/estdlib/test_binary.erl index 2549e045d..8138200fb 100644 --- a/tests/libs/estdlib/test_binary.erl +++ b/tests/libs/estdlib/test_binary.erl @@ -26,6 +26,7 @@ test() -> ok = test_split(), + ok = test_hex(), ok. test_split() -> @@ -34,3 +35,12 @@ test_split() -> ?ASSERT_MATCH(binary:split(<<"foobar">>, <<"o">>), [<<"f">>, <<"obar">>]), ?ASSERT_MATCH(binary:split(<<"foobar">>, <<"o">>, [global]), [<<"f">>, <<>>, <<"bar">>]), ok. + +test_hex() -> + RawBinary = <<"Hello, AtomVM!">>, + ?ASSERT_MATCH(binary:encode_hex(RawBinary), <<"48656C6C6F2C2041746F6D564D21">>), + ?ASSERT_MATCH(binary:encode_hex(RawBinary, lowercase), <<"48656c6c6f2c2041746f6d564d21">>), + ?ASSERT_MATCH(RawBinary, binary:decode_hex(<<"48656C6C6F2C2041746F6D564D21">>)), + ?ASSERT_EXCEPTION(binary:decode_hex(<<"48656C6C6F2C2041746F6D564D2">>), error, badarg), + ?ASSERT_EXCEPTION(binary:decode_hex(<<"ABCDEFGH">>), error, badarg), + ok. From 11b23abd024f44cf50fad45fda4a622066699e4d Mon Sep 17 00:00:00 2001 From: Yuto Oguchi Date: Wed, 2 Oct 2024 18:15:22 +0900 Subject: [PATCH 17/43] Add support for `Base.decode16/2` and `Base.encode16/2` Signed-off-by: Yuto Oguchi --- CHANGELOG.md | 1 + libs/exavmlib/lib/Base.ex | 227 +++++++++++++++++++++++++++++++ libs/exavmlib/lib/CMakeLists.txt | 1 + 3 files changed, 229 insertions(+) create mode 100644 libs/exavmlib/lib/Base.ex diff --git a/CHANGELOG.md b/CHANGELOG.md index 7af3af56e..d63f3ccfb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -41,6 +41,7 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` [#1100](https://github.com/atomvm/AtomVM/issues/1100) - Support for `binary_to_integer/2` - Support for `binary:decode_hex/1` and `binary:encode_hex/1,2` +- Support for Elixir `Base.decode16/2` and `Base.encode16/2` ### Changed diff --git a/libs/exavmlib/lib/Base.ex b/libs/exavmlib/lib/Base.ex new file mode 100644 index 000000000..27d15381c --- /dev/null +++ b/libs/exavmlib/lib/Base.ex @@ -0,0 +1,227 @@ +# +# This file is part of elixir-lang. +# +# Copyright 2014-2023 Elixir Contributors +# https://github.com/elixir-lang/elixir/commits/v1.17.3/lib/elixir/lib/base.ex +# +# Copyright 2024 Yuto Oguchi , Realglobe Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +defmodule Base do + @moduledoc """ + This module provides data encoding and decoding functions + according to [RFC 4648](https://tools.ietf.org/html/rfc4648). + + This document defines the commonly used base 16, base 32, and base + 64 encoding schemes. + + ## Base 16 alphabet + + | Value | Encoding | Value | Encoding | Value | Encoding | Value | Encoding | + |------:|:---------|------:|:---------|------:|:---------|------:|:---------| + | 0 | 0 | 4 | 4 | 8 | 8 | 12 | C | + | 1 | 1 | 5 | 5 | 9 | 9 | 13 | D | + | 2 | 2 | 6 | 6 | 10 | A | 14 | E | + | 3 | 3 | 7 | 7 | 11 | B | 15 | F | + + ## Base 32 alphabet + + | Value | Encoding | Value | Encoding | Value | Encoding | Value | Encoding | + |------:|:---------|------:|:---------|------:|:---------|------:|:---------| + | 0 | A | 9 | J | 18 | S | 27 | 3 | + | 1 | B | 10 | K | 19 | T | 28 | 4 | + | 2 | C | 11 | L | 20 | U | 29 | 5 | + | 3 | D | 12 | M | 21 | V | 30 | 6 | + | 4 | E | 13 | N | 22 | W | 31 | 7 | + | 5 | F | 14 | O | 23 | X | | | + | 6 | G | 15 | P | 24 | Y | (pad) | = | + | 7 | H | 16 | Q | 25 | Z | | | + | 8 | I | 17 | R | 26 | 2 | | | + + + ## Base 32 (extended hex) alphabet + + | Value | Encoding | Value | Encoding | Value | Encoding | Value | Encoding | + |------:|:---------|------:|:---------|------:|:---------|------:|:---------| + | 0 | 0 | 9 | 9 | 18 | I | 27 | R | + | 1 | 1 | 10 | A | 19 | J | 28 | S | + | 2 | 2 | 11 | B | 20 | K | 29 | T | + | 3 | 3 | 12 | C | 21 | L | 30 | U | + | 4 | 4 | 13 | D | 22 | M | 31 | V | + | 5 | 5 | 14 | E | 23 | N | | | + | 6 | 6 | 15 | F | 24 | O | (pad) | = | + | 7 | 7 | 16 | G | 25 | P | | | + | 8 | 8 | 17 | H | 26 | Q | | | + + ## Base 64 alphabet + + | Value | Encoding | Value | Encoding | Value | Encoding | Value | Encoding | + |------:|:----------|------:|:---------|------:|:---------|------:|:---------| + | 0 | A | 17 | R | 34 | i | 51 | z | + | 1 | B | 18 | S | 35 | j | 52 | 0 | + | 2 | C | 19 | T | 36 | k | 53 | 1 | + | 3 | D | 20 | U | 37 | l | 54 | 2 | + | 4 | E | 21 | V | 38 | m | 55 | 3 | + | 5 | F | 22 | W | 39 | n | 56 | 4 | + | 6 | G | 23 | X | 40 | o | 57 | 5 | + | 7 | H | 24 | Y | 41 | p | 58 | 6 | + | 8 | I | 25 | Z | 42 | q | 59 | 7 | + | 9 | J | 26 | a | 43 | r | 60 | 8 | + | 10 | K | 27 | b | 44 | s | 61 | 9 | + | 11 | L | 28 | c | 45 | t | 62 | + | + | 12 | M | 29 | d | 46 | u | 63 | / | + | 13 | N | 30 | e | 47 | v | | | + | 14 | O | 31 | f | 48 | w | (pad) | = | + | 15 | P | 32 | g | 49 | x | | | + | 16 | Q | 33 | h | 50 | y | | | + + ## Base 64 (URL and filename safe) alphabet + + | Value | Encoding | Value | Encoding | Value | Encoding | Value | Encoding | + |------:|:---------|------:|:---------|------:|:---------|------:|:---------| + | 0 | A | 17 | R | 34 | i | 51 | z | + | 1 | B | 18 | S | 35 | j | 52 | 0 | + | 2 | C | 19 | T | 36 | k | 53 | 1 | + | 3 | D | 20 | U | 37 | l | 54 | 2 | + | 4 | E | 21 | V | 38 | m | 55 | 3 | + | 5 | F | 22 | W | 39 | n | 56 | 4 | + | 6 | G | 23 | X | 40 | o | 57 | 5 | + | 7 | H | 24 | Y | 41 | p | 58 | 6 | + | 8 | I | 25 | Z | 42 | q | 59 | 7 | + | 9 | J | 26 | a | 43 | r | 60 | 8 | + | 10 | K | 27 | b | 44 | s | 61 | 9 | + | 11 | L | 28 | c | 45 | t | 62 | - | + | 12 | M | 29 | d | 46 | u | 63 | _ | + | 13 | N | 30 | e | 47 | v | | | + | 14 | O | 31 | f | 48 | w | (pad) | = | + | 15 | P | 32 | g | 49 | x | | | + | 16 | Q | 33 | h | 50 | y | | | + + """ + + @type encode_case :: :upper | :lower + @type decode_case :: :upper | :lower | :mixed + + @doc """ + Decodes a base 16 encoded string into a binary string. + + ## Options + + The accepted options are: + + * `:case` - specifies the character case to accept when decoding + + The values for `:case` can be: + + * `:upper` - only allows upper case characters (default) + * `:lower` - only allows lower case characters + * `:mixed` - allows mixed case characters + + ## Examples + + iex> Base.decode16("666F6F626172") + {:ok, "foobar"} + + iex> Base.decode16("666f6f626172", case: :lower) + {:ok, "foobar"} + + iex> Base.decode16("666f6F626172", case: :mixed) + {:ok, "foobar"} + + """ + @spec decode16(binary, case: decode_case) :: {:ok, binary} | :error + def decode16(string, ops \\ []) do + {:ok, decode16!(string, ops)} + rescue + ArgumentError -> :error + end + + @doc """ + Decodes a base 16 encoded string into a binary string. + + ## Options + + The accepted options are: + + * `:case` - specifies the character case to accept when decoding + + The values for `:case` can be: + + * `:upper` - only allows upper case characters (default) + * `:lower` - only allows lower case characters + * `:mixed` - allows mixed case characters + + An `ArgumentError` exception is raised if the padding is incorrect or + a non-alphabet character is present in the string. + + ## Examples + + iex> Base.decode16!("666F6F626172") + "foobar" + + iex> Base.decode16!("666f6f626172", case: :lower) + "foobar" + + iex> Base.decode16!("666f6F626172", case: :mixed) + "foobar" + + """ + @spec decode16!(binary, case: decode_case) :: binary + def decode16!(string, opts \\ []) + + def decode16!(string, _ops) when is_binary(string) and rem(byte_size(string), 2) == 0 do + # TODO: support :case option + :binary.decode_hex(string) + end + + def decode16!(string, _opts) when is_binary(string) do + raise ArgumentError, + "string given to decode has wrong length. An even number of bytes was expected, got: #{byte_size(string)}. " <> + "Double check your string for unwanted characters or pad it accordingly" + end + + @doc """ + Encodes a binary string into a base 16 encoded string. + + ## Options + + The accepted options are: + + * `:case` - specifies the character case to use when encoding + + The values for `:case` can be: + + * `:upper` - uses upper case characters (default) + * `:lower` - uses lower case characters + + ## Examples + + iex> Base.encode16("foobar") + "666F6F626172" + + iex> Base.encode16("foobar", case: :lower) + "666f6f626172" + + """ + @spec encode16(binary, case: encode_case) :: binary + def encode16(data, opts \\ []) do + case Keyword.get(opts, :case, :upper) do + :upper -> :binary.encode_hex(data, :uppercase) + :lower -> :binary.encode_hex(data, :lowercase) + end + end +end diff --git a/libs/exavmlib/lib/CMakeLists.txt b/libs/exavmlib/lib/CMakeLists.txt index c536159f4..19264acc1 100644 --- a/libs/exavmlib/lib/CMakeLists.txt +++ b/libs/exavmlib/lib/CMakeLists.txt @@ -24,6 +24,7 @@ include(BuildElixir) set(ELIXIR_MODULES AVMPort + Base Bitwise Code Console From 7dd50db206485a76f07f9262d1a982566ec81f47 Mon Sep 17 00:00:00 2001 From: Jakub Gonet Date: Fri, 11 Oct 2024 12:11:36 +0200 Subject: [PATCH 18/43] Add support for list_to_integer/2 Signed-off-by: Jakub Gonet --- CHANGELOG.md | 1 + libs/estdlib/src/erlang.erl | 14 ++++++++ src/libAtomVM/nifs.c | 40 +++++++++++++++------ src/libAtomVM/nifs.gperf | 1 + tests/erlang_tests/test_list_to_integer.erl | 18 +++++++--- 5 files changed, 59 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 30115467a..b59c408e6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -44,6 +44,7 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - Support for `binary_to_integer/2` - Support for `binary:decode_hex/1` and `binary:encode_hex/1,2` - Support for Elixir `Base.decode16/2` and `Base.encode16/2` +- Support for `erlang:list_to_integer/2` ### Changed diff --git a/libs/estdlib/src/erlang.erl b/libs/estdlib/src/erlang.erl index 5a1164c95..581753b7c 100644 --- a/libs/estdlib/src/erlang.erl +++ b/libs/estdlib/src/erlang.erl @@ -52,6 +52,7 @@ list_to_existing_atom/1, list_to_binary/1, list_to_integer/1, + list_to_integer/2, list_to_tuple/1, iolist_to_binary/1, binary_to_atom/1, @@ -601,6 +602,19 @@ list_to_binary(_IOList) -> list_to_integer(_String) -> erlang:nif_error(undefined). +%%----------------------------------------------------------------------------- +%% @param String string to convert to integer +%% @param Base string to convert to integer +%% @returns an integer value from its string representation +%% @doc Convert a string (list of characters) to integer in specified base. +%% Errors with `badarg' if the string is not a representation of an integer or +%% the base is out of bounds. +%% @end +%%----------------------------------------------------------------------------- +-spec list_to_integer(String :: string(), Base :: 2..36) -> integer(). +list_to_integer(_String, _Base) -> + erlang:nif_error(undefined). + %%----------------------------------------------------------------------------- %% @param List list to convert to tuple %% @returns a tuple with elements of the list diff --git a/src/libAtomVM/nifs.c b/src/libAtomVM/nifs.c index 7549e1249..0a185988f 100644 --- a/src/libAtomVM/nifs.c +++ b/src/libAtomVM/nifs.c @@ -118,7 +118,7 @@ static term nif_erlang_link(Context *ctx, int argc, term argv[]); static term nif_erlang_float_to_binary(Context *ctx, int argc, term argv[]); static term nif_erlang_float_to_list(Context *ctx, int argc, term argv[]); static term nif_erlang_list_to_binary_1(Context *ctx, int argc, term argv[]); -static term nif_erlang_list_to_integer_1(Context *ctx, int argc, term argv[]); +static term nif_erlang_list_to_integer(Context *ctx, int argc, term argv[]); static term nif_erlang_list_to_float_1(Context *ctx, int argc, term argv[]); static term nif_erlang_list_to_atom_1(Context *ctx, int argc, term argv[]); static term nif_erlang_list_to_existing_atom_1(Context *ctx, int argc, term argv[]); @@ -377,7 +377,7 @@ static const struct Nif list_to_binary_nif = static const struct Nif list_to_integer_nif = { .base.type = NIFFunctionType, - .nif_ptr = nif_erlang_list_to_integer_1 + .nif_ptr = nif_erlang_list_to_integer }; static const struct Nif list_to_float_nif = @@ -2510,9 +2510,30 @@ static term nif_erlang_list_to_binary_1(Context *ctx, int argc, term argv[]) return bin_res; } -static term nif_erlang_list_to_integer_1(Context *ctx, int argc, term argv[]) +static avm_int_t to_digit_index(avm_int_t character) { - UNUSED(argc); + if (character >= '0' && character <= '9') { + return character - '0'; + } else if (character >= 'a' && character <= 'z') { + return character - 'a' + 10; + } else if (character >= 'A' && character <= 'Z') { + return character - 'A' + 10; + } else { + return -1; + } +} + +static term nif_erlang_list_to_integer(Context *ctx, int argc, term argv[]) +{ + avm_int_t base = 10; + if (argc == 2) { + term t = argv[1]; + VALIDATE_VALUE(t, term_is_integer); + base = term_to_int(t); + if (UNLIKELY(base < 2 || base > 36)) { + RAISE_ERROR(BADARG_ATOM); + } + } term t = argv[0]; int64_t acc = 0; @@ -2531,22 +2552,21 @@ static term nif_erlang_list_to_integer_1(Context *ctx, int argc, term argv[]) while (term_is_nonempty_list(t)) { term head = term_get_list_head(t); - VALIDATE_VALUE(head, term_is_integer); - avm_int_t c = term_to_int(head); - if (UNLIKELY((c < '0') || (c > '9'))) { + avm_int_t digit = to_digit_index(c); + if (UNLIKELY(digit == -1 || digit >= base)) { RAISE_ERROR(BADARG_ATOM); } - //TODO: fix this - if (acc > INT64_MAX / 10) { + // TODO: fix this + if (acc > INT64_MAX / base) { // overflow error is not standard, but we need it since we are running on an embedded device RAISE_ERROR(OVERFLOW_ATOM); } - acc = (acc * 10) + (c - '0'); + acc = (acc * base) + digit; digits++; t = term_get_list_tail(t); if (!term_is_list(t)) { diff --git a/src/libAtomVM/nifs.gperf b/src/libAtomVM/nifs.gperf index 0ca5f61c6..180e24ee8 100644 --- a/src/libAtomVM/nifs.gperf +++ b/src/libAtomVM/nifs.gperf @@ -73,6 +73,7 @@ erlang:integer_to_list/2, &integer_to_list_nif erlang:link/1, &link_nif erlang:list_to_binary/1, &list_to_binary_nif erlang:list_to_integer/1, &list_to_integer_nif +erlang:list_to_integer/2, &list_to_integer_nif erlang:list_to_float/1, &list_to_float_nif erlang:list_to_tuple/1, &list_to_tuple_nif erlang:iolist_size/1, &iolist_size_nif diff --git a/tests/erlang_tests/test_list_to_integer.erl b/tests/erlang_tests/test_list_to_integer.erl index d01a17999..e37851d05 100644 --- a/tests/erlang_tests/test_list_to_integer.erl +++ b/tests/erlang_tests/test_list_to_integer.erl @@ -23,10 +23,16 @@ -export([start/0, sum_integers/2, append_0/1]). start() -> - sum_integers(append_0("10"), "-1") + safe_list_to_integer("--") - safe_list_to_integer(nan) + - safe_list_to_integer("+10") - 10 + safe_list_to_integer("-") - 5 + safe_list_to_integer("+") - - 5 + - safe_list_to_integer("") - 5. + sum_integers(append_0("10"), "-1") + + safe_list_to_integer("--") - 5 + + safe_list_to_integer(nan) - 5 + + safe_list_to_integer("+10") - 10 + + safe_list_to_integer("-") - 5 + + safe_list_to_integer("+") - 5 + + safe_list_to_integer("") - 5 + + safe_list_to_integer("0a", 16) - 10 + + safe_list_to_integer("-0a", 16) + 10 + + safe_list_to_integer("1010", 2) - 10. append_0(L) -> L ++ "0". @@ -35,7 +41,9 @@ sum_integers(A, B) -> list_to_integer(A) + list_to_integer(B). safe_list_to_integer(A) -> - try list_to_integer(A) of + safe_list_to_integer(A, 10). +safe_list_to_integer(A, Base) -> + try list_to_integer(A, Base) of AnyValue -> AnyValue catch error:badarg -> From a9375e7c8b626ee8b606dba83b5a2aab90380c26 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Fri, 11 Oct 2024 22:22:06 +0200 Subject: [PATCH 19/43] ESP32 UART: fix warnings Fix unused vars warning. Signed-off-by: Davide Bettio --- src/platforms/esp32/components/avm_builtins/uart_driver.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/platforms/esp32/components/avm_builtins/uart_driver.c b/src/platforms/esp32/components/avm_builtins/uart_driver.c index ea6a1ebc9..22f9ed398 100644 --- a/src/platforms/esp32/components/avm_builtins/uart_driver.c +++ b/src/platforms/esp32/components/avm_builtins/uart_driver.c @@ -355,7 +355,6 @@ static void uart_driver_do_write(Context *ctx, GenMessage gen_message) term pid = gen_message.pid; term ref = gen_message.ref; - term cmd = term_get_tuple_element(msg, 0); term data = term_get_tuple_element(msg, 1); size_t buffer_size; @@ -400,7 +399,6 @@ static void uart_driver_do_close(Context *ctx, GenMessage gen_message) { GlobalContext *glb = ctx->global; struct UARTData *uart_data = ctx->platform_data; - term msg = gen_message.req; term pid = gen_message.pid; term ref = gen_message.ref; From b44e4fa2f83896f4dd58d37bbbaaf94ddaaf4451 Mon Sep 17 00:00:00 2001 From: Peter M Date: Fri, 11 Oct 2024 13:22:13 +0200 Subject: [PATCH 20/43] Add esp:mount/umount to programmers-guide.md Adds a Storage section to the Programmers guide, with the recently added esp:mount/umount https://github.com/atomvm/AtomVM/pull/1289 Signed-off-by: Peter M --- doc/src/programmers-guide.md | 76 ++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/doc/src/programmers-guide.md b/doc/src/programmers-guide.md index 3e7d6be9e..3e466ed33 100644 --- a/doc/src/programmers-guide.md +++ b/doc/src/programmers-guide.md @@ -1124,6 +1124,82 @@ NVS entries are currently stored in plaintext and are not encrypted. Applicatio sensitive security information, such as account passwords, are stored in NVS storage. ``` +### Storage + +AtomVM provides support for mounting and unmounting storage on ESP32 devices, such as SD cards or internal flash memory. This functionality is accessible through the [`esp:mount/4`](./apidocs/erlang/eavmlib/esp.md#mount-4) and [`esp:umount/1`](./apidocs/erlang/eavmlib/esp.md#umount-1) functions. + +#### Mounting MMC SD card + +To mount a MMC SD card, use the `esp:mount/4` function: + +```erlang +case esp:mount("sdmmc", "/sdcard", fat, []) of + {ok, MountedRef} -> + io:format("SD card mounted successfully~n"), + {ok, MountedRef}; + {error, Reason} -> + io:format("Failed to mount SD card: ~p~n", [Reason]), + {error, Reason} +end. +``` + +#### Mounting SPI SD card + +To mount a SPI SD card, first create a SPI instance configured for your specific board, then use the `esp:mount/4` function: + +```erlang +SPIConfig = [ + {bus_config, [ + {miso, 19}, + {mosi, 23}, + {sclk, 18}, + {peripheral, "spi3"} + ]}], +SPI = spi:open(SPIConfig), +case esp:mount("sdspi", "/sdcard", fat, [{spi_host, SPI}, {cs, 5}]) of + {ok, MountedRef} -> + io:format("SD card mounted successfully~n"), + {ok, MountedRef}; + {error, Reason} -> + io:format("Failed to mount SD card: ~p~n", [Reason]), + {error, Reason} +end. +``` + +#### Mounting internal flash + +To mount internal flash, use the `esp:mount/4` function: + +```erlang +case esp:mount("/dev/partition/by-name/partition_name", "/test", fat, []) of + {ok, MountedRef} -> + io:format("Flash mounted successfully~n"), + {ok, MountedRef}; + {error, Reason} -> + io:format("Failed to mount partition: ~p~n", [Reason]), + {error, Reason} +end. +``` + +#### Unmounting Storage + +To unmount a previously mounted storage device, use the `esp:umount/1` function, with the reference returned from `esp:mount/4`: + +```erlang +case esp:umount(MountedRef) of + ok -> + io:format("Storage unmounted successfully~n"); + {error, Reason} -> + io:format("Failed to unmount storage: ~p~n", [Reason]) +end. +``` + +These functions allow you to work with external storage devices or partitions on your ESP32, enabling you to read from and write to files on the mounted filesystem. This can be particularly useful for applications that need to store or access large amounts of data that don't fit in the device's main memory or non-volatile storage. + +```{important} +Remember to properly unmount any mounted filesystems before powering off or resetting the device to prevent data corruption. +``` + ### Restart and Deep Sleep You can use the [`esp:restart/0`](./apidocs/erlang/eavmlib/esp.md#restart0) function to immediately restart the ESP32 device. This function does not return a value. From 7b6abf17c770f96ab3fcb1f6ec53539e72db7fd8 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Sat, 12 Oct 2024 18:50:32 +0200 Subject: [PATCH 21/43] CI: fix esp32-mkimage.yaml GitHub workflow It was making Elixir images without Elixir. Also add an additional debug message to mkimage.erl. Signed-off-by: Davide Bettio --- .github/workflows/esp32-mkimage.yaml | 4 ++-- src/platforms/esp32/tools/mkimage.erl | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/esp32-mkimage.yaml b/.github/workflows/esp32-mkimage.yaml index ff59d0e2f..001a15d69 100644 --- a/.github/workflows/esp32-mkimage.yaml +++ b/.github/workflows/esp32-mkimage.yaml @@ -142,8 +142,8 @@ jobs: ./mkimage.sh else FLAVOR_SUFFIX=$(echo "${{ matrix.flavor }}" | sed 's/-//g') - BOOT_FILE="build/libs/esp32boot/${FLAVOR_SUFFIX}_esp32boot.avm" - ./mkimage.sh --boot="$BOOT_FILE" + BOOT_FILE="../../../../build/libs/esp32boot/${FLAVOR_SUFFIX}_esp32boot.avm" + ./mkimage.sh --boot "$BOOT_FILE" mv atomvm-${{ matrix.soc }}.img atomvm-${{ matrix.soc }}${{ matrix.flavor }}.img fi ls -l *.img diff --git a/src/platforms/esp32/tools/mkimage.erl b/src/platforms/esp32/tools/mkimage.erl index fd9d73ed2..e6518ee5d 100644 --- a/src/platforms/esp32/tools/mkimage.erl +++ b/src/platforms/esp32/tools/mkimage.erl @@ -132,6 +132,7 @@ get_build_dir(Opts, RootDir) -> %% @private mkimage(RootDir, BuildDir, BootFile, OutputFile, Segments) -> io:format("Writing output to ~s~n", [OutputFile]), + io:format("boot file is ~s~n", [BootFile]), io:format("=============================================~n"), case file:open(OutputFile, [write, binary]) of {ok, Fout} -> From 451540b5e58b5d1f62c131a2c685ea8672802d04 Mon Sep 17 00:00:00 2001 From: Winford Date: Sat, 12 Oct 2024 07:57:28 -0700 Subject: [PATCH 22/43] Emergengy documentation build fix Recent CI updates bumped to the latest OTP-26 which has led to documentation build faulures. These changes temporarily revert to OTP-25 for building docs. Signed-off-by: Winford --- .github/workflows/build-docs.yaml | 8 ++++---- .github/workflows/publish-docs.yaml | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/build-docs.yaml b/.github/workflows/build-docs.yaml index 499eaa72c..87ce8908a 100644 --- a/.github/workflows/build-docs.yaml +++ b/.github/workflows/build-docs.yaml @@ -52,8 +52,8 @@ jobs: os: [ ubuntu-24.04 ] # The type of runner that the job will run on runs-on: ${{ matrix.os }} - # Documentation currently fails to build with OTP-27 - container: erlang:26 + # Documentation currently fails to build with OTP-27 and recent OTP-26. + container: erlang:25 # Steps represent a sequence of tasks that will be executed as part of the job steps: @@ -61,7 +61,7 @@ jobs: - name: Install Deps run: | apt update -y - DEBIAN_FRONTEND=noninteractive apt install -y git cmake doxygen graphviz python3-pip python3-virtualenv python3.11-venv python3-setuptools python3-stemmer wget + DEBIAN_FRONTEND=noninteractive apt install -y git cmake doxygen graphviz python3-pip python3-virtualenv python3-setuptools python3-stemmer wget - uses: actions/cache@v4 id: sphinx-cache @@ -72,7 +72,7 @@ jobs: - name: Install Sphinx if: ${{ steps.sphinx-cache.outputs.cache-hit != 'true' }} run: | - python3 -m venv /home/runner/python-env/sphinx + virtualenv /home/runner/python-env/sphinx . /home/runner/python-env/sphinx/bin/activate python3 -m pip install sphinx python3 -m pip install myst-parser diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index e64a7293a..54569343d 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -63,7 +63,7 @@ jobs: - name: Install Sphinx if: ${{ steps.sphinx-cache.outputs.cache-hit != 'true' }} run: | - python3 -m venv /home/runner/python-env/sphinx + virtualenv /home/runner/python-env/sphinx . /home/runner/python-env/sphinx/bin/activate python3 -m pip install sphinx python3 -m pip install myst-parser @@ -76,7 +76,7 @@ jobs: - uses: erlef/setup-beam@v1 with: - otp-version: "26" + otp-version: "25" elixir-version: "1.17" hexpm-mirrors: | https://builds.hex.pm From 00d1ef618557a70baf1e77f30467040373e75b2d Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Sat, 12 Oct 2024 23:23:46 +0200 Subject: [PATCH 23/43] Fix AVMPort Use instead `port` module rather than implementing it again from scratch. Signed-off-by: Davide Bettio --- libs/exavmlib/lib/AVMPort.ex | 31 ++----------------------------- 1 file changed, 2 insertions(+), 29 deletions(-) diff --git a/libs/exavmlib/lib/AVMPort.ex b/libs/exavmlib/lib/AVMPort.ex index b55b2c8f7..12a333e7d 100644 --- a/libs/exavmlib/lib/AVMPort.ex +++ b/libs/exavmlib/lib/AVMPort.ex @@ -31,39 +31,12 @@ defmodule AVMPort do @spec call(pid(), term()) :: term() def call(pid, message) do - case :erlang.is_process_alive(pid) do - false -> - {:error, :noproc} - - true -> - ref = :erlang.make_ref() - send(pid, {self(), ref, message}) - - receive do - :out_of_memory -> :out_of_memory - {^ref, reply} -> reply - end - end + :port.call(pid, message) end @spec call(pid(), term(), non_neg_integer()) :: term() def call(pid, message, timeoutMs) do - case :erlang.is_process_alive(pid) do - false -> - {:error, :noproc} - - true -> - ref = :erlang.make_ref() - send(pid, {self(), ref, message}) - - receive do - :out_of_memory -> :out_of_memory - {^ref, reply} -> reply - after - timeoutMs -> - {:error, :timeout} - end - end + :port.call(pid, message, timeoutMs) end @spec open(term(), list()) :: pid() From fcf97638803d804919a39a9013b7bd00f2111887 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Sat, 12 Oct 2024 23:25:33 +0200 Subject: [PATCH 24/43] Elixir HelloWorld: fix warning Use ~c"" for charlists instead of ''. Signed-off-by: Davide Bettio --- examples/elixir/HelloWorld.ex | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/elixir/HelloWorld.ex b/examples/elixir/HelloWorld.ex index d422906b1..657cfc537 100644 --- a/examples/elixir/HelloWorld.ex +++ b/examples/elixir/HelloWorld.ex @@ -25,7 +25,7 @@ defmodule HelloWorld do def start() do Console.print("Hello World\n") Console.puts("Console.puts() and Console.print() work with binary ") - Console.puts('or charlist strings.\n') + Console.puts(~c"or charlist strings.\n") Console.flush() end end From 463268d43754d0fad4116a5dcde65e60a5d6bee0 Mon Sep 17 00:00:00 2001 From: Kevin Schweikert <54439512+kevinschweikert@users.noreply.github.com> Date: Sun, 13 Oct 2024 09:03:07 +0200 Subject: [PATCH 25/43] Update code to use .uf2 files Signed-off-by: Kevin Schweikert --- doc/src/getting-started-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/getting-started-guide.md b/doc/src/getting-started-guide.md index ec0ffcddf..df1d50325 100644 --- a/doc/src/getting-started-guide.md +++ b/doc/src/getting-started-guide.md @@ -353,7 +353,7 @@ total 16 -rwxrwxrwx 1 joe staff 241 Sep 5 2008 INDEX.HTM* -rwxrwxrwx 1 joe staff 62 Sep 5 2008 INFO_UF2.TXT* -$ cp ~/Downloads/atomvmlib-v0.6.0.avm /Volumes/RPI-RP2/. +$ cp ~/Downloads/atomvmlib-v0.6.0.uf2 /Volumes/RPI-RP2/. ``` ... and again, at this point, the device will auto-unmount. From 7d5074b8bd60ac622fda7fa6dd3ceadf0f53fbe1 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Sun, 13 Oct 2024 12:52:31 +0200 Subject: [PATCH 26/43] CI: build-and-test: disable OTP master A recent change made impossible to use beam files compiled with OTP master, so let's disable it. Signed-off-by: Davide Bettio --- .github/workflows/build-and-test.yaml | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/.github/workflows/build-and-test.yaml b/.github/workflows/build-and-test.yaml index 74047161e..b2440c9d1 100644 --- a/.github/workflows/build-and-test.yaml +++ b/.github/workflows/build-and-test.yaml @@ -197,12 +197,14 @@ jobs: cflags: "" elixir_version: "1.14" - # master/main version of OTP/Elixir - - os: "ubuntu-24.04" - cc: "cc" - cxx: "c++" - otp: "master" - elixir_version: "main" +# TODO: enable master again +# master will not work until we don't adapt to atom table changes +# # master/main version of OTP/Elixir +# - os: "ubuntu-24.04" +# cc: "cc" +# cxx: "c++" +# otp: "master" +# elixir_version: "main" # Additional default compiler builds - os: "ubuntu-20.04" From 9624f63667bd6fb547d596e675dc84e2337a3893 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Sun, 13 Oct 2024 15:20:50 +0200 Subject: [PATCH 27/43] externalterm: add externalterm_to_term_copy Works like externalterm_to_term, but it makes a copy of data stored in the buffer, so it can be safely used from NIFs. Signed-off-by: Davide Bettio --- CHANGELOG.md | 1 + src/libAtomVM/externalterm.c | 6 ++++++ src/libAtomVM/externalterm.h | 16 ++++++++++++++++ 3 files changed, 23 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f32b25eb..19d8e22f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,7 @@ also non string parameters (e.g. `Enum.join([1, 2], ",")` - Make external term serialize functions available without using `externalterm_to_binary` so terms can be written directly to a buffer. - Support for `erlang:list_to_integer/2` +- Add `externalterm_to_term_copy` that can be safely used from NIFs taking temporary buffers ### Changed diff --git a/src/libAtomVM/externalterm.c b/src/libAtomVM/externalterm.c index 0936ad6b0..186a5591e 100644 --- a/src/libAtomVM/externalterm.c +++ b/src/libAtomVM/externalterm.c @@ -133,6 +133,12 @@ term externalterm_to_term(const void *external_term, size_t size, Context *ctx, return externalterm_to_term_internal(external_term, size, ctx, opts, &bytes_read, false); } +term externalterm_to_term_copy(const void *external_term, size_t size, Context *ctx, ExternalTermOpts opts) +{ + size_t bytes_read = 0; + return externalterm_to_term_internal(external_term, size, ctx, opts, &bytes_read, true); +} + enum ExternalTermResult externalterm_from_binary(Context *ctx, term *dst, term binary, size_t *bytes_read) { if (!term_is_binary(binary)) { diff --git a/src/libAtomVM/externalterm.h b/src/libAtomVM/externalterm.h index 6a4b363fb..a1c7082d1 100644 --- a/src/libAtomVM/externalterm.h +++ b/src/libAtomVM/externalterm.h @@ -66,6 +66,22 @@ typedef enum term externalterm_to_term( const void *external_term, size_t size, Context *ctx, ExternalTermOpts opts); +/** + * @brief Gets a term from external term data, and makes a copy of all data. + * + * @details Deserialize an external term from external format and returns a term. + * @param external_term the external term that will be deserialized. + * @param size to allocate for term. + * @param ctx the context that owns the memory that will be allocated. + * @param opts if non-zero, use a heap fragment to store the generated + * terms. Otherwise, use the heap in the provided context. Note that when using the + * context heap, this function may call the GC, if there is insufficient space to + * store the generated terms. + * @returns a term. + */ +term externalterm_to_term_copy( + const void *external_term, size_t size, Context *ctx, ExternalTermOpts opts); + /** * @brief Create a term from a binary. * From 69ee6a6ff5c9f2fd583a2efd23c047c9436d56d8 Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 11:42:30 -0700 Subject: [PATCH 28/43] Pin OTP to specific version for documentation workflows Building and publishing documentation is failing again, let's try pinning to a specific OTP version known to be capable of building the documentation. Signed-off-by: Winford --- .github/workflows/build-docs.yaml | 2 +- .github/workflows/publish-docs.yaml | 16 ++++------------ 2 files changed, 5 insertions(+), 13 deletions(-) diff --git a/.github/workflows/build-docs.yaml b/.github/workflows/build-docs.yaml index 87ce8908a..799db582d 100644 --- a/.github/workflows/build-docs.yaml +++ b/.github/workflows/build-docs.yaml @@ -53,7 +53,7 @@ jobs: # The type of runner that the job will run on runs-on: ${{ matrix.os }} # Documentation currently fails to build with OTP-27 and recent OTP-26. - container: erlang:25 + container: erlang:26.0.2 # Steps represent a sequence of tasks that will be executed as part of the job steps: diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index 54569343d..e0e41fdef 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -44,6 +44,7 @@ jobs: os: [ ubuntu-24.04 ] # The type of runner that the job will run on runs-on: ${{ matrix.os }} + container: erlang:26.0.2 # Steps represent a sequence of tasks that will be executed as part of the job steps: @@ -51,14 +52,14 @@ jobs: - name: Install Deps run: | - sudo apt update -y - DEBIAN_FRONTEND=noninteractive sudo apt install -y git cmake doxygen graphviz python3-pip python3-virtualenv python3-setuptools python3-stemmer wget + apt update -y + DEBIAN_FRONTEND=noninteractive apt install -y git cmake doxygen graphviz python3-pip python3-virtualenv python3-setuptools python3-stemmer wget - uses: actions/cache@v4 id: sphinx-cache with: path: /home/runner/python-env/sphinx - key: ${{ matrix.os }}-sphinx-install + key: ${{ matrix.os }}-${{ job.container.id }}-sphinx-install - name: Install Sphinx if: ${{ steps.sphinx-cache.outputs.cache-hit != 'true' }} @@ -74,15 +75,6 @@ jobs: python3 -m pip install breathe python3 -m pip install pygments - - uses: erlef/setup-beam@v1 - with: - otp-version: "25" - elixir-version: "1.17" - hexpm-mirrors: | - https://builds.hex.pm - https://repo.hex.pm - https://cdn.jsdelivr.net/hex - - name: Install rebar3 working-directory: /tmp run: | From a0528f7220061a7ef579a61c44ea387a680e3719 Mon Sep 17 00:00:00 2001 From: Peter M Date: Sun, 13 Oct 2024 22:31:43 +0200 Subject: [PATCH 29/43] Better message after wifi config (esp32devmode) I got confused as nothing was happening. I assumed device was restarting and configuration going into use. This makes it clear for a beginner and improves DX. Signed-off-by: Peter M --- libs/esp32devmode/src/esp32devmode.erl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libs/esp32devmode/src/esp32devmode.erl b/libs/esp32devmode/src/esp32devmode.erl index b3c7bf6b4..91ae7098e 100644 --- a/libs/esp32devmode/src/esp32devmode.erl +++ b/libs/esp32devmode/src/esp32devmode.erl @@ -302,7 +302,7 @@ handle_req("POST", [], Conn) -> "\n" " \n" "

Configuration

\n" - "

Configured.

\n" + "

Configured, restart device to apply wifi configuration.

\n" " \n" "" >>, From 11f26722666807263cb68ec5392093013e83f15e Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 14:46:06 -0700 Subject: [PATCH 30/43] Check for content change before publishing Adds a check to make sure there are acutally changes to be commited and pushed. This is needed to address a situation where the publish docs workflow is triggered, but there are no changes to the documentation content. Changes `cancel-in-progress` to false, to avoid missing publishing documentation, when another publish workflow is triggerd by a commit that might not have any changes to the documentation to add. Signed-off-by: Winford --- .github/workflows/publish-docs.yaml | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index e0e41fdef..1e594bb2d 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -29,7 +29,7 @@ on: concurrency: group: ${{ github.workflow }}-${{ github.ref_name }} - cancel-in-progress: true + cancel-in-progress: false env: AVM_DOCS_NAME: ${{ github.ref_name }} @@ -119,17 +119,20 @@ jobs: git checkout Production git config --local user.email "atomvm-doc-bot@users.noreply.github.com" git config --local user.name "AtomVM Doc Bot" - git add . - git commit -m "Update Documentation" + if [ -n "$(git status --porcelain=v1)" ]; then + git add . + git commit -m "Update Documentation" + fi - name: Push changes if: github.repository == 'atomvm/AtomVM' working-directory: /home/runner/work/AtomVM/AtomVM/www run: | - eval `ssh-agent -t 60 -s` - echo "${{ secrets.PUBLISH_ACTION_KEY }}" | ssh-add - - mkdir -p ~/.ssh/ - ssh-keyscan github.com >> ~/.ssh/known_hosts - git remote add push_dest "git@github.com:atomvm/atomvm_www.git" - git fetch push_dest - git push --set-upstream push_dest Production - + if [ -n "$(git status --porcelain=v1)" ]; then + eval `ssh-agent -t 60 -s` + echo "${{ secrets.PUBLISH_ACTION_KEY }}" | ssh-add - + mkdir -p ~/.ssh/ + ssh-keyscan github.com >> ~/.ssh/known_hosts + git remote add push_dest "git@github.com:atomvm/atomvm_www.git" + git fetch push_dest + git push --set-upstream push_dest Production + fi From ce0c5fe79578ba2bac7be797328d70d861973484 Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 15:20:40 -0700 Subject: [PATCH 31/43] Update build instructions for stm32 Changes the device names to lowercase, to match how the deivce name should be passed to CMake during configuration. Signed-off-by: Winford --- doc/src/build-instructions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/src/build-instructions.md b/doc/src/build-instructions.md index b6eba42cc..13677604e 100644 --- a/doc/src/build-instructions.md +++ b/doc/src/build-instructions.md @@ -668,7 +668,7 @@ $ make ### Changing the target device The default build is based on the STM32F4Discovery board chip (`stm32f407vgt6`). If you want to target a different -chip, pass the `-DDEVICE` flag when invoking cmake. For example, to use the BlackPill V2.0, pass `-DDEVICE=STM32F411CEU6`. At this time any `STM32F4` or `STM32F7` device with 512KB or more of on package flash should work with AtomVM. If an unsupported device is passed with the `DEVICE` parameter the configuration will fail. For devices with either 512KB or 768KB of flash the available application flash space will be limited to 128KB. Devices with only 512KB of flash may also suffer from slightly reduced performance because the compiler must optimize for size rather than performance. +chip, pass the `-DDEVICE` flag when invoking cmake. For example, to use the BlackPill V2.0, pass `-DDEVICE=stm32f411ceu6`. At this time any `STM32F4` or `STM32F7` device with 512KB or more of on package flash should work with AtomVM. If an unsupported device is passed with the `DEVICE` parameter the configuration will fail. For devices with either 512KB or 768KB of flash the available application flash space will be limited to 128KB. Devices with only 512KB of flash may also suffer from slightly reduced performance because the compiler must optimize for size rather than performance. ```{attention} For devices with only 512KB of flash the application address is different and must be adjusted when flashing your @@ -678,7 +678,7 @@ devices is `0x8060000`. ### Configuring the Console -The default build for any `DEVICE` will use `USART2` and output will be on `PA2`. This default will work well for most `Discovery` and generic boards that do not have an on-board TTL to USB-COM support (including the `STM32F411CEU6` A.K.A. `BlackPill V2.0`). For `Nucleo` boards that do have on board UART to USB-COM support you may pass the `cmake` parameter `-DBOARD=nucleo` to have the correct USART and TX pins configured automatically. The `Nucleo-144` series use `USART3` and `PD8`, while the supported `Nucleo-64` boards use `USART2`, but passing the `BOARD` parameter along with `DEVICE` will configure the correct `USART` for your model. If any other boards are discovered to have on board USB UART support pull requests, or opening issues with the details, are more than welcome. +The default build for any `DEVICE` will use `USART2` and output will be on `PA2`. This default will work well for most `Discovery` and generic boards that do not have an on-board TTL to USB-COM support (including the `stm32f411ceu6` A.K.A. `BlackPill V2.0`). For `Nucleo` boards that do have on board UART to USB-COM support you may pass the `cmake` parameter `-DBOARD=nucleo` to have the correct USART and TX pins configured automatically. The `Nucleo-144` series use `USART3` and `PD8`, while the supported `Nucleo-64` boards use `USART2`, but passing the `BOARD` parameter along with `DEVICE` will configure the correct `USART` for your model. If any other boards are discovered to have on board USB UART support pull requests, or opening issues with the details, are more than welcome. Example to configure a `NUCLEO-F429ZI`: From 412232a632dc22c8e664a220dba39b83f065f99c Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 16:53:42 -0700 Subject: [PATCH 32/43] Fix error in publish docs workflow check The previos fix would always skip publishing. This sets an environmental output in the commit files stage that can be checked before pushing changes. Signed-off-by: Winford --- .github/workflows/publish-docs.yaml | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index 1e594bb2d..23efde316 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -113,6 +113,7 @@ jobs: make GitHub_CI_Publish_Docs - name: Commit files + id: commit_files if: github.repository == 'atomvm/AtomVM' working-directory: /home/runner/work/AtomVM/AtomVM/www run: | @@ -122,17 +123,16 @@ jobs: if [ -n "$(git status --porcelain=v1)" ]; then git add . git commit -m "Update Documentation" + echo "PUBLISH=true" >> "$GITHUB_OUTPUT" fi - name: Push changes - if: github.repository == 'atomvm/AtomVM' + if: github.repository == 'atomvm/AtomVM' && steps.commit_files.outputs.PUBLISH == 'true' working-directory: /home/runner/work/AtomVM/AtomVM/www run: | - if [ -n "$(git status --porcelain=v1)" ]; then - eval `ssh-agent -t 60 -s` - echo "${{ secrets.PUBLISH_ACTION_KEY }}" | ssh-add - - mkdir -p ~/.ssh/ - ssh-keyscan github.com >> ~/.ssh/known_hosts - git remote add push_dest "git@github.com:atomvm/atomvm_www.git" - git fetch push_dest - git push --set-upstream push_dest Production - fi + eval `ssh-agent -t 60 -s` + echo "${{ secrets.PUBLISH_ACTION_KEY }}" | ssh-add - + mkdir -p ~/.ssh/ + ssh-keyscan github.com >> ~/.ssh/known_hosts + git remote add push_dest "git@github.com:atomvm/atomvm_www.git" + git fetch push_dest + git push --set-upstream push_dest Production From 94643aab89cbd3895c090ecdde4da402a53fe966 Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 17:14:43 -0700 Subject: [PATCH 33/43] Add notes to UPDATING.md Adds notes about not currently suporting STM32 devices with only 512k of flash, and the just discovered `i2c:write_bytes/2` bug on ESP32 platform. Signed-off-by: Winford --- UPDATING.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/UPDATING.md b/UPDATING.md index d914fe423..f2b604f60 100644 --- a/UPDATING.md +++ b/UPDATING.md @@ -13,6 +13,10 @@ - ESP32: partitioning schema for Elixir flavor is different, so app offset has been changed for Elixir images. Make sure to use `0x250000` as offset in your mix.exs or when performing manual flashing. +- ESP32 a bug was discovered in `i2c:write_bytes/2` that has not been fixed yet. Writing bytes +sequentally using `i2c:write_byte/2` still works as a temporary workaround. +- STM32 devices with 512k of flash are not supported in this release, due to lack of +flash space. Support may return in a future release. ## v0.6.0-beta.1 -> v0.6.0-rc.0 From b306712f087dde519d12fc8dbdd979d5b0f60dcf Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 17:24:29 -0700 Subject: [PATCH 34/43] Add missing paths to documentation workflow triggers Adds missing file triggers for files included in the documentation. Signed-off-by: Winford --- .github/workflows/build-docs.yaml | 8 ++++++++ .github/workflows/publish-docs.yaml | 4 ++++ 2 files changed, 12 insertions(+) diff --git a/.github/workflows/build-docs.yaml b/.github/workflows/build-docs.yaml index 799db582d..34cb4a63f 100644 --- a/.github/workflows/build-docs.yaml +++ b/.github/workflows/build-docs.yaml @@ -23,6 +23,10 @@ on: - 'doc/**' - 'libs/**' - 'src/libAtomVM/**' + - 'UPDATING.md' + - 'CONTRIBUTING.md' + - 'CHANGELOG.md' + - 'CODE_OF_CONDUCT.md' push: repositories: - '!atomvm/AtomVM' @@ -32,6 +36,10 @@ on: - 'doc/**' - 'libs/**' - 'src/libAtomVM/**' + - 'UPDATING.md' + - 'CONTRIBUTING.md' + - 'CHANGELOG.md' + - 'CODE_OF_CONDUCT.md' # Allows you to run this workflow manually from the Actions tab workflow_dispatch: diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index 23efde316..0d282d03b 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -23,6 +23,10 @@ on: - 'doc/**' - 'libs/**' - 'src/libAtomVM/**' + - 'UPDATING.md' + - 'CONTRIBUTING.md' + - 'CHANGELOG.md' + - 'CODE_OF_CONDUCT.md' # Allows you to run this workflow manually from the Actions tab workflow_dispatch: From be10e50a7e301087e47a86ad6f89ae33674bcfa8 Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 15:25:57 -0700 Subject: [PATCH 35/43] Fix bug in STM32 GPIO driver There was an incorrect case of passing the pointer to the atom string, instead of the atom index itself in the gpiodriver_set_int function. Signed-off-by: Winford --- src/platforms/stm32/src/lib/gpio_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/platforms/stm32/src/lib/gpio_driver.c b/src/platforms/stm32/src/lib/gpio_driver.c index 1a8fd5ff0..27fcec4b3 100644 --- a/src/platforms/stm32/src/lib/gpio_driver.c +++ b/src/platforms/stm32/src/lib/gpio_driver.c @@ -812,7 +812,7 @@ static term gpiodriver_set_int(Context *ctx, int32_t target_pid, term cmd) term pid = term_get_tuple_element(cmd, 3); if (UNLIKELY(!term_is_pid(pid) && !term_is_atom(pid))) { AVM_LOGE(TAG, "Invalid listener parameter, must be a pid() or registered process!"); - return create_pair(ctx, ERROR_ATOM, invalid_listener_atom); + return create_pair(ctx, ERROR_ATOM, globalcontext_make_atom(ctx->global, invalid_listener_atom)); } if (term_is_pid(pid)) { target_local_pid = term_to_local_process_id(pid); From bfd80d5b5b5d52ee350969622f78e7e415f394cd Mon Sep 17 00:00:00 2001 From: Winford Date: Sun, 13 Oct 2024 22:15:34 -0700 Subject: [PATCH 36/43] Use newlib-nano by default for 512k flash STM32 boards For STM32 devices with 512k flash newlib-nano needs to be used otherwise the compiled binary is too large to leave space for user applications. Signed-off-by: Winford --- src/platforms/stm32/cmake/compile-flags.cmake | 1 + 1 file changed, 1 insertion(+) diff --git a/src/platforms/stm32/cmake/compile-flags.cmake b/src/platforms/stm32/cmake/compile-flags.cmake index 1eeb5f6b5..759c0a2f7 100644 --- a/src/platforms/stm32/cmake/compile-flags.cmake +++ b/src/platforms/stm32/cmake/compile-flags.cmake @@ -36,6 +36,7 @@ set(CXX_WARN_FLAGS "${COMMON_WARN_FLAGS}") # Use C and C++ compiler optimizatons for size and speed. if (${CMAKE_FLASH_SIZE} STREQUAL "ROM_512K") set(OPTIMIZE_FLAG "-Os") +set(LINKER_FLAGS "${LINKER_FLAGS} -specs=nano.specs") else() set(OPTIMIZE_FLAG "-O2") endif() From 7908a99c0ae2477b1c67211d17aeb09cf0176f28 Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Mon, 14 Oct 2024 13:30:35 +0200 Subject: [PATCH 37/43] CI: publish-docs: do not make it conditional Always run it, and just skip single commit and push commands. Signed-off-by: Davide Bettio --- .github/workflows/publish-docs.yaml | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index 0d282d03b..bba3fe001 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -124,13 +124,11 @@ jobs: git checkout Production git config --local user.email "atomvm-doc-bot@users.noreply.github.com" git config --local user.name "AtomVM Doc Bot" - if [ -n "$(git status --porcelain=v1)" ]; then - git add . - git commit -m "Update Documentation" - echo "PUBLISH=true" >> "$GITHUB_OUTPUT" - fi + ls -la doc/ + git add . + git diff --exit-code || git commit -m "Update Documentation" - name: Push changes - if: github.repository == 'atomvm/AtomVM' && steps.commit_files.outputs.PUBLISH == 'true' + if: github.repository == 'atomvm/AtomVM' working-directory: /home/runner/work/AtomVM/AtomVM/www run: | eval `ssh-agent -t 60 -s` @@ -139,4 +137,4 @@ jobs: ssh-keyscan github.com >> ~/.ssh/known_hosts git remote add push_dest "git@github.com:atomvm/atomvm_www.git" git fetch push_dest - git push --set-upstream push_dest Production + git diff --exit-code origin/Production || git push --set-upstream push_dest Production From 8ec79124425df31149da9ebc6216a55f12f089ca Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Mon, 14 Oct 2024 18:59:04 +0200 Subject: [PATCH 38/43] CI: pin GitHub ssh keys There is likely some problem related to ssh keys, pin them. Signed-off-by: Davide Bettio --- .github/workflows/publish-docs.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index bba3fe001..92968aab7 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -134,7 +134,9 @@ jobs: eval `ssh-agent -t 60 -s` echo "${{ secrets.PUBLISH_ACTION_KEY }}" | ssh-add - mkdir -p ~/.ssh/ - ssh-keyscan github.com >> ~/.ssh/known_hosts + echo "github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl" > ~/.ssh/known_hosts + echo "github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=" >> ~/.ssh/known_hosts + echo "github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=" >> ~/.ssh/known_hosts git remote add push_dest "git@github.com:atomvm/atomvm_www.git" git fetch push_dest git diff --exit-code origin/Production || git push --set-upstream push_dest Production From 9e623bd54aec6f300499c01f8663cfc5cb0aee9a Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Mon, 14 Oct 2024 19:30:20 +0200 Subject: [PATCH 39/43] CI: fix publish-docs workflow For some reason, even if we add ssh keys to known_hosts, it keeps failing. The failure is related to confirm prompt, that tries to open tty (that is not available on CI). Signed-off-by: Davide Bettio --- .github/workflows/publish-docs.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index 92968aab7..f126f6f0a 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -131,6 +131,7 @@ jobs: if: github.repository == 'atomvm/AtomVM' working-directory: /home/runner/work/AtomVM/AtomVM/www run: | + export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" eval `ssh-agent -t 60 -s` echo "${{ secrets.PUBLISH_ACTION_KEY }}" | ssh-add - mkdir -p ~/.ssh/ @@ -139,4 +140,4 @@ jobs: echo "github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=" >> ~/.ssh/known_hosts git remote add push_dest "git@github.com:atomvm/atomvm_www.git" git fetch push_dest - git diff --exit-code origin/Production || git push --set-upstream push_dest Production + git diff --exit-code push_dest/Production || git push --set-upstream push_dest Production From 0235723b53a92391565a495354ff7b2299026dbd Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Mon, 14 Oct 2024 20:49:29 +0200 Subject: [PATCH 40/43] CI: publish-docs: fix conditional logic git add . makes diff always empty, so compare against last commit. Signed-off-by: Davide Bettio --- .github/workflows/publish-docs.yaml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index f126f6f0a..54ed524f5 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -121,12 +121,13 @@ jobs: if: github.repository == 'atomvm/AtomVM' working-directory: /home/runner/work/AtomVM/AtomVM/www run: | - git checkout Production git config --local user.email "atomvm-doc-bot@users.noreply.github.com" git config --local user.name "AtomVM Doc Bot" ls -la doc/ git add . - git diff --exit-code || git commit -m "Update Documentation" + git diff --exit-code Production || echo "Going to commit" + git diff --exit-code Production || git commit -m "Update Documentation" + git log -1 - name: Push changes if: github.repository == 'atomvm/AtomVM' working-directory: /home/runner/work/AtomVM/AtomVM/www @@ -140,4 +141,5 @@ jobs: echo "github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=" >> ~/.ssh/known_hosts git remote add push_dest "git@github.com:atomvm/atomvm_www.git" git fetch push_dest + git diff --exit-code push_dest/Production || echo "Going to push" git diff --exit-code push_dest/Production || git push --set-upstream push_dest Production From aaeb16758d9c0e525debbb4b394a45d168242ff0 Mon Sep 17 00:00:00 2001 From: Peter M Date: Mon, 14 Oct 2024 23:35:16 +0200 Subject: [PATCH 41/43] Fix compilation USE_USB_SERIAL Compiling with USE_USB_SERIAL for usb console on s2, would fail as init_usb_serial() is defined after it's called. Safe for 0.6.5 inclusion. Signed-off-by: Peter M --- src/platforms/esp32/main/main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/platforms/esp32/main/main.c b/src/platforms/esp32/main/main.c index 45f916e97..88f3f5b89 100644 --- a/src/platforms/esp32/main/main.c +++ b/src/platforms/esp32/main/main.c @@ -41,6 +41,7 @@ // idf.py add-dependency esp_tinyusb // and enable CDC in menu config #ifdef USE_USB_SERIAL +void init_usb_serial(void); #include "tinyusb.h" #include "tusb_cdc_acm.h" #include "tusb_console.h" From 391ae578c047523fcd7ca4f56dcd75b20b3f9e3d Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Mon, 14 Oct 2024 23:20:32 +0200 Subject: [PATCH 42/43] CI: publish-docs: workaround issue Switching to containers changed how paths are handled, making it difficult to debug. There is some kind of magic here, caution when changing this. Signed-off-by: Davide Bettio --- .github/workflows/publish-docs.yaml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/publish-docs.yaml b/.github/workflows/publish-docs.yaml index 54ed524f5..93cd1f8d2 100644 --- a/.github/workflows/publish-docs.yaml +++ b/.github/workflows/publish-docs.yaml @@ -115,7 +115,8 @@ jobs: cmake .. cd doc make GitHub_CI_Publish_Docs - + rm -frv "/__w/AtomVM/AtomVM/www/doc/${{ github.ref_name }}" + cp -av html "/__w/AtomVM/AtomVM/www/doc/${{ github.ref_name }}" - name: Commit files id: commit_files if: github.repository == 'atomvm/AtomVM' @@ -124,6 +125,8 @@ jobs: git config --local user.email "atomvm-doc-bot@users.noreply.github.com" git config --local user.name "AtomVM Doc Bot" ls -la doc/ + git status "doc/${{ github.ref_name }}" + git add "doc/${{ github.ref_name }}" git add . git diff --exit-code Production || echo "Going to commit" git diff --exit-code Production || git commit -m "Update Documentation" From dae330da71bc6d13115ce4887701bfcecb17130a Mon Sep 17 00:00:00 2001 From: Davide Bettio Date: Tue, 15 Oct 2024 01:04:57 +0200 Subject: [PATCH 43/43] Prepare v0.6.5 release Signed-off-by: Davide Bettio --- CHANGELOG.md | 2 +- version.cmake | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19d8e22f5..455899175 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [0.6.5] - Unreleased +## [0.6.5] - 2024-10-15 ### Added diff --git a/version.cmake b/version.cmake index 0de8e7d8e..e9c601316 100644 --- a/version.cmake +++ b/version.cmake @@ -20,4 +20,4 @@ # Please, keep also in sync src/libAtomVM/atomvm_version.h set(ATOMVM_BASE_VERSION "0.6.5") -set(ATOMVM_DEV TRUE) +set(ATOMVM_DEV FALSE)