Squashed 'main/lispBM/' changes from cd6b6e2a..f0c1ec21

f0c1ec21 fixing type
563be0ef adding memory usage statistics that is collected a the end of each garbage collection round.
135593ea making the repl  vescif_print_ctx clearer
ce8cdaa1 more info about symbols involved in error messages. A binding is looked up if it exists.
3494b2d4 Merge branch 'master' of github.com:svenssonjoel/lispbm
ea1a92db update bear command for recent version!
669c2fc4 added an all contexts iterator
2ab1fd60 update REPL Makefile for SDL 32 and 64bit builds
d00f0472 update REPL makefile for 32 and 64bit SDL builds
9ea2659a update to scan-build script
c34ba6fb added some explanatory comments at the different mark implementaions in heap.c
568fe490 Pointer-reversal GC now works with CALL-CC and Lisp-Arrays. Passes all tests
13138b8e small tweak to cppcheck script
a9627b36 update to cppcheck scripts and a number of refactorings
8e335412 update readme
c06ae021 removing some redundant conditions and reducing scope on a number of variables
66a99c0f reducing scope of a tmp variable in lbm_caar and lbm_cadr
f946c98e fix some tests that were a bit broken and reduce code complexity in print_internal and fundamental_numeq
04ca92ee update to run_cppcheck.sh
8a1ff24b adding scripts for running cppcheck on repl and the libraries the repl uses (most of LBM)
e98cdf33 simplify and clearify share_const_array in lbm_c_interop
9764dafd fix index out of bounds and other issues in repl
64e3f367 added tests that differentiate 32/64bit. mostly a check to make sure one is running the version one thinkgs one is running
4e53e2f0 upping version to differentiate master from 0.27.0
4cf588f9 numbered version 0.27.0
2555dcfc Numbered version 0.27.0
59d5012f fix mistake in tests Makefile
c7ad2521 tweaks to test scripts and test makefile
92f4aeef all 64 bit tests pass. update testing MAkefile to simplify running 32 and 64 bit tests in parallell.
020f4226 corrections related to 64bit testing
ad73aaae default memory size if 10K for the repl
9ccbdad3 profiling now works on the vescif repl
13d5dd49 :prof commands now work in the vescif mode of the repl
ad26c58b added support for the :reset command over the vescif in the REPL
ba4da435 :ctxs now output to vesc_tool if in vesc mode
4b644e25 program upload from vesc-tool to LBM repl seems to work
7a11f0a4 towards vesctcp interoperability, program storage
f94d8822 tweak type in repl help
282c9803 Using a commands_printf_lisp when in vescif mode as the lbm print implementation.
795ba718 You can now set the size of the simulated constants memory for the repl
462034ea update makefile for docs to set an appropriate mem size
97257cd5 close some warnings from buffer.c in repl
ee10423b the LBM linux repl allows selecting lbm_memory size
0dba96fe small tweak to how the repl prints help text upon --help flag
7cb31338 added another example to the repl/examples directory
be1af47a remove unused code from env.c/h
5dfe7813 added a size.txt file to keep track of size (as part of a chibios build) of LBM
819f0f0d consoldating 3 different continuations into one
c02e5f22 remove const-symbol-strings

git-subtree-dir: main/lispBM
git-subtree-split: f0c1ec2124b7d5b14eb370302ac4deb2201e63fe

README.md

@@ -17,28 +17,8 @@ The LispBM mascot, Lispy the llama, was created by
 
 ## Want to get involved and help out?
 
-There are lots of interesting things to code on in and around the
-LispBM runtime system. I would love to interact with people who are interested
-in high-level programming languages on microcontrollers (or elsewhere) so please
-do not be shy to make contact. Together we can make great stuff happen ;)
-
-1. Are you interested in microcontrollers and programming languages?
-2. You find it fun to mess around in C code with close to zero comments?
-3. Then join in the fun. Lots to do, so little time!
-4. Poke me by mail bo(dot)joel(dot)svensson(whirly-a)gmail(dot)com.
-
-There are areas where insights and help would matter greatly. Some I can
-think of are:
-
-1. Running LispBM on system simulators.
-2. LispBM efficiency, compilation, byte-code and real time garbage collection.
-3. Code size optimisation. Do more with less, while maintaining performance.
-4. Documentation and testing.
-5. Useful extension libraries.
-6. Lisp scriptable home automation.
-7. Lisp scriptable MIDI sequencers.
-8. Lisp scriptable Audio synthesizers.
-9. ...
+ - Check out the [lispbm.com](http://www.lispbm.com) website.
+ - Join the [discord server](https://discord.gg/urtJUUMnwQ).
 
 ## Getting started 
 

benchmarks/bench_chibi/size.txt

@@ -0,0 +1,3 @@
+   text	   data	    bss	    dec	    hex	filename             date
+ 132044	   2520	 128552	 263116	  403cc	build/lisp_test.elf  oct 26 2024
+ 131916	   2520	 128552	 262988	  4034c	build/lisp_test.elf  oct 27 2024

doc/Makefile

@@ -5,22 +5,22 @@
 all: lbmref.md displayref.md runtimeref.md strange.md efficient.md
 
 doclib.env: doclib.lisp
-	lbm -H 100000 --src="doclib.lisp" --store_env=doclib.env --terminate
+	lbm -H 100000 -M 11 --src="doclib.lisp" --store_env=doclib.env --terminate
 
 lbmref.md: doclib.env lbmref.lisp
-	lbm -H 10000000 --src="lbmref.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
+	lbm -H 10000000 -M 11 --src="lbmref.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
 
 displayref.md: doclib.env displayref.lisp
-	lbm -H 10000000 --src="displayref.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
+	lbm -H 10000000 -M 11 --src="displayref.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
 
 runtimeref.md: doclib.env runtimeref.lisp
-	lbm -H 10000000 --src="runtimeref.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
+	lbm -H 10000000 -M 11 --src="runtimeref.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
 
 strange.md: doclib.env strange.lisp
-	lbm -H 10000000 --src="strange.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
+	lbm -H 10000000 -M 11 --src="strange.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
 
 efficient.md: doclib.env efficient.lisp
-	lbm -H 10000000 --src="efficient.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
+	lbm -H 10000000 -M 11 --src="efficient.lisp" --eval="(render-manual)" --load_env="doclib.env" --terminate
 
 clean:
 	rm -f doclib.env

doc/lbmref.lisp

@@ -2725,16 +2725,16 @@
 
 ;; Flash memory
 
-(define const-symbol-strings
-  (ref-entry "@const-symbol-strings"
-             (list
-              (para (list "`@const-symbol-strings` functionality have been combined with `@const-start` and `@const-end`."
-                          "Now symbols created while in a const block, end up in flash storage."
-                          ))
-              (para (list "~~if `@const-symbol-strings` directive is placed in a file, symbols will be created"
-                          "in flash memory instead of the arrays memory.~~"
-                          ))
-              end)))
+;; (define const-symbol-strings
+;;   (ref-entry "@const-symbol-strings"
+;;              (list
+;;               (para (list "`@const-symbol-strings` functionality have been combined with `@const-start` and `@const-end`."
+;;                           "Now symbols created while in a const block, end up in flash storage."
+;;                           ))
+;;               (para (list "~~if `@const-symbol-strings` directive is placed in a file, symbols will be created"
+;;                           "in flash memory instead of the arrays memory.~~"
+;;                           ))
+;;               end)))
 
 (define const-start
   (ref-entry "@const-start"
@@ -2800,7 +2800,7 @@
                         "reading, there are `@`directives."
                         ))
             'hline
-            const-symbol-strings
+            ;const-symbol-strings
             const-start
             const-end
             flash-move

doc/lbmref.md

@@ -7710,7 +7710,7 @@ Use `self` to obtain the thread-id of the thread in which `self` is evaluated. T
 <td>
 
 ```clj
-3131
+3155
 ```
 
 
@@ -7915,7 +7915,7 @@ The `val-expr` can be observed if the thread exit status is captured using `spaw
 
 
 ```clj
-(exit-ok 186695 kurt-russel)
+(exit-ok 178054 kurt-russel)
 ```
 
 
@@ -8565,18 +8565,6 @@ The `variable_not_bound` symbol is returned when evaluating a variable (symbol)
 Flash memory can be used to store data and functions that are constant. Things can be moved to flash explicitly using the `move-to-flash` function or as part of the reading procedure. To move things automatically to flash during reading, there are `@`directives. 
 
 
----
-
-
-### @const-symbol-strings
-
-`@const-symbol-strings` functionality have been combined with `@const-start` and `@const-end`. Now symbols created while in a const block, end up in flash storage. 
-
-~~if `@const-symbol-strings` directive is placed in a file, symbols will be created in flash memory instead of the arrays memory.~~ 
-
-
-
-
 ---
 
 

doc/runtimeref.md

@@ -1,6 +1,7 @@
 # LispBM Runtime Extensions Reference Manual
 
 The runtime extensions, if present, can be either compiled in a minimal or a full mode. In the minimal mode only `set-eval-quota` is present. Minimal mode is the default when compiling LBM. To get the full mode the `-DFULL_RTS_LIB` flag must be used when compiling. 
+
 ## Environments
 
 

doc/strange.lisp

@@ -15,6 +15,7 @@
 
 
 (def ch-constant
+    (list 
      (section 2 "Constant code and data"
               (list
                (para (list "As LBM is targeting microcontrollers RAM is a limited resource while FLASH can be quite abundant!"
@@ -28,17 +29,20 @@
      (section 3 "Literals are constant in constant blocks"
               (list
 
+	       )
+	      )
      )
-
+  )
+
 
 (define manual
     (list
      (section 1 "The LBM guide to the strange and unexpected"
-              (list ch-intro
-                    ch-constant
+	      (list ch-intro
+		    ch-constant
 		    info
-                    )
-              )
+		    )
+	      )
      )
   )
 

include/env.h

@@ -61,13 +61,6 @@ bool lbm_env_lookup_b(lbm_value *res, lbm_value sym, lbm_value env);
  * \return True on success or false otherwise.
  */
 bool lbm_global_env_lookup(lbm_value *res, lbm_value sym);
-/** Lookup a value in from the global environment.
- *
- * \param sym The key to look for in the environment
- * \param env The environment to search for the key.
- * \return The value bound to key or lbm_enc_sym(SYM_NOT_FOUND).
- */
-lbm_value lbm_env_lookup(lbm_value sym, lbm_value env);
 /** Create a new binding on the environment or replace an old binding.
  *
  * \param env Environment to modify.

include/eval_cps.h

@@ -48,13 +48,13 @@ extern "C" {
 /** The eval_context_t struct represents a lispbm process.
  *
  */
-#define LBM_THREAD_STATE_READY      (uint32_t)0
-#define LBM_THREAD_STATE_BLOCKED    (uint32_t)1
-#define LBM_THREAD_STATE_TIMEOUT    (uint32_t)2
-#define LBM_THREAD_STATE_SLEEPING   (uint32_t)4
-#define LBM_THREAD_STATE_RECV_BL    (uint32_t)8
-#define LBM_THREAD_STATE_RECV_TO    (uint32_t)16
-#define LBM_THREAD_STATE_GC_BIT     (uint32_t)(1 << 31)
+#define LBM_THREAD_STATE_READY      (uint32_t)0u
+#define LBM_THREAD_STATE_BLOCKED    (uint32_t)1u
+#define LBM_THREAD_STATE_TIMEOUT    (uint32_t)2u
+#define LBM_THREAD_STATE_SLEEPING   (uint32_t)4u
+#define LBM_THREAD_STATE_RECV_BL    (uint32_t)8u
+#define LBM_THREAD_STATE_RECV_TO    (uint32_t)16u
+#define LBM_THREAD_STATE_GC_BIT     (uint32_t)(1u << 31)
 
 #define LBM_IS_STATE_TIMEOUT(X) (X & (LBM_THREAD_STATE_TIMEOUT | LBM_THREAD_STATE_RECV_TO))
 #define LBM_IS_STATE_WAKE_UP_WAKABLE(X) (X & (LBM_THREAD_STATE_SLEEPING | LBM_IS_STATE_TIMEOUT(X)))
@@ -300,6 +300,14 @@ bool lbm_unblock_ctx_r(lbm_cid cid);
  * \return True on successfully unblocking. False otherwise.
  */
 bool lbm_unblock_ctx_unboxed(lbm_cid cid, lbm_value unboxed);
+/**  Iterate over ALL contexts and apply function on each context.
+ *   This includes the currently running context, if there is one.
+ *
+ * \param f Function to apply to each context.
+ * \param arg1 Pointer argument that can be used to convey information back to user.
+ * \param arg2 Same as above.
+ */
+void lbm_all_ctxs_iterator(ctx_fun f, void *arg1, void *arg2);
 /**  Iterate over all ready contexts and apply function on each context.
  *
  * \param f Function to apply to each context.

include/lbm_memory.h

@@ -142,6 +142,14 @@ lbm_uint lbm_memory_num_words(void);
  * \return The number of free words in the symbols and arrays memory.
  */
 lbm_uint lbm_memory_num_free(void);
+/** Get the maximum of memory usage. Divide this value
+ *
+ * \return Maximal memory usage.
+ */
+lbm_uint lbm_memory_maximum_used(void);
+/** Update memory usage statistics. called by GC automatically
+  */
+void lbm_memory_update_min_free(void);
 /** Find the length of the longest run of consecutire free indices
  *  in the LBM memory.
  */

include/lbm_version.h

@@ -1,5 +1,5 @@
 /*
-    Copyright 2022, 2023 Joel Svensson  [email protected]
+    Copyright 2022 - 2024 Joel Svensson  [email protected]
 
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
@@ -25,16 +25,21 @@ extern "C" {
 #endif
 
 /** LBM major version */
-#define LBM_MAJOR_VERSION 0
+#define LBM_MAJOR_VERSION 0u
 /** LBM minor version */
-#define LBM_MINOR_VERSION 26
+#define LBM_MINOR_VERSION 27u
 /** LBM patch revision */
-#define LBM_PATCH_VERSION 0
-
-#define LBM_VERSION_STRING "0.26.0"
+#define LBM_PATCH_VERSION 1u
 
+#define LBM_VERSION_STRING "0.27.1"
 
 /*! \page changelog Changelog
+NOV 5 2024: Version 0.27.0
+  - A number of important bugfixes. A write out of bounds among the most important.
+  - Addition of integer division operation (By Rasmus S)
+  - A lot of refactoring for readability of code and reduce code complexity.
+  - A defragmentable memory (compactible memory area).
+  - Vesc interfacing in the LBM REPL.
 AUG 6 2024: Version 0.26.0
   - Bugfix related to restargs.
   - coverage report in relation to the unit tests.

include/tokpar.h

@@ -37,7 +37,6 @@
 #define TOKCLOSECURL    13u     // "}"
 #define TOKCONSTSTART   14u     // "@const-start"
 #define TOKCONSTEND     15u     // "@const-end"
-#define TOKCONSTSYMSTR  16u     // "@const-symbol-strings"
 
 #define TOKTYPEBYTE     100u
 #define TOKTYPEI        101u

release.sh

@@ -15,6 +15,16 @@ echo $reportdir
 
 mkdir -p $reportdir
 mkdir -p $reportdir/scan-build
+mkdir -p $reportdir/cppcheck
+
+cd repl
+
+cppcheck32log="../${reportdir}/cppcheck/cppcheck_32bit_${release}.txt"
+cppcheck64log="../${reportdir}/cppcheck/cppcheck_64bit_${release}.txt"
+
+./run_cppcheck.sh $cppcheck32log $cppcheck64log
+
+cd ..
 
 cd tests
 

repl/Makefile

@@ -40,11 +40,17 @@ cov: repl_cov
 all64: 	CCFLAGS += -DLBM64
 all64: repl
 
-sdl: CCFLAGS += -DLBM64 -DWITH_SDL
-sdl: LISPBM_FLAGS += -I/usr/include/SDL2 -lSDL2_image -lSDL2
+sdl64: CCFLAGS += -DLBM64 -DWITH_SDL -I/usr/include/SDL2
+sdl64: LIBS += -lSDL2_image -lSDL2
+sdl64: REPL_SRC += lbm_sdl.c
+sdl64: repl
+
+sdl: CCFLAGS += -DWITH_SDL -m32 -I/usr/include/SDL2
+sdl: LIBS += -lSDL2_image -lSDL2
 sdl: REPL_SRC += lbm_sdl.c
 sdl: repl
 
+
 clean_cl.h: ./scripts/clean.lisp
 	./repl --store_env="clean_cl.env" --src=./scripts/clean.lisp --terminate
 	xxd -i clean_cl.env clean_cl.h

repl/buffer.c

@@ -137,7 +137,7 @@ void buffer_append_float32_auto(uint8_t* buffer, float number, int32_t *index) {
 		e += 126;
 	}
 
-	uint32_t res = ((e & 0xFF) << 23) | (sig_i & 0x7FFFFF);
+	uint32_t res = ((uint32_t)((e & 0xFF) << 23)) | (sig_i & 0x7FFFFF);
 	if (sig < 0) {
 		res |= 1U << 31;
 	}
@@ -153,26 +153,26 @@ void buffer_append_float64_auto(uint8_t* buffer, double number, int32_t *index)
 }
 
 int16_t buffer_get_int16(const uint8_t *buffer, int32_t *index) {
-	int16_t res =	((uint16_t) buffer[*index]) << 8 |
-					((uint16_t) buffer[*index + 1]);
-	*index += 2;
-	return res;
+  int16_t res =	(int16_t)((buffer[*index] << 8) |
+                          buffer[*index + 1]);
+  *index += 2;
+  return res;
 }
 
 uint16_t buffer_get_uint16(const uint8_t *buffer, int32_t *index) {
-	uint16_t res = 	((uint16_t) buffer[*index]) << 8 |
-					((uint16_t) buffer[*index + 1]);
-	*index += 2;
-	return res;
+  uint16_t res = 	(uint16_t)((buffer[*index] << 8) |
+                                   buffer[*index + 1]);
+  *index += 2;
+  return res;
 }
 
 int32_t buffer_get_int32(const uint8_t *buffer, int32_t *index) {
-	int32_t res =	((uint32_t) buffer[*index]) << 24 |
-					((uint32_t) buffer[*index + 1]) << 16 |
-					((uint32_t) buffer[*index + 2]) << 8 |
-					((uint32_t) buffer[*index + 3]);
-	*index += 4;
-	return res;
+  int32_t res =	(int32_t)((buffer[*index] << 24) |
+                          (buffer[*index + 1] << 16) |
+                          (buffer[*index + 2] << 8) |
+                          buffer[*index + 3]);
+  *index += 4;
+  return res;
 }
 
 uint32_t buffer_get_uint32(const uint8_t *buffer, int32_t *index) {
@@ -185,16 +185,16 @@ uint32_t buffer_get_uint32(const uint8_t *buffer, int32_t *index) {
 }
 
 int64_t buffer_get_int64(const uint8_t *buffer, int32_t *index) {
-	int64_t res =	((uint64_t) buffer[*index]) << 56 |
-					((uint64_t) buffer[*index + 1]) << 48 |
-					((uint64_t) buffer[*index + 2]) << 40 |
-					((uint64_t) buffer[*index + 3]) << 32 |
-					((uint64_t) buffer[*index + 4]) << 24 |
-					((uint64_t) buffer[*index + 5]) << 16 |
-					((uint64_t) buffer[*index + 6]) << 8 |
-					((uint64_t) buffer[*index + 7]);
-	*index += 8;
-	return res;
+  int64_t res =	(int64_t)(((uint64_t)buffer[*index] << 56) |
+                          ((uint64_t)buffer[*index + 1] << 48) |
+                          ((uint64_t)buffer[*index + 2] << 40) |
+                          ((uint64_t)buffer[*index + 3] << 32) |
+                          ((uint64_t)buffer[*index + 4] << 24) |
+                          ((uint64_t)buffer[*index + 5] << 16) |
+                          ((uint64_t)buffer[*index + 6] << 8) |
+                          ((uint64_t)buffer[*index + 7]));
+  *index += 8;
+  return res;
 }
 
 uint64_t buffer_get_uint64(const uint8_t *buffer, int32_t *index) {