Add arena allocator to cpp86, c86 and as86

[ghaerr]

Adds the ELKS arena allocator to CPP86, C86 and AS86, as discussed in ghaerr/elks#2140 (comment).

Also renames nasm -> nasm86 and ndisasm -> ndisasm86.

@rafael2k, I got CPP and C86 to function well, but could not get the "big pig" nasm86 to work with the arena allocator. It seems it uses a LOT of memory. If you turn on sysctl kern.debug=1 you will be able to see the FMEMALLOC it attempts doing. Running sysctl malloc.debug=2 (or 3) displays heap debugging in detail. I'm going to think about how to best proceed a little more; I am thinking of adding yet more malloc/fmemalloc monitoring, with an option that just shows the totals used, rather than all the detail. NASM isn't linked with the debug or arena allocator so it won't have as much debugging info.

LD86 does not need changing at this time, as it only uses fmemalloc in one place. So I've left it alone for now, pending getting NASM86 working.

You'll see the changes made for CPP, C86 and AS86 are very straightforward. I did the same thing for NASM86, basically changing #if __ELKS__ to #if 0 and adding mem.c (see compiler/mem.c or cpp/mem.c for details). But it runs out of memory. You're welcome to try to get it to work with the new allocator. The problem seems to be it does a lot of fmemalloc 640 byte allocations, then a big one from which there is no memory. All in all, we need to learn more about the actual allocations NASM86 is doing in order to figure out what to do better.

On another note, when a memory failure happens, the new code runs meminfo displayed to the console, showing what the memory layout looks like. The bad news is there is far less free memory than I thought, since make86 is running, and it seems it forks itself then execs a /bin/sh -c which is huge, to run the make targets. This results in tons of memory being used. I am working on a solution to that.

C86 itself would not run with the arena allocator getting 64K of arena at startup, I had to reduce it to 32K. Here's a look at what meminfo showed before that change:

Note the largest free space is 49K after C86 starts up! Wow.

Also something to consider is that we don't need the larger heapsize= specifications for the default data segment when using the arena allocator, since it doesn't allocate from the default data segment. This mean they can be drastically reduced. You're welcome to play with that too. We should discuss this more so we fully understand which programs are using what memory, as I can see the system is going to remain tight.

Overall, this is ready to commit, as everything still works as nasm86 has been left alone. I wasn't able to test building chess though, as stdint.h got all messed up and I don't have the "dev kit" installed on my root directory since I am testing on QEMU with a 2880k floppy image which is almost full. I did test with test.c "Hello world" which builds and runs.

Let me know what you think the next steps should be. Ultimately, we have the allocators now, but need to figure the best ways to use them for NASM.

[rafael2k]

So with the malloc arena we are losing the old heap space?

[rafael2k]

I'll think about the nasm case and do some tests.

I tested the toolchain here and it keeps working.
: )

[ghaerr]

So with the malloc arena we are losing the old heap space?

Yes - meaning that the old heap space is not needed, since almost all uses of the heap are replaced with the arena malloc, which uses fmemalloc to get its own heap. This then means the heapsize= for these applications can be reduced greatly, taking that memory out of the process space when it is started and making it available for other applications. I would suggest setting heapsize=512 for all applications using arena malloc and see how that goes. You can then play around with the MALLOC_ARENA_SIZE, which is the actual heap size required for the application. I set this originally at 65520, but that ended up being to large a heap (it must be contiguous) to be found in ELKS main memory! Most are set to 32000. The point is we need to have an understanding of the "working set" of memory that each tool needs, both arena heap and far (main memory) heap.

This is all a bit complicated, please ask questions if you have them. I'm going to build a mechanism into both debug and arena malloc that allows us to see summary allocations. You can see detailed allocations by setting sysctl as suggested above.

[ghaerr]

I'll think about the nasm case and do some tests.

Great. I would suggest adding the arena malloc into it, like I did with the other tools, then playing with it. I didn't leave it in this PR since I did not have time to get it working, and also not enough time to analyze its memory use using sysctl. I will have more time coming up, but that's also why I am going to work on some summary debug display, rather than so much detail. It seems NASM is failing on the fmemalloc allocations, not the arena allocations.

[rafael2k]

Added new memory function with arena to nasm here:
#22