Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[config] Do not read MK_ARCH & MK_CPU if already defined #1760

Merged
merged 1 commit into from
Dec 2, 2023

Conversation

and3rson
Copy link
Contributor

@and3rson and3rson commented Dec 2, 2023

While investigating why my local bulds were taking so long, I noticed that grep is being called a lot. I figured that by running make SHELL='sh -x'.
This fix significantly improves rebuild times by not setting MK_ARCH & MK_CPU if they are already set somewhere up the Makefile chain.

On my machine, when running make all with some minor changes to the code (or no changes at all), this fix reduces build time from over 10 minutes to ~48s.

Similar change might be applied to some other places - I've seen some other heavy find/grep calls in the Makefile.

@and3rson and3rson changed the title make: do not read MK_ARCH & MK_CPU if already defined [config] Do not read MK_ARCH & MK_CPU if already defined Dec 2, 2023
@ghaerr
Copy link
Owner

ghaerr commented Dec 2, 2023

Wow @and3rson, nice catch!!

The code you found running a shell and grep for every Makefile is quite ancient, I wasn't even aware of it. In addition, the CONFIG_CPU_xxx defines and ifdefs have been completely removed elsewhere, as ELKS is now always built using 8088/8086 instructions only, for wider compatibility (including PC/Jr with only having short conditional jumps using .arch i8086,nojumps in .s files). Thus, MK_CPU should always be set to '8086'.

In addition, while CONFIG_ARCH_xxx is used with IBMPC, PC98 and 8018X, the older CONFIG_ARCH_SIBO config which Makefile-rules is using to set MK_ARCH is (or at least I thought it was) completely removed. MK_ARCH is only being used to set the unused ARCH_LD setting for the SIBO port, which was removed some time ago.

I figured that by running make SHELL='sh -x'.

Nice idea - I'll have to try this myself to see what short of other shenanigans might be occurring :)

On my machine, when running make all with some minor changes to the code (or no changes at all), this fix reduces build time from over 10 minutes to ~48s.

That's fantastic. There have been some other reports by users, unreproducible on my machines, that the ELKS make mechanism was running very slowly (IIRC during 'make clean' but nonetheless with excessive shell spawns). I am wondering why the sh -c grep ... is running so slowly on your OS/machine, do you suppose that's another reason or just old-fashioned too much overhead?

Similar change might be applied to some other places - I've seen some other heavy find/grep calls in the Makefile.

Please point them out, I'd love to get them fixed.

Given the current uses of MK_ARCH and MK_CPU, I'll commit your fix now, then delete the shell/grepping entirely in a follow-on commit, as described above its really not needed.

Thank you!

@ghaerr ghaerr merged commit fdc8fca into ghaerr:master Dec 2, 2023
1 check passed
@and3rson
Copy link
Contributor Author

and3rson commented Dec 2, 2023

I am wondering why the sh -c grep ... is running so slowly on your OS/machine, do you suppose that's another reason or just old-fashioned too much overhead?

I think it might be just because it's called a lot; I haven't gotten the fastest machine (i5-10210U), but it isn't too slow either. :)

Please point them out, I'd love to get them fixed.
Sure, I've grepped the most frequently called shell commands as such:

make SHELL='sh -c' 2>&1 | tee log
sort log | uniq -c | sort -hr | less

Here are the most frequently executed commands, each prepended with number of occurences:

   1701 + tr A-Z a-z
   1701 + cut -d _ -f 3-
   1701 + cut -d = -f 1
   1463 + sed 's:^/home/anderson/src/elks:.:'
   1463 + pwd
   1457 + printf %u.%u.%u%s 0 8 0 -dev
   1427 + read -r line
   1427 + IFS=
   1413 + link=no
   1413 ++ cut '-d ' -f 2
   1413 ++ cut '-d ' -f 1
   1382 + export allow_null_glob_expansion=Y
   1382 + allow_null_glob_expansion=Y
   1372 + '[' no == yes ']'
   1305 + mkdir -p /home/anderson/src/elks/target
   1305 + instdir=bin
   1305 ++ dirname /home/anderson/src/elks/target/bin
   1188 + for f in $(cd /home/anderson/src/elks/target; find * -name '*')
   1052 + echo '*.?' '*.cc' '*.html' config.in '*.png' '*.sh' '*.tk' '*.txt' Makefile Makefile-rules 'README*'
    851 + grep '^CONFIG_CPU_' /home/anderson/src/elks/.config
    850 + grep '^CONFIG_ARCH_' /home/anderson/src/elks/.config
    424 + ./scripts/setdir /home/anderson/src/elks/elks-/

(Some of them, such as IFS=, are obviously part of a bigger command.)

@ghaerr
Copy link
Owner

ghaerr commented Dec 2, 2023

Here are the most frequently executed commands, each prepended with number of occurences:

This is great! I'm already finding tons of useless stuff going on... improvements coming :)

On my machine, when running make all

You may want to try make kclean; make. I use this all the time to rebuild the kernel only, if libc/ and elkscmds/ haven't changed.... much quicker!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants