current build logic is slow

[rofl0r]

it should create a process which downloads all stuff in parallel (at least 4 parallel downloads, at best as many as needed for saturating the line bandwidth) and as soon as one is finished, launch a build job per cpu core.

i.e. first download finished, launch build script
second download finished: check if a free core is available, if so launch another build job. else wait for buildjob #1 to finish
etc

[rofl0r]

i implemented a new build/package manager. it is available in my fork.
works extraordinarily well.
currently it spawns 16 download threads and 2 build threads, which are launched as soon as a download is completed/all dependencies satisfied. this could be easily changed to a userdefined value (i.e. 6 build threads on a hexacore machine).
once the tool is installed, it can be launched as "butch prefetch stage0 stage1 pkg xorg" to download all packages at once.
another change i made is that the tarballs are all stored in one directory, which will be mounted into the chroot.
the build manager will also retry on download error. on build error of a package it will continue building all non-dependant packages.
it will save the state and continue the build on next invocation with only the packages left. it drove me crazy that i have to rebuild everything again with the old mk build system whenever a build error occured.
"butch" https://github.com/rofl0r/butch can also easily be extended to support sha256 hashes and filesize information for the download, for integrity.
it already supports multiple mirrors (added mirrors for all kernel.org packages, which is currently down)