Running src libc tests

Here are some results from running the FreeBSD lib c tests

Here's what I tried

• copied /usr/src into my home directory
• alternatively, git clone https://git.freebsd.org/src.git
• git checkout releng/14.0 (or whatever corresponds to the system I'm testing on)
• did a build with debug enabled - below is a normal build

FreeBSD doesn't seem to have an easy way to build everything with debug info. Ports have WITH_DEBUG=yes but not core.

So I just added

CFLAGS+=-g
CXXFLAGS+=-g

to /etc/make.conf

Obviously that's only for this kind of testing.

#!/usr/local/bin/ksh93

export BUILD_ROOT=/usr/home/paulf/build/src

export MAKEOBJDIRPREFIX="$BUILD_ROOT/obj"
export CHROOTDIR="$BUILD_ROOT/var"

#ncpu="$(sysctl -n hw.ncpu)"

mkdir -p $MAKEOBJDIRPREFIX
mkdir -p $CHROOTDIR

cd $BUILD_ROOT/src
make -DNO_CLEAN buildworld "$@"

• takes about 2.5 hours
• use -j 4 for a bit of speed
• incremental builds seem to have problems copying over files requiring running without NO_CLEAN which is a full build
• cd to src/lib/libnetbsd and run make there
• cd src/lib/libc/tests and then to a subdir
• run make in the subdir
• run the tests : valgrind --trace-children=yes -q kyua test -k Kyuafile
• look a the results : kyua report

Some examples

In stdlib, 1 fail dylib not found

In gen floatunditf_test:floatunditf -> failed: 14 checks failed; see output for more details [0.666s]
glob2_test:glob_pathological_test -> failed: 1 checks failed; see output for more details [0.680s]
posix_spawn/spawn_test:t_spawn_missing -> failed: /usr/home/paulf/build/src/contrib/netbsd-tests/lib/libc/gen/posix_spawn/t_spawn.c:112: expected error 2, got 0 when spawning /usr/home/paulf/build/src/lib/libc/tests/gen/posix_spawn/h_nonexist [0.637s]
posix_spawn/spawn_test:t_spawn_nonexec -> failed: /usr/home/paulf/build/src/contrib/netbsd-tests/lib/libc/gen/posix_spawn/t_spawn.c:132: expected error 2, got 0 when spawning /usr/home/paulf/build/src/lib/libc/tests/gen/posix_spawn/h_nonexec [0.652s]
posix_spawn/spawn_test:t_spawn_zero -> failed: /usr/home/paulf/build/src/contrib/netbsd-tests/lib/libc/gen/posix_spawn/t_spawn.c:92: expected error 8, got 0 when spawning /usr/home/paulf/build/src/lib/libc/tests/gen/posix_spawn/h_zero [0.650s]

Float diffs are to be expected, sort of.
Spawn and glob - would need to analyze.

In sys
brk_test:mlockfuture -> failed: brk_test.c:132: unexpected page flags 0 [0.616s]
dup_test:dup3_max -> failed: /usr/home/paulf/build/src/contrib/netbsd-tests/lib/libc/sys/t_dup.c:259: Expected true value in dup3(STDERR_FILENO, res.rlim_cur + 1, O_CLOEXEC) == -1 [0.625s]
dup_test:dup_max -> failed: system call error [0.870s]
getrusage_test:getrusage_sig -> failed: getrusage(2) did not record signals [0.618s]
setrlimit_test:setrlimit_nofile_1 -> failed: RLIMIT_NOFILE not enforced [0.683s]
setrlimit_test:setrlimit_nofile_2 -> failed: RLIMIT_NOFILE not enforced [0.688s]
sigqueue_test:sigqueue_rt -> failed: /usr/home/paulf/build/src/contrib/netbsd-tests/lib/libc/sys/t_sigqueue.c:222: 0: ordered 67 != delivered 2 [1.643s]
truncate_test:truncate_err -> failed: /usr/home/paulf/build/src/contrib/netbsd-tests/lib/libc/sys/t_truncate.c:167: Expected true value in truncate(buf, 999) == -1 [0.635s]
umask_test:umask_fork -> failed: umask(2) was not inherited [0.637s]
wait_test:wait6_coredumped -> failed: /usr/home/paulf/build/src/contrib/netbsd-tests/lib/libc/sys/t_wait.c:172: WIFSIGNALED(st) && WTERMSIG(st) == SIGSEGV && WCOREDUMP(st) not met [0.698s]

The rlimit/FILENO fails are because the VG host uses up some file descriptors.

getrusage is probably because the VG host generates extra signals