Release 0.56.0 "Nightingale"

This new release brings new modern version of musl (1.1.24), greatly improved support of aarch64 architecture and many improvements of Virtio-FS including DAX window support to speed-up read I/O.

Highlights

• Musl
  • Upgraded from 0.9.12 to 1.1.24
  • Added new Bionic unit tests
• AArch64
  • Added Cadence UART driver
  • Added support to compile kernel natively on aarch64 host
  • Added support to cross-compile kernel on x64 Ubuntu and CentOS 7
  • Added support to build modules
  • Implemented mmio serial console for Firecracker
  • Added floating-point support
  • Supports ROFS
  • Greatly improved debugging
  • Fixed debug build which involved implementing context switch logic in assembly
  • Got most unit tests to pass
  • Added support of static TLS (Thread Local Storage)
  • Implemented PL031 driver
  • Added support of Virtio-FS
  • Improved SMP support (#vCPUs >= 2)
  • Tested on QEMU in TCG mode (full emulation)
  • Tested on QEMU and Firecracker with KVM on real ARM hardware - Raspberry PI 4 and Odroid N2 Plus
  • Tested many new apps:
    • nginx
    • rust examples
    • java 11
    • graalvm
    • python
    • iperf
    • ffmpeg
    • node
    • REST cli app
    • lighttpd
• Virtio-FS
  • Boots application from virtio-fs
  • Implemented DAX
  • Implemented readdir
• Added support to build with GCC 10 and 11
• VFS
  • Added mkdirat
  • Added statx
  • Added minimal unlinkat
• Hypervisors
  • Boots on Intel's cloud-hypervisor - see this Wiki for details
• Apps/runtimes
  • Latest golang (1.15.x)
  • nginx from host

Acknowledgments

We want to thank all contributors to the project. But the special thanks go to:

• Nadav Har’El for contributing and reviewing many patches and providing guidance for many others
• Fotis Xenakis for further improving Virtio-FS support (most notably DAX window)
• Stewart Hildebrand for making OSv boot on LYNX Secure hypervisor and fixing many AAarch64-related bugs
• Waldemar Kozaczuk for contributing most patches

Commits by Author

David Smith (1):

• Fix mkdir syscall issue and implement mkdirat

David Woodhouse (1):

• x64: Parse X2APIC CPU definitions in MADT table

Fotis Xenakis (29):

• docker: update default to ubuntu 20.04
• virtio-fs: update fuse protocol header
• virtio-fs: implement FUSE_INIT map_alignment field
• virtio-fs: add basic read using the DAX window
• virtio-fs: refactor driver / fs
• virtio-fs: implement dax window manager
• virtio-fs: use multiple dax mappings in filesystem
• virtio-fs: minor driver code improvements
• virtio-fs: fix lost wakeup in driver
• virtio-fs: fix allocation failure condition
• virtio-fs: expose FUSE reply length
• virtio-fs: use FUSE_OPENDIR and FUSE_RELEASEDIR
• virtio-fs: implement readdir
• virtio-fs: add error check in fs
• virtio-fs: use exclusive device ids
• vfs: homogenize mount_rofs_rootfs and mount_zfs_rootfs
• loader: add rootfs type command line option
• loader: add support for booting from virtio-fs
• scripts/build: don't exit when exporting files
• virtio: fix return type of get_driver_features()
• virtio-fs: set the VIRTIO_F_VERSION_1 feature bit
• scripts: add kernel command-line options to run.py
• scripts: enhance virtiofsd functionality in run.py
• virtio-fs: update fuse protocol header
• virtio-fs: make DAX manager testable
• tests: add DAX manager white-box unit tests
• loader: extend root fs auto-discovery
• fs: support d_off in readdir for all filesystems
• scripts: add virtio-fs DAX option to run.py

Inacio Klassmann (1):

• fixed NameError: name 'file' is not defined

Matthew Kenigsberg (1):

• Handle newline in /etc/os-release

Nadav Har'El (23):

• libosv.so: fix warnings on truncated symbol names
• memset_base(): avoid warnings in stricter C++ compiler
• lzloader: fix memset() implementation
• setup.py: support Fedora 32
• Add new mechanism for function aliases - outside original source file
• syscall_to_function.h: simplify, and avoid C++20 extension
• scripts/setup.py: simplify Fedora's Lua requirement
• musl: fix __stdio_read.c to conform to new Musl
• Add test for fread() hang when reading from a directory
• fix compatibility features.h for Fedora 34
• Fix __cxa_finalize() signature
• bsd/sha2.c: fix mismatched signatures
• Fix warnings about misleading indentation
• xen: fix overread and overwrite bug
• Makefile: avoid warning in one Musl source file
• fs/vfs/main.cc: fix gcc 11 build error
• core/trace.cc: free aligned_alloc() with free()
• sched: free aligned_alloc() with free()
• core/mmu.cc: silence warning
• add __libc_single_threaded symbol
• scripts/setup.py: add Fedora 34
• tests: fix warning in tst-epoll.cc
• Implement statx() function

Punit Agrawal (3):

• scripts: Update script to download RPM packages
• Update boost library detection logic for cross compiling
• scripts/build: Simplify libgcc_s.so.1 detection

Robin Gögge (1):

• scripts: fix setup.py for Ubuntu versions

Samuel Laberge (3):

• Make getpid() return non-zero number
• Fix kill tests for non-zero PID
• Reject executables with non-matching machine type

Shawn Barber (1):

• Modifying times function to gracefully handle being passed a null pointer

Stewart Hildebrand (19):

• build: set arch in arg-parsing loop
• Add script to download aarch64 cross compiler toolchain
• aarch64: re-introduce dtb relocation
• aarch64: add function to get PSCI method from device tree
• aarch64: use PSCI method from device tree
• Introduce Cadence UART driver
• Cadence: initialize from device tree
• Cadence: add barrier
• aarch64: move fpu_state_save/load to entry.S
• boost: add missing library
• setup.py: add readelf/grep
• aarch64: populate image size in header
• scripts/boost: update URL
• aarch64: Support downloading packages on Ubuntu 18.04 x64
• aarch64/boot.S: fix preboot pagetables
• build: use CROSS_PREFIX if defined
• virtio-mmio: add NULL check in register_mmio_devices
• aarch64: use proper data size for __loader_argc
• Cadence: add device tree parsing function

Waldemar Kozaczuk (201):

• Merge pull request #1107 from punitagrawal/master
• Merge pull request #1133 from codegik/fix-open-file
• lua: added missing lpeg library
• setup.py: support Ubuntu 20.04
• maven: fix "recursive expression cycle" problem
• pthreads: expose pthread_attr_getscope() symbol properly
• serial console: refactor common logic to allow re-use when implementing mmio version
• arm: add memory-mapped isa console
• arm mmio serial console: use id passed in DTB for irqid instead of a hardcoded number
• arm on firecracker: detect and enable mmio isa early console
• arm on firecracker: map first 6GB one-to-one during early boot
• arm on firecracker: clean cmdline from early console artifacts
• arm on firecracker: recognize GIC v2 on arm and skip parsing PCI if count 0
• aarch64: until MSI-X support becomes functional enable regular PCI interrupt
• scripts: use common binary partition header file to build disk images
• scripts: support running aarch64 rofs images
• aarch64: until MSI-X support becomes functional do not try to setup vqueue MSI-X vector
• debugging: fix loader.py to handle "osv runqueue" properly
• debugging: enhance loader.py to support both x64 and aarch64
• dynamic linker: fix the old bug in the logic to calculate the ELF base
• scripts/run.py: do not hardcode memory size and cpu count for aarch64
• scripts/build: refine build process for aarch64
• aarch64: support virtio mmio devices
• aarch64: parse and activate virtio mmio devices
• aarch64: support building unit tests image
• aarch64: fix memmove bug
• licenses: add newlib license
• scripts: update firecracker.py to support aarch64
• scripts: make firecracker.py default to host architecture
• tests: fix shell if/else syntax to make it work on both Ubuntu and Fedora
• scripts: enhance run.py to allow running aarch64 images with KVM enabled
• tests: do not include java tests for aarch64
• libc: revert some files with "#undef" to point back to musl
• libc: revert some files with "__restrict" to point back to musl
• libc: remove more string functions from libc/ in favor of the original musl ones
• libc: revert more files with "#undef" to point back to musl
• libc: remove unused libc/regex/glob.c
• libc: replace two more files with the musl copies
• libc: replace unistd/ttyname.c with the musl copy
• libc: replace identical headers with the symlinks to musl copies
• libc: replace weak_alias and wrapper functions with linker symbol copy mechanism
• libc: replace more include/api/ headers with their musl copies
• libc: point 3 more files from libc/ to musl/
• libc: remove unused file
• libc: replace __strndup.c by adding entry to aliases.ld
• libc: replace network/gai_strerror.cc with its identical C copy from musl
• libc: replace 5 more headers in include/api with musl copies
• fs:change uiomove() to use size_t in the signature instead of int
• libc: replace some headers libc/internal with symlinks to identical copies under musl/src/
• tests: add verbatim copies of Bionic tests for w/ctype
• tests: adapted w/ctype tests to work with boost unittest and added tst-string
• libc: replace iswctype_l and wctype_l.c with musl equivalent
• libc: drop OSv toupper_l in favor of musl copy and add alias __toupper_l
• libc: drop towlower_l and towupper_l in favor of musl copies
• libc: drop setlocale.c and duplocale.c and add __setlocale alias
• libc: replace locale/wcsxfrm*.c (from old musl) with musl copy of locale/wcsxfrm.c
• libc: replace locale/wcscoll*.c (from old musl) with musl copy of locale/wcscoll.c
• libc: replace locale/strxfrm*.c (from old musl) with musl copy of locale/strxfrm.c
• libc: replace locale/strfmon.c with the musl copy
• libc: replace locale/strcoll*.c (from old musl) with musl copy of locale/strcoll.c
• libc: replaced 6 files in /libc/stdio/ that uses syscall instruction with musl copies and macros
• tests: fix compilation of tst-wctype.cc on older versions of gcc
• libc: replace floatscan.c, intscan.c and shgetc.c with the symlinks to its almost identical copies
• libc: replace string/strsignal.c with musl copy
• tests: added bionic-based unit test to test time libc functions
• libc: copy time/__tz.c from musl as is
• libc: replaced most libc/time/ files with their musl copies
• libc: replace langinfo with musl copies
• libc: replaced wcsftime files with their almost identical musl copies
• bsd: rename __unused* macros to __bsd_unused* to avoid conflicts with musl headers
• libc: replace 8 headers in include/api with their symlinked copies
• api: replace 3 more include/api headers with musl copies
• libc: replace include/api/x64/bits/socket.h with a symlink to musl copy
• libc: replace include/api/sys/eventfd.h with symlink to musl copy
• libc: replace libc/internal/libm.h with a symlink to musl original
• libc: replace include/api/aarch64/bits/mman.h with a symlink to musl copy
• libc: drop 5 more headers with symlinks to musl copies
• headers: get rid of fenv.h copy
• includes: fixed fenv.h symlink to point to the right musl directory
• tests: added bionic based tst-stdio.cc unit test
• rename musl submodule to musl_0.9.12 and add new submodule musl_1.1.24
• Upgrade musl from 0.9.12 to 1.1.24
• tests: pass AARCH64_PORT_STUB macro when building aarch64 image
• tests: enhance test.py to support running unit tests on aarch64
• aarch64: implement feenableexcept/fedisableexcept/fegetexcept
• tests: add simple tst-setjmp.cc to verify setjmp/longjmp/sigsetjmp/siglongjmp functionality
• aarch64: implement sigsetjmp/siglongjmp
• aarch64: fix memory mapping initialization when booting with #vCPUs >= 2
• tests: make TLS related tests build for aarch64
• tests: Fixed dependabot junit security alert
• elf: refine logging
• aarch64: support static TLS (Thread Local Storage)
• scripts: fixed build-capstan-mpm-packages to use correct parameters with new capstan 0.5
• firecracker: upgrade to the latest 0.23.0
• virtio: removed commented out "sleep()" from virtio::fs() constructor
• replaced offensive names of variables and methods with better ones
• tests: enable tst-feexcept when running with KVM on
• build: do not run maven for java modules if nothing changes
• tests: make irrelevant build messages very quiet
• aarch64: implement dump_registers()
• aarch64: store ESR (Exception Syndrom Register) in the exception frame
• x64: fix assembly warning in vmlinux-boot64.S
• scripts: add optional mode option to manifest_from_host.sh
• java: make libfreeblpriv3.so optional for openjdk8-from-host module
• setup: support Ubuntu 20.10
• scripts: make test script work with latest Python on Fedora 33
• setup: support Fedora 33
• libc: add alias of getauxval
• build: enforce Partial RELRO for internal apps
• aarch64: properly handle unexpected synchronous exceptions
• vfs: add minimal implementation of unlinkat
• aarch64: update syscall.h with correct numbers
• kernel: enforce object files are compiled with '-fno-pie'
• makefile: added comments justifying building kernel objects with "-fno-pie"
• aarch64: synchronize data and instruction caches after loading code into memory
• travis: upgrade to newer docker base image - Ubuntu 20.10
• aarch64: do not disable rofs cache in run.py
• bootchart: do not print empty entries
• aarch64: make boot messages quieter and consistent with x86_64
• aarch64: implement processor_to_nano in armclock
• setup: install aarch64 C++ cross-compile packages for x86_64 hosts only
• aarch64: support downloading aarch64 packages on Ubuntu x64
• aarch64: support compiling kernel natively on aarch64 host
• scripts: enhance manifest_from_host.sh to work on aarch64 host
• syscalls: ignore missing SYS_mkdir for aarch64
• aarch64: enhance setup.py to install multi-arch gdb package on Ubuntu
• include: support compiling OSv on CentOS 7
• modules: add common makefile to allow cross-compiling
• tests: simplify makefile by including new modules/common.gmk
• libtools: include common.gmk
• httpserver-monitoring-api: include common.gmk
• httpserver-api: include common.gmk
• tests/tst-threadcomplete.cc: disable part testing pinning when cpu == 1
• aarch64: fix loader.py to use correct IP register
• aarch64: handle R_AARCH64_COPY relocations
• scripts/run.py: use QEMU_PATH to use arbitrary qemu executable
• tests: handle more generic CROSS_PREFIX for aarch64 builds
• aarch64: make dl_tests makefile build for aarch64
• boost: enhance main makefile to build against arbitrary boost
• boost: enhance modules/common.gmk to build against arbitrary boost
• setup: support CentOS 7
• scripts: download and build arbitrary version of boost
• scripts: allow building QEMU from sources
• scripts: make firecracker.py point to artifacts in the last
• aarch64: align up size of the kernel TLS to 64 bytes
• aarch64: fix TLS kernel variables access from apps
• aarch64: fix building ramfs images on Fedora
• aarch64: use memory barrier to force completion of PTE writes
• tests: improve makefile by using more portable case statement
• aarch64: make kernel_vm_base equal to kernel_base
• aarch64: implement reschedule_from_interrupt() in assembly
• aarch64: re-enable unit tests fixed by latest patches
• arch64: enable sampler and corresponding unit test
• tst-stdio.cc: disable slow fread_unbuffered_pathological_performance test
• aarch64: change pthread_mutexattr_t, pthread_condattr_t and pthread_barrierattr_t to unsigned int
• tst-condvar.cc: run SMP test only if #vCPUs >= 2
• aarch64: add thread_main to improve stacktraces
• aarch64: add CFI directives to the page fault and interrupt handler assembly
• loader.py: improve "osv info threads"
• aarch64: mask interrupt and disable timer in the handler
• aarch64: implement minimal PL031 support
• aarch64: enable detection of virtio-fs driver
• scripts: enhanced download_and_build_qemu.sh to build QEMU from arbitrary repo and enable virtiofsd
• aarch64: unmask clock event interrupt on secondary CPUs
• dynamic linker: properly handle STT_NOTYPE symbols
• mempool: resize l1_pool_stats once
• aarch64: handle AT_HWCAP in getauxval
• aarch64: make RCU work correctly in SMP mode
• libc: add strerrordesc_np
• syscalls: minimally implement tgkill
• libc: add brk()/sbrk() stubs
• travis: authenticate to docker hub
• java tests: upgrade commons-io to 2.7 to silence github.com warning
• travis: attempt to enable KVM
• tests: fix style issues in tst-kill.cc
• tests: fix tst-kill.cc
• scripts: fix trace.py
• aarch64: make wait_record with weak memory model
• aarch64: make queue_mpsc work with weak memory model
• aarch64: make synch_thread/synch_port work with weak memory model
• sigaction: handle null sa_sigaction as SIG_DFL
• scripts/build: default to rofs when building aarch64 images
• aarch64: enable test_net.py and test_tracing.py
• aarch64: do not probe for pci if opt_pci_disabled
• scripts: update firecracker.py to properly support aarch64
• java: add OpenJDK module for java 9 and above
• openjdk8: make it work with aarch64
• aarch64: make java-base and related modules compile
• tst-async.cc: include to silence error in GCC 11
• boost: update path of function_output_iterator to silence warnings
• futex: implement FUTEX_WAIT_BITSET
• acpi: capture ACPI root address when booting in PVH mode
• tests: disable java tests when cross-compiling to aarch64
• travis: enable running unit tests
• travis: disable running tst-time.so
• aarch64: on Fedora 34 download the aarch64 artifacts from the version 33
• vfs: harden drele() to check for null v_op and vop_inactive
• cirp: pass --travis-api-url https://api.travis-ci.com to work a round a bug in cirp plugin
• setup: add Ubuntu 21.04 distribution
• scripts: updated test.py to reflect most up-to-date state of tests on aarch64
• replace AARCH64_PORT_STUB with aarch64 and x86_64

Closed Issues

• #1148 vfs: missing statx
• #1147 futex: missing implementation of FUTEX_WAIT_BITSET
• #1146 acpi: detect the root table pointer using the address field rsdp_paddr passed during PVH boot
• #1144 Build on Fedora 34
• #1142 [firecracker][aarch64] This code only works correctly for OSV_KERNEL_VM_SHIFT = 0x40000000 or 0
• #1134 aarch64: the do_main_thread hangs in osv::rcu_flush() in SMP mode
• #1133 fixed NameError: name 'file' is not defined
• #1132 aarch64: unmask interrupts on non-primary cores
• #1130 aarch64: tst-threadcomplete.cc hangs most of the time with 2 vCPUs or more
• #1129 aarch64: support LSE instructions
• #1127 aarch64: some unit tests occasionally hang when running on QEMU in emulated mode (TCG)
• #1126 aarch64: does not build on Fedora 31
• #1125 aarch64: does not build on Ubuntu 20.04/10
• #1124 aarch64: kernel built in debug mode crashes with page faults
• #1123 aarch64: early boot hangs sometimes in SMP mode (#vCPUs >=2)
• #1122 aarch64: TLS kernel variables incorrectly accessed by applications
• #1121 aarch64: tst-pin.cc crashes with cpus >=2
• #1120 aarch64: boot fails with debug build due to TLS assert
• #1117 aarch64: virtio::register_mmio_devices() sometimes page faults on real hardware like RockPro64 or Odroid
• #1115 Boot failure in Fedora 33
• #1114 tst-stdio-rofs fails in x86_64 too
• #1113 corrupt ELF notes in loader.elf generated on Fedora 33 causes PVH boot failure
• #1112 arch_init_early_console crashes when kernel built with certain version of gcc
• #1111 x64: Parse X2APIC CPU definitions in MADT table
• #1109 Make getpid() return non-zero number
• #1107 scripts: Update script to download RPM packages
• #1106 aarch64: enhance main makefile to support building from host for aarch64
• #1105 aarch64: enhance build system to support building aarch64 versions of modules and apps
• #1104 Identify and reject executables not matching kernel machine (x64, aarch64)
• #1103 arm64: Unable to build using instructions
• #1101 aarch64: support static TLS (Thread Local Storage)
• #1100 aarch64: sometimes triggers a synchronous exception of type IL (Instruction Length) on arm64 native host
• #1099 x86_64 build error
• #1097 Clean fails
• #1095 Unable to read very large file in C++
• #1093 aarch64: OSv crashes frequently during power-off process with ROFS
• #1092 aarch64: hangs while booting on KVM with #vCPUs >= 2
• #1091 aarch64: bugs in arm-clock
• #1090 aarch64: broken memmove
• #1086 scripts: fix setup.py for Ubuntu versions
• #1085 scripts: setup.py broken for Ubuntu versions < 19.10
• #1084 Build fails to upload image on Fedora
• #1083 Assembler warnings in build on Fedora 32
• #1062 Support read-only subset of Virtio-FS file system
• #1047 Provide limited implementation of the tgkill syscall needed by Golang PIE apps
• #1003 gdb did not know about "osv syms" command.
• #991 aarch64 system call numbers are probably wrong
• #922 Can't run osv-memcached in osv
• #916 scripts/loader.py is not python3 compatible
• #895 NFS Makefile cleanups
• #813 mount-fs.so should gracefully handle the case of missing filesystem library
• #796 JVM ballooning is disabled
• #786 Add Mono (MITized) or CoreRT to OSV
• #760 syscall set_mempolicy missing
• #752 Sigar and (G)LibC
• #747 Need sigsuspend and openat64 support to run Nginx in OSv
• #717 OSv build on ARM processor
• #687 Inconsistent mix of build_env = host / external
• #675 "python" interpreter in scripts
• #661 vnode v_mode = 0100444, even though ls -al says different
• #648 When not using AWS default instance the script release-ec2.py will fail
• #646 Read-only file system
• #626 Added java "wrapper" module
• #619 external/x64: "make build_env=external" fails
• #544 Need to implement "pthread_setcancell_state"
• #524 Fault when exiting the internal cli
• #506 major() and friends macros are not Linux compatible
• #504 Cassandra gets stuck when running ycsb workload-C
• #498 build: failure because of old file modules/tests/usr.manifest
• #455 Augment tracepoint backtrace with Java backtrace
• #404 API: Use the swagger validator - to verify our swagger file is compatible with the spec
• #385 JSON friendly value for processor features in the RESTful API
• #379 Lost of network on GCE
• #312 New performance regression in ctxsw micro-benchmark
• #212 Allow running a statically-compiled Linux executable
• #201 Intel MPX support for buffer overflow detection
• #191 run "git submodule update --init" automatically
• #189 Block IO tracing and visualization
• #164 Our clock, even osv::clock::uptime, is not really monotonic
• #151 fix miscompile with ld.gold
• #102 Hang in tlb_flush()
• #90 Support per-thread standard input/output
• #84 per-module search path
• #77 Integrate Apache Portable Runtime native library with OSv
• #61 Hook the JVM dtrace to OSv shell
• #59 Investigate EC2 network disconnects
• #47 dlopen() and program::add_object() needs reference counting
• #27 Integrate VisualVM support for JVM