OSv Release 0.09

We are pleased to announce the release of OSv 0.09 Alpha Release. 0.09 release was focus on performance improvement and bug fixing.

Noteworthy performance improvement

• memcpy - copying small objects was optimized, leading to a performance improvement on some workloads.
• mempool - Improved across-CPU memory reclamation by making it fully lock-free.
• pagecache - cache of disk-backed pages kept in RAM by OSv for a quicker access.
• per-CPU-Tx framework - a per-CPU lock-less queues. Previously when a caller (sender) took a lock and blocks other senders until its packet is fully handled. In the new implementation caller never blocks unless the HW and a per-CPU queue of its CPU are full.

Noteworthy additions and updates

• AArch64: porting effort continues with the implementation of page fault handling, GIC Interrupt Controller model, Timers, Task Switching and more.
• /dev/urandom implementation
• A new dmesg REST API operation - returning the boot up debug messages
• Fix OSv boot on VirtualBox on older Macs.

For a full list of contributions, see the list below.

Latest enabled applications

• Oracle nosql DB
• Solr - enterprise search platform

Please note Capstan project have its own release cycle. New Capstan features introduced lately are: Ability to specify whole folders to be copied to the image in Capstanfile Cool download progress bar
More on Capstan here

OSv v0.09 is available for direct download here

or using >capstan search

Full list of OSv contributions:

Asias He (2):
      zfs: Make zpool and zfs list work
      libc: Add __h_errno_location

Avi Kivity (19):
      Update file API
      Update mgmt for upload API
      virtio-net: fix debug build due to missing try_lock_running()
      sched: relax migration lock barriers
      sched: simplify load_balance() determination of pinned threads
      pagecache: define access_scanner static members
      Merge branch 'mempool' of https://github.com/tgrabiec/osv
      tests: test more sizes for memcpy
      memcpy: aggressive unrolling
      Revert "memcpy: aggressive unrolling"
      Merge branch 'pagecache' of https://github.com/gleb-cloudius/osv
      fpu: early save/restore in interrupt/exception context
      elf: allow elf_resolve_pltgot() to use the fpu
      rwlock: helpers for WITH_LOCK and friends
      Revert "Revert "memcpy: aggressive unrolling""
      rwlock: replace condition variables with waitqueues
      poll: fix poll_sync() on unconnected sockets
      poll: fix poll_sync() early termination
      mgmt: update

Boqun Feng (4):
      mmu: fix an unsigned/signed comparison error
      elf: add unused attribute to unused function
      libc: fix an unused-value error in LOCK/UNLOCK
      build.mk: introduce *_env variables to make system

Claudio Fontana (25):
      early-console: implement arch independent early console
      aarch64: gic: implement GICv2
      aarch64: mmu: remove spurious include
      aarch64: mmu: add include to the proper place
      aarch64: exceptions: implement interrupt table
      debug: allow debug_early to work really early again
      aarch64: fix build broken by recent changes
      loader: use early console to display OSV string
      aarch64: exceptions: fix debug statements
      aarch64: remove some verbosity from processor.hh
      aarch64: mmu: distinguish between device and normal memory
      aarch64: smp: prepare GIC initialization per cpu
      aarch64: smp: do not forget to create incoming_wakeups
      aarch64: implement basic General Timers infrastructure
      aarch64: arch-switch: implement switch to first
      runtime: for backtrace subtract instruction size from ret addr
      aarch64: loader: allow execution to main_cont
      loader: move leftover arch-dependent code to arch
      aarch64: build.mk: enable real bootfs.bin
      aarch64: bsd: atomic.h: change atomic_store_rel_int
      aarch64: bsd: atomic.h: use STLR for atomic_store_rel_int
      tls: move more details about initialization to arch
      aarch64: entry.S: show exception vector entries in backtraces
      aarch64: arch-switch: implement thread::switch_to
      aarch64: mmu: fix page_fault assert for daif_i

Dmitry Fleytman (9):
      ec2-upload: Fix batch upload with small instances
      vmxnet3: Fix RX rings initialization
      vmxnet3: Fix long RX packets processing
      vmxnet3: Drop memory allocation error handling on RX path
      vmxnet3: Fix RX descriptor length field overflow
      syscalls: implement getpeername()
      release-ec2: always authenticate with AWS credentials
      release-ec2: Choose availability zone dynamically
      release-ec2: add parameter for OSv modules list specification

Glauber Costa (23):
      shrinker: export to C
      netchannels: specialize new and delete operators
      build.mk: Define a constant for OSv kernel builds
      sched: make current inline
      sched: make preempt functions inline
      rework OSV_SYM macro
      tests: initial malloc benchmark
      vfs: LFS64 versions of two more functions
      tests: memcpy benchmark
      memcpy: improve performance for x86's memcpy
      runtime: stub daemon symbol
      tests: increment memcpy tests to test memset too
      memset: make memset faster for small sizes
      stub setsid
      pthread: implement sem_destroy
      libc: implement lockf
      pipe: implement pipe2
      libc: implement ffsl and ffsll functions
      libc: implement gai_strerror
      pthread: implement raise() function
      libc: implement some more terminal functions
      libc: implement the whole mnt* family of functions
      mount: read initial entries from fstab

Gleb Natapov (23):
      pagecache: map zero page instead of ARC page for hole in a file
      vfs: prepare read uio in get_arcbuf()
      vfs: remove unnecessary vcount() checks in rename()
      pagecache: save vnode pointer instead of dentry in write page cache
      fix memory leak in bio handling
      pagecache, zfs: make hole detection logic more efficient
      mmu, x64: get maximum supported phys/virt bits from cpuid
      mmu, x64: fix getting/setting address in a pte
      mmu, x64: optimize detection of PROT_NONE mem region using PT reserved bit
      mmu: rename intermediate_page() to intermediate_page_post()
      mmu: remove reference to noexistent hw_pte_ref class
      mmu,x64: optimize detection of PROT_READ mem region by checking cow bit in a pte
      fix "osv info threads" gdb command
      mmu: fix split_large_page()
      pagecache: add accessed bit scanner thread
      pagecache: unify rmap flush and access scanning classes.
      pagecache: make  msync pagecache aware
      mmu: clear pte in unmap() function.
      pagecache: do not write back clean pages during write pages eviction
      mmu: fix madvice(MADV_DONTNEED)
      mmu: remove unused count_maps code
      mmu: implement madvise(MADV_NOHUGEPAGE)
      gdb: fix "osv info thread" to show correct thread address

Jani Kokkonen (6):
      mmu: do not destroy pt roots with double initialization
      aarch64: implement exception entry and exit
      aarch64: implement initial page fault handling
      aarch64: implement fixup fault and backtrace
      arch: move common fixup code to arch/common directory
      x64: arch-switch: remove unused stack_trampoline code

Jaspal Singh Dhillon (5):
      assert: fix __assert_fail()
      loader: poweroff() if cpus.size()>max_cpus
      Remove warning from mgmt.log when no tests are present
      remove vnc option during make
      java: Don't throw exceptions at user when the main class is not found

KANATSU Minoru (8):
      libc: implement stub function for execve
      libc: Fixed include directive
      add execle() to libc
      Reflect add libc/process and code
      Reflect add libc/process
      Add execv() to libc
      Add execl() to libc
      Reflect add execv() and execl()

Li Yu (1):
      net: don't abort when receive a VLAN frame

Nadav Har'El (14):
      VFS: cleanup readdir() and friends
      VFS: implement rewinddir(), telldir(), seekdir()
      Fix bug booting with 64 CPUs
      build.mk: fix breakage caused by MAKEFLAGS passed to module build
      sched: high-resolution thread::current()->thread_clock()
      Reindent __newlocale()
      Don't abort() on unknown locale
      locale: fix shifted ctype array
      locale: add test for C++ locale support
      acpi: Fix early-ACPI partial linear map
      drivers/clock: Fix comment
      tests: allow running tests/tst-mmap.cc
      sched: remove unnecessary reschedule_from_interrupt() parameter.
      sched: remove a bit of dead code.

Pawel Dziepak (16):
      elf: use symbol_binding() where appropriate
      libc: LOCK() and UNLOCK() do not have to return a value
      sched: reimplement incoming_wakeup_queue
      sched: relax memory ordering in cpu_set::fetch_clear
      post-processing/memory: make the script work with python3
      post-processing/memory: remove unnecessary gdb import
      post-processing/memory: be less strict about trace format
      mempool: add more information to tracepoints
      memory_analyzer: major rework
      elf: resolve R_X86_64_DPTMOD64 properly
      trace: integrate memory-analyzer with trace.py
      trace: filter memory analyzer output by minimum allocation count
      trace: sort memory allocations by given criteria
      trace: group memory allocations by given criteria
      trace: show backtraces in memory allocator
      trace: allow custom printers in memory analyzer

Paweł Dziepak (1):
      tests: fix unsigned/signed comparison in misc-free-perf.cc

Pekka Enberg (7):
      mgmt: update to latest
      apps: update to latest
      include/api: Remove dead code
      scripts: Separate Capstan and VM image building
      libc: Move getpwuid to user.cc
      libc: Fix getpwuid_r() to use OSv user info
      mgmt: update to latest

Prasad Joshi (1):
      vfs: enable support for DEBUG_VFS

Raphael S. Carvalho (23):
      bsd: Commit original BSD random files in their unmodified state
      bsd: crypto: Port rijndael to OSv
      bsd: Add snippets of code needed by random infrastructure
      bsd: Port random infrastructure from FreeBSD
      drivers: Change random to use Yarrow CPRNG and add urandom as an alias
      drivers: Register virtio-rng as a hardware source of entropy
      tests: Add simple test case to /dev/urandom
      timerfd: Add nsec validity check to timerfd_settime
      libzfs: Add missing files into libzfs build to fix missing symbols
      zfs: Port lz4 compression algorithm from FreeBSD
      tests: Add read-only fsop benchmark
      mkfs: Code refactoring and allow instances of the same shared object
      zfs: Enable compression on zfs dataset when creating the image
      zfs: lz4: Recognize __aarch64__ when detecting CPU word size.
      console: Fix isatty on a fd that refers to /dev/console
      libc: Add fdopendir from musl to fix missing symbol
      tests: Add testcase to /dev/console
      /etc/hosts: Remove non-relevant entry
      shrinker: Fix osv_register_shrinker visibility for C apps
      vfs: Implement fallocate system call
      vfs: Fix corner-cases where stat variants report out-dated file size
      zfs: Implement zfs_fallocate
      tests: Add fallocate testcase

Takuya ASADA (23):
      dhcp: Print IP address by default
      run.py: Add support VMware
      libc: get interval to correct field of itimerval
      Add linearize_uio_write
      Add console init_prio
      Move Console class to console namespace
      Merge debug_console to console.cc and isa-serial.cc
      Fix including header for console
      Implement console multiplexer
      Replace --vga with --console
      trace: Add remote gdb option to debug on VMware
      vmware: shrink vmx config
      vmware: add serial forwarding to tcp port
      vmware: add gdb stub
      build.mk: remove --vga from cmdline on osv.vmdk/.vdi
      vmxnet3: support net channel on vmxnet3.
      Add lookup_name_demangled(), which converts addr to demangled symbol name.
      Use lookup_name_demangled() on abort backtrace.
      Add symbol name on dump_registers
      Show current RIP and symbol name on sigsegv, by using dump_registers
      README: Add warning that we won't support Debian stable, and add qemu package on apt-get line
      run.py: add --qemu-path argument
      README: Add Debian stable description

Tomasz Grabiec (46):
      README: Advertise need of tcpdump and gdb
      trace: support tracepoint signatures of arbitrary length
      tests: add test for tracepoint with nine arguments
      net: hide tcpcb.t_state behind setters and getter
      net: add tcp_state tracepoint
      trace: cleanup packet tracing
      net: convert SYN cache to use async::serial_timer_task instead of callouts
      net: keep SYN cache entry if expansion failed
      core: add 'latch' synchronizer
      tests: add test exploiting client RST problem
      trace: include inlined-by functions in the backtrace
      trace: suppress displaying of _M_invoke in the backtrace
      trace: introduce trace.py prof-lock command
      trace: fix symbol resolving for addr2line v2.23
      net: cleanup syncache_head construction
      trace: fix default backtrace formatter construction
      gdb: make 'osv leak show' work with python3
      gdb: fix 'osv leak show'
      trace: fix trace list-timed
      trace: unify timed trace extraction
      trace: introduce 'prof-timed' command
      tests: increase coverage of trace.py
      trace: make timed-trace recognition handle discontinuous streams properly
      trace: ensure buffers are initialized when tracepoint is enabled
      trace: allow atomic changes of _log_backtrace via tracepoint_base::log_backtrace()
      trace: strip leading unimportant frames from backtrace
      trace: introduce sampling profiler
      tests: add sampler enabling/disabling test
      trace: Do not hide operator() functions
      trace: update unimportant_prefixes
      sched: add barrier() to migrate_disable/migrate_enable()
      sched: strengthen the fence in migrate_disable()
      mmu: make flush_tlb_all() migration-safe
      trace: use seconds as the default unit of timestamp
      mempool: make mempool_cpuid() inline
      mempool: drop redundant assert
      mempool: erase page range using an iterator rather than a reference
      lockfree: introduce unordered_queue_mpsc
      lockfree: introduce unordered_queue_spsc
      tests: introduce latch.is_released()
      tests: introduce latch.await_for()
      tests: add test for free() throughput across CPUs
      mempool: remove contention inside free_different_cpu()
      trace: support time slicing options in `trace summary`
      poll: special case POLLIN on one file descriptor
      test: add test for poll() on a socket

Vlad Zolotarov (22):
      virtio-vring: prevent delaying a kick() for more than half of a u16 range
      ring_spsc: Use bitwise AND instead of modulo for calculating the entry index
      ring_spsc: Use size() method instead of duplicating its code
      virtio-vring: fix kick() to return a correct value.
      virtio-vring: rework the interface of get_buf_finalize()
      virtio-net: coalesce doorbells to the host when finalizing Tx buffers.
      virtio-net: use indirect buffer descriptors
      virtio-net: Move the Tx related code into the txq class.
      virtio-net: make Tx drop-less.
      nway_merger helper class implementation
      Added a test for nway_merger with ring_spsc as a base sorted container.
      cpu_queue class
      xmitter class
      virtio-net: use per-CPU-Tx framework
      virtio-net: some white spaces, 80 columns and comments spelling fixes.
      ring_spsc: use static_assert() instead of assert() for template MaxSize parameter checking
      ring_spsc: put proper memory barriers when reading and storing _begin
      virtio-net: fix spelling in comments
      virtio::net::txq: Fix the bug causing the false asserts
      vitio::net: Fix the description of try_xmit_one_locked(net_req*)
      virtio::net: implement tx_xmit_iterator using boost::function_output_iterator
      virtio-vring: rename db_used() -> update_used_event()

Zifei Tong (1):
      Fix nits in README.md