Merge pull request #336 from sshanks-kx/refactor

refactor
KxSystems · Dec 2, 2024 · 96904fa · 96904fa
2 parents a630bc9 + 0ddf1d4
commit 96904fa
Show file tree

Hide file tree

Showing 15 changed files with 196 additions and 162 deletions.
diff --git a/docs/basics/debug.md b/docs/basics/debug.md
@@ -98,7 +98,7 @@ q)). / down
 q))
 ```
 
-In a debugger session, `.z.ex` and `.z.ey` are set to the failed primitive and its argument list.
+In a debugger session, [`.z.ex`](../ref/dotz.md#zex-failed-primitive) and [`.z.ey`](../ref/dotz.md#zey-argument-to-failed-primitive) are set to the failed primitive and its argument list.
 
 ```q
 q)).z.ex
@@ -159,7 +159,7 @@ q)):17
 
 ## Abort
 
-Use `\` to exit the debugger and abort execution.
+Use [`\`](syscmds.md#terminate) to exit the debugger and abort execution.
 
 ```q
 q))\
@@ -247,7 +247,7 @@ View calculations and system commands, including [`\l`](syscmds.md#l-load), corr
 
 ### Trap
 
-[`.Q.trp[f;x;g]`](../ref/dotq.md#trp-extend-trap-at) extends `trap at` (`@[f;x;g]`) to collect backtrace. Along with the error string, `g` gets called with the backtrace object as a second argument. You can format it with `.Q.sbt` to make it legible.
+[`.Q.trp[f;x;g]`](../ref/dotq.md#trp-extend-trap-at) extends [`trap at` (`@[f;x;g]`)](../ref/apply.md#trap-at) to collect backtrace. Along with the error string, `g` gets called with the backtrace object as a second argument. You can format it with [`.Q.sbt`](../ref/dotq.md#sbt-string-backtrace) to make it legible.
 
 ```q
 q)f:{`hello+x}

diff --git a/docs/basics/errors.md b/docs/basics/errors.md
@@ -11,6 +11,10 @@ description: Errors signalled by the interpreter, and what triggers them
 
 <style>dt {color:#F23A66}</style>
 
+{directory}/q.k. OS reports: No such file or directory
+
+:   Using the environment variable `QHOME` (or `<HOME DIRECTORY>/q` if not set), `q.k` was not found in the directory specified. Check that the `QHOME` environment variable is correctly set to the directory containing `q.k`, which is provided in the kdb+ installation files.
+
 [](){#access}
 access
 
@@ -623,15 +627,15 @@ error | explanation
 {timestamp} couldn't connect to license daemon | Could not connect to KX license server ([kdb+ On Demand](../learn/licensing.md#licensing-server))
 [](){#cores}cores | The license is for [fewer cores than available](../learn/licensing.md#core-restrictions)
 [](){#cpu}cpu | The license is for fewer CPUs than available
-[](){#exp}exp | License expiry date is prior to system date
+[](){#exp}exp | License expiry date is prior to system date. The license has expired. Commercial license holders should have their Designated Contacts reach out to [email protected] or contact [email protected] to begin a new commercial agreement.
 [](){#host}host | The hostname reported by the OS does not match the hostname or hostname-pattern in the license.<br><br>If you see `255.255.255.255` in the kdb+ banner, the machine likely cannot resolve its hostname to an IP address, which will cause a `host` error.<br><br>Since 4.1t 2022.07.01,4.0 2022.07.01 the detected hostname is printed. It can be used to compare with the hostname used within the license.
-[](){#k4.lic}k4.lic | `k4.lic` file not found, check contents of environment variables [`QHOME`](../learn/install.md#step-2-put-kdb-in-qhome) and [`QLIC`](../learn/licensing.md#keeping-the-license-key-file-elsewhere)
+[](){#k4.lic}k4.lic | `k4.lic` file not found. If the environment variable [`QLIC`](../learn/licensing.md#keeping-the-license-key-file-elsewhere) is set, check it is set to the directory containing the license file. **Note** that it should not be set to the location of the license file itself, but to the directory that contains the license file.  If `QLIC` is not set, check that the directory specified by the environment variables [`QHOME`](../learn/install.md#step-2-put-kdb-in-qhome) contains the license file.
 [](){#os}os | Wrong OS or operating-system error (if runtime error)
 [](){#srv}srv | Client-only license in server mode
 [](){#upd}upd | Version of kdb+ more recent than update date, _or_ the function `upd` is undefined (sometimes encountered during ``-11!`:logfile``)
 [](){#user}user | Unlicensed user
-[](){#wha}wha | System date is prior to kdb+ version date
-[](){#wrong-q.k-version}wrong q.k version | `q` and `q.k` versions do not match
+[](){#wha}wha | System date is prior to kdb+ version date. Check that the system date shows the correct date.
+[](){#wrong-q.k-version}wrong q.k version | `q` and `q.k` versions do not match. Check that the `q.k` file found in the directory specified by the `QHOME` environment variable is the same version as that supplied with the q binary.
 
 
 License-related errors are reported with the prefix `licence error: ` since V4.0 2019.10.22.

diff --git a/docs/basics/internal.md b/docs/basics/internal.md
@@ -18,7 +18,7 @@ The operator `!` with a negative integer as left argument calls an internal func
 [-11!](#-11-streaming-execute)        streaming execute             -6!   [eval](../ref/eval.md)
 [-14!x](#-14x-quote-escape)       quote escape                  -7!   [hcount](../ref/hcount.md)
 [-16!x](#-16x-ref-count)       ref count                     -12!  [.Q.host](../ref/dotq.md#host-ip-to-hostname)
-[-18!x](#-18x-compress-byte)       compress byte                 -13!  [.Q.addr](../ref/dotq.md#addr-iphost-as-int)
+[-18!x](#-18x-compress-bytes)       compress bytes                -13!  [.Q.addr](../ref/dotq.md#addr-iphost-as-int)
 [-21!x](#-21x-compressionencryption-stats)       compression/encryption stats  -15!  [md5](../ref/md5.md)
 [-22!x](#-22x-uncompressed-length)       uncompressed length           -19!  [set](../ref/get.md#set)
 [-23!x](#-23x-memory-map)       memory map                    -20!  [.Q.gc](../ref/dotq.md#gc-garbage-collect)
@@ -98,6 +98,9 @@ q)-8!1 2 3
 0x010000001a000000060003000000010000000200000003000000
 ```
 
+:fontawesome-solid-hand-point-right:
+[`-9!x`](#-9x-from-bytes) (from bytes)
+
 
 ## `-9!x` (from bytes)
 
@@ -108,6 +111,9 @@ q)-9!-8!1 2 3
 1 2 3
 ```
 
+:fontawesome-solid-hand-point-right:
+[`-8!x`](#-8x-to-bytes) (to bytes), [-18!x](#-18x-compress-bytes) (compress bytes)
+
 ## `-10!x` (type enum)
 
 Resolve a [type](datatypes.md) number to an [enum](enumerations.md) vector and check if it is available.
@@ -174,9 +180,31 @@ q)-16!a
 ```
 
 
-## `-18!x` (compress byte)
+## `-18!x` (compress bytes)
+
+Returns the IPC byte representation of `x` according to [`-8!x`](#-8x-to-bytes), applying compression based on [IPC compression rules](ipc.md#compression):
+
+* Uncompressed serialized data has a length greater than 2000 bytes
+* Size of compressed data is less than &frac12; the size of uncompressed data
+
+```q
+q)count -8!til 1000     / uncompressed
+8014
+q)count -18!til 1000    / compressed
+3276
+```
 
-Returns compressed IPC byte representation of `x`, see notes about network compression in [Changes in V2.6](../releases/ChangesIn2.6.md)
+[-9!x](#-9x-from-bytes) can be used to uncompress and deserialise.
+
+```q
+q)a:til 1000           / original data to convert 
+q)x:-18!a              / serialize and compression to bytes using IPC serialisation
+q)a~-9!x               / test if deserialised version is same as original
+1b
+```
+
+:fontawesome-solid-hand-point-right:
+[`-22!x`](#-22x-uncompressed-length) (uncompressed length), [`-9!x`](#-9x-from-bytes) (from bytes), [.Q.gz](../ref/dotq.md#gz-gzip) (gzip)
 
 <!-- 
 ## `-19!` (compress file)
@@ -262,6 +290,9 @@ q)(-22!v)=count -8!v
 1b
 ```
 
+:fontawesome-solid-hand-point-right:
+[`-18!x`](#-18x-compress-bytes) (compress bytes)
+
 
 ## `-23!x` (memory map)
 
@@ -342,7 +373,7 @@ In the result, all keys except `SSLEAY_VERSION` are initialized from their corre
 [SSL](../kb/ssl.md)
 
 
-## `-27!(x;y)` (format)
+## `-27!(x;y)` (IEEE754 precision format)
 
 Where
 
@@ -351,7 +382,7 @@ Where
 
 returns `y` as a string or strings formatted as a float to `x` decimal places.
 (Since V3.6 2018.09.26.)
-It is atomic and doesn’t take `\P` into account. e.g.
+It is atomic and doesn’t take [`\P`](syscmds.md#p-precision) into account. For example:
 
 ```q
 q)-27!(3i;0 1+123456789.4567)

diff --git a/docs/basics/ipc.md b/docs/basics/ipc.md
@@ -386,7 +386,7 @@ For releases since 2012.05.29, kdb+ and the C-API will compress an outgoing mess
 -   Connection does not resolve to being localhost (since 4.1t 2021.06.04)
 -   Size of compressed data is less than &frac12; the size of uncompressed data
 
-The compression/decompression algorithms are proprietary and implemented as the `compress` and `uncompress` methods in `c.java`. The message validator does not validate the integrity of compressed messages.
+The compression/decompression algorithms are proprietary and implemented as the `compress` and `uncompress` methods in the [Java API](https://github.com/kxsystems/javakdb). The message validator does not validate the integrity of compressed messages.
 
 !!! note "Enumerations are automatically converted to values before sending through IPC."
 

diff --git a/docs/basics/syscmds.md b/docs/basics/syscmds.md
@@ -536,7 +536,7 @@ q)1%3
 :fontawesome-solid-book-open:
 [Precision](precision.md),
 [`-P` command-line option](cmdline.md#-p-display-precision),
-[`-27!` internal function](internal.md#-27xy-format)
+[`-27!` precision format with IEEE754 rounding](internal.md#-27xy-ieee754-precision-format)
 <br>
 :fontawesome-solid-globe:
 [What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)

diff --git a/docs/index.md b/docs/index.md
@@ -55,7 +55,3 @@ IDE: KX Developer](https://code.kx.com/developer/ "Download and install the free
 :fontawesome-solid-cloud:
 For cloud deployment, see [kdb Insights Core](https://code.kx.com/insights/core).
 
-
-:fontawesome-solid-globe:
-[Careers at KX and FD Technologies](http://www.firstderivatives.com/careers/)
-
diff --git a/docs/interfaces/using-c-functions.md b/docs/interfaces/using-c-functions.md
@@ -4,13 +4,13 @@ description: Functions can be written in C or C++ as shared objects and loaded i
 author: Charles Skelton
 keywords: api, c, c++, interface, kdb+, library, q
 ---
-# C/C++ functions
+# Extending q using C/C++ functions
 
 
 
-Q functionality can be extended using dynamically-loaded modules. 
+Q functionality can be extended using dynamically-loaded modules, allowing C/C++ functions to be called directly from the q language.
 
-To make a function `foo` defined in a shared object `bar.so` available in a q session, we use [2:](../ref/dynamic-load.md) to load the function dynamically.
+For example, to make a function `foo` defined in a shared object `bar.so` available in a q session, we use [2:](../ref/dynamic-load.md) to load the function dynamically.
 
 ```q
 q)foo:`bar 2:(`foo;n)

diff --git a/docs/kb/file-compression.md b/docs/kb/file-compression.md
@@ -6,9 +6,6 @@ date: August 2022
 ---
 # File compression
 
-
-
-
 kdb+ can compress data as it is written to disk.
 Q operators and keywords read both compressed and uncompressed files.
 
@@ -44,7 +41,6 @@ Cautions:
 -   Use `set` and not **gzip**: they produce different results.
 
 
-
 ### Compression parameters
 
 Compression is specified by three integers representing logical block size,   algorithm, and compression level.
@@ -229,32 +225,39 @@ However, multiple files can be read or written from their own threads concurrent
 
 ## Requirements
 
-Libraries for Gzip and Snappy may already be installed on your system.
-kdb+ binds dynamically to [Zlib](http://zlib.net) and looks for certain files for Snappy.
+Compression libraries may already be installed on your system.
+kdb+ binds dynamically to the compression libraries when required.
 
 !!! detail "64-bit and 32-bit kdb+ require corresponding 64-bit and 32-bit libs"
 
 If in doubt, consult your system administrator for assistance.
 
-algorithm | source | :fontawesome-brands-linux: Linux | :fontawesome-brands-apple: macOS | :fontawesome-brands-windows: Windows
----|---|---|---|---
-2 (Gzip) | [Zlib](http://zlib.net) | `zlib` | (pre-installed) | [WinImage](http://www.winimage.com/zLibDll/index.html "winimage.com")
-3 (Snappy) | [GitHub](http://google.github.io/snappy/) | `libsnappy.so.1` | `libsnappy.dylib` | `snappy.dll`
-4 (Lz4hc) | [GitHub](https://github.com/lz4/lz4) | `liblz4.so.1` | `liblz4.dylib` | `liblz4.dll`
-5 (zstd) | [GitHub](https://github.com/facebook/zstd) | `libzstd.so.1` | `libzstd.1.dylib` | `libzstd.dll`
+### Gzip
+
+Compression algoritm `2` uses Gzip. Source and algorithm details can be found [here](http://zlib.net).
+The following libraries are required by kdb+:
 
-=== ":fontawesome-brands-apple: macOS"
+| :fontawesome-brands-linux: Linux | :fontawesome-brands-apple: macOS | :fontawesome-brands-windows: Windows
+---|---|---
+libz.so.1 | libz.dylib<br>(pre-installed) | zlibwapi.dll<br>(32-bit and 64-bit versions available from [WinImage](http://www.winimage.com/zLibDll/index.html "winimage.com"))
 
-    To install Snappy or Lz4 on macOS, use a package manager such as [Homebrew](https://brew.sh/) or [MacPorts](https://www.macports.org/):
+### Snappy
 
-        # install with MacPorts
-        sudo port install snappy +universal
-        export LD_LIBRARY_PATH=/opt/local/lib
+Compression algoritm `3` uses Snappy. Source and algorithm details can be found [here](http://google.github.io/snappy/).
+The following libraries are required by kdb+:
 
-=== ":fontawesome-brands-windows: Windows"
+| :fontawesome-brands-linux: Linux | :fontawesome-brands-apple: macOS | :fontawesome-brands-windows: Windows
+---|---|---
+libsnappy.so.1 | libsnappy.dylib<br>(available via package managers such as [Homebrew](https://brew.sh/) or [MacPorts](https://www.macports.org/)) | snappy.dll
 
-    Build the `liblz4-dll` project on Windows as outlined in the [README at GitHub](https://github.com/lz4/lz4/tree/release/build).
+### LZ4
 
+Compression algoritm `4` uses LZ4. Source and algorithm details can be found [here](https://github.com/lz4/lz4).
+The following libraries are required by kdb+:
+
+| :fontawesome-brands-linux: Linux | :fontawesome-brands-apple: macOS | :fontawesome-brands-windows: Windows
+---|---|---
+liblz4.so.1 | liblz4.dylib<br>(available through package managers such as [Homebrew](https://brew.sh/) or [MacPorts](https://www.macports.org/)) | liblz4.dll <br>(build the `liblz4-dll` project on Windows as outlined in the [README at GitHub](https://github.com/lz4/lz4/tree/release/build))
 
 !!! danger "Certain releases of `lz4` do not function correctly within kdb+"
 
@@ -264,6 +267,15 @@ algorithm | source | :fontawesome-brands-linux: Linux | :fontawesome-brands-appl
     `lz4-1.8.3` works.
     We recommend using the latest `lz4` [release](https://github.com/lz4/lz4/releases) available.
 
+### Zstd
+
+Compression algoritm `5` uses zstd (Zstandard). Source and algorithm details can be found [here](https://github.com/facebook/zstd).
+The following libraries are required by kdb+:
+
+| :fontawesome-brands-linux: Linux | :fontawesome-brands-apple: macOS | :fontawesome-brands-windows: Windows
+---|---|---
+libzstd.so.1 | libzstd.1.dylib<br>(available via package managers such as [Homebrew](https://brew.sh/) or [MacPorts](https://www.macports.org/)) | libzstd.dll
+
 
 ## Running kdb+ under Gdb