Skip to content

Commit

Permalink
MR for 2.5.2 feature additions
Browse files Browse the repository at this point in the history
  • Loading branch information
@cmccarthy1
cmccarthy1 committed Jul 5, 2024
1 parent 57a42b4 commit 875e047
Show file tree
Hide file tree
Showing 16 changed files with 403 additions and 123 deletions.
225 changes: 132 additions & 93 deletions docs/getting-started/q_magic_command.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,31 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "5c1e9b66",
"metadata": {},
"source": [
"# Jupyter q Magic Command Notebook\n",
"\n",
"_The purpose of this notebook is to exemplify how to use the q Magic command in a Jupyter notebook._\n",
"\n",
"\n",
"The Jupyter q magic command in PyKX allows you to execute q code within a Jupyter notebook. It provides seamless integration with the q programming language.\n",
"\n",
"This example Notebook has the following sections:\n",
"\n",
"1. [Import PyKX](#1-import-pykx)\n",
"1. [Create the external q process](#2-create-the-external-q-process)\n",
"1. [Execute against Embedded q](#3-execute-against-embedded-q)\n",
"1. [SQL interface](#4-sql-interface)\n",
"1. [q namespaces](#5-q-namespaces)\n",
"1. [(Advanced) q over IPC](#6-advanced-q-over-ipc)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f25482a",
"metadata": {
"tags": [
"hide_code"
Expand All @@ -15,6 +38,16 @@
"os.environ['PYKX_Q_LOADED_MARKER'] = '' # Only used here for running Notebook under mkdocs-jupyter during document generation."
]
},
{
"cell_type": "markdown",
"id": "688b9ed0",
"metadata": {},
"source": [
"## 1. Import PyKX\n",
"\n",
"To run this example, first import the PyKX library:"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -30,12 +63,9 @@
"id": "9c520c21",
"metadata": {},
"source": [
"#### Create the external q process\n",
"To run this example, the Python code in the following cell will do the equivalent to executing the following in a terminal:\n",
"## 2. Create the external q process\n",
"\n",
"```\n",
"$ q -p 5001\n",
"```"
"You can run an external q process by using the following Python code:"
]
},
{
Expand All @@ -58,13 +88,27 @@
" raise kx.QError('Unable to create q process on port 5000')"
]
},
{
"cell_type": "markdown",
"id": "1b318ba2",
"metadata": {},
"source": [
"\n",
"Or execute this command in a terminal:\n",
"\n",
"```sh\n",
"$ q -p 5000\n",
"```\n"
]
},
{
"cell_type": "markdown",
"id": "bc7219fb",
"metadata": {},
"source": [
"#### Executing against Embedded q\n",
"A cell beginning with `%%q` will execute q within `PyKX`'s `EmbeddedQ` module."
"## 3. Execute against Embedded q\n",
"\n",
"To execute q code within PyKX's `EmbeddedQ` module, type `%%q` at the beginning of the cell:"
]
},
{
Expand All @@ -83,23 +127,18 @@
"id": "89ec26e4",
"metadata": {},
"source": [
"#### Execution options\n",
"\n",
"Execution options can also be included after `%%q`.\n",
"After `%%q` you can further add two execution options:\n",
"\n",
"Here is the list of currently supported execution options.\n",
"\n",
"```\n",
"--debug: prints the q backtrace before raising a QError\n",
" if the cell errors\n",
"--display: calls display rather than the default print\n",
" on returned objects\n",
"```\n"
"| **Execution option** | **Description** |\n",
"|---------------|----------------------------------------------------|\n",
"| --debug | Prints the q backtrace before raising a QError if the cell gives an error.|\n",
"| --display | Calls display rather than the default print on returned objects.|"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f9ed8310",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -110,6 +149,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "daa9a196",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -119,156 +159,155 @@
},
{
"cell_type": "markdown",
"id": "89ec26e4",
"id": "2905895e",
"metadata": {},
"source": [
"#### Executing against an external q process over IPC\n",
"## 4. SQL interface\n",
"\n",
"Connection information can also be included after the `%%q` to connect to a remote `q` process over\n",
"IPC.\n",
"The `s)` syntax runs SQL queries against local tables within the `q` process.\n",
"\n",
"Here is the list of currently supported connection parameters.\n",
"If they specify a type a second value is expected to follow them to be used as the parameter.\n",
"If no type follows them they can be used as a stand alone flag.\n",
"\n",
"```\n",
"--host: A string object denoting the host to connect to\n",
"--port: An int object denoting the port to connect over\n",
"--user: A str object denoting the username to use when connecting\n",
"--password: A str object denoting the password to use when connecting\n",
"--timeout: A float object denoting the time in seconds before the query\n",
" times out, defaults to no timeout\n",
"--nolarge: Disable messages over 2GB being sent / received\n",
"--tls: Use a tls connection\n",
"--unix: Use a unix connection\n",
"--reconnection_attempts: An int object denoting how many\n",
" reconnection attempts to make\n",
"--noctx: Disable the context interface\n",
"```\n",
"\n",
"Connect to a q server running on `localhost` at port `5001` as `user` using password `password`\n",
"and disable the context interface."
"Note: To use the SQL interface, first you need to load the `s.k_` library."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1faca1e1",
"metadata": {},
"id": "56220bb5",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"%%q --host localhost --port 5000 --user user --pass password --noctx\n",
"til 10"
"%%q\n",
"\\l s.k_\n",
"tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);\n",
"s) select * from tab where a>500 and b<250.0 limit 5"
]
},
{
"cell_type": "markdown",
"id": "f046ebb6",
"id": "da906296",
"metadata": {},
"source": [
"All connection arguments are optional with the exception of the `--port` argument. If `--host` is not provided `localhost` will be used as the default host."
"## 5. q namespaces\n",
"\n",
"You can use `q` namespaces, and switch between them with `\\d`.\n",
"\n",
"Note: The namespace is reset back to the base namespace `.` between cells."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "615d7d2e",
"id": "502af937",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);"
]
},
{
"cell_type": "markdown",
"id": "d756f342",
"metadata": {},
"source": [
"It is possible to execute `q` code spanning multiple lines."
"%%q\n",
"\\d .example\n",
"f: {[x] til x};"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c739a80a",
"metadata": {
"scrolled": true
},
"id": "58d0c7c9",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"afunc: {[x; y]\n",
" x + y \n",
" };\n",
"afunc[0; 1]\n",
"afunc[2; 3]"
"%%q\n",
"\\d\n",
".example.f[10]"
]
},
{
"cell_type": "markdown",
"id": "2905895e",
"id": "52ca850e",
"metadata": {},
"source": [
"#### Using the SQL interface\n",
"The `s)` syntax to run SQL queries against local tables within the `q` process.\n",
"## 6. (Advanced) q over IPC\n",
"\n",
"After `%%q` you can include connection information, if you wish to connect to a remote `q` process over IPC. \n",
"\n",
"Note: The `s.k_` library must be loaded first to use the SQL interface"
"The list of supported connection parameters is below. The rule is:\n",
"\n",
"- If they have a type, it must be followed by a second value/parameter.\n",
"- If there's no type after them, you can use them as a standalone flag.\n",
"\n",
"| **Parameter** &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| **Object type and description**|\n",
"|-----------------------|-----------------------------------------------|\n",
"|--host | (string) The host to connect to. |\n",
"|--port | (integer) The port to connect over. |\n",
"|--user | (string) The username to use when connecting. |\n",
"|--password | (string) The password to use when connecting. |\n",
"|--timeout | (float) The time in seconds before the query times out. Defaults to no timeout.|\n",
"|--nolarge | Disable messages over 2GB being sent / received. |\n",
"|--tls | Use a tls connection. |\n",
"|--unix | Use a unix connection. |\n",
"|--reconnection_attempts| (integer) How many reconnection attempts to make.|\n",
"|--noctx | Disable the context interface. |\n",
"\n",
"Connect to a q server running on `localhost` at port `5000` as `user` using password `password`\n",
"and disable the context interface."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56220bb5",
"metadata": {
"scrolled": true
},
"id": "a282e069",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"\\l s.k_\n",
"s) select * from tab where a>500 and b<250.0 limit 5"
"%%q --host localhost --port 5000 --user user --pass password --noctx\n",
"til 10"
]
},
{
"cell_type": "markdown",
"id": "da906296",
"id": "a1fe3b8e",
"metadata": {},
"source": [
"#### Using namespaces\n",
"You can also use `q` namespaces, and switch between them using `\\d`.\n",
"\n",
"Note: The namespace is reset back to the base namespace `.` between cells."
"All connection arguments are optional, except the `--port` argument. If `--host` is not provided `localhost` is the default host."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "502af937",
"id": "18d8416b",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"\\d .example\n",
"f: {[x] til x};"
"tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG);"
]
},
{
"cell_type": "markdown",
"id": "e143c382",
"metadata": {},
"source": [
"Note that it's possible to execute `q` code spanning multiple lines:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "58d0c7c9",
"id": "ccb197e8",
"metadata": {},
"outputs": [],
"source": [
"%%q --port 5000\n",
"\\d\n",
".example.f[10]"
"afunc: {[x; y]\n",
" x + y \n",
" };\n",
"afunc[0; 1]\n",
"afunc[2; 3]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "334d0ac5",
"id": "c12a7d38",
"metadata": {},
"outputs": [],
"source": [
Expand Down
Loading

0 comments on commit 875e047

Please sign in to comment.