Skip to content

Commit

Permalink
Added null check methods (#27)
Browse files Browse the repository at this point in the history
* Added initial isnull isna and notna implementations and tests

* Proper alias, added tests and documentation

* Reworked tests and added notnull method
  • Loading branch information
nipsn authored Feb 9, 2024
1 parent d28e93a commit 460a218
Show file tree
Hide file tree
Showing 3 changed files with 186 additions and 1 deletion.
140 changes: 139 additions & 1 deletion docs/user-guide/advanced/Pandas_API.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2335,6 +2335,144 @@
"tab.any()"
]
},
{
"cell_type": "markdown",
"id": "5e21bef1",
"metadata": {},
"source": [
"### Table.isna()\n",
"\n",
"```\n",
"Table.isna()\n",
"```\n",
"\n",
"Detects null values on a Table object.\n",
"\n",
"**Parameters:**\n",
"\n",
"\n",
"**Returns:**\n",
"\n",
"| Type | Description |\n",
"| :----------------: | :------------------------------------------------------------------- |\n",
"| Table | A Table with the same shape as the original but containing boolean values. 1b represents a null value was on its place and 0b represents the opposite. |"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d8ff16e1",
"metadata": {},
"outputs": [],
"source": [
"tab.isna()"
]
},
{
"cell_type": "markdown",
"id": "47d20b00",
"metadata": {},
"source": [
"### Table.isnull()\n",
"\n",
"```\n",
"Table.isnull()\n",
"```\n",
"\n",
"Alias of Table.isna().\n",
"\n",
"Detects null values on a Table object.\n",
"\n",
"**Parameters:**\n",
"\n",
"\n",
"**Returns:**\n",
"\n",
"| Type | Description |\n",
"| :----------------: | :------------------------------------------------------------------- |\n",
"| Table | A Table with the same shape as the original but containing boolean values. 1b represents a null value was on its place and 0b represents the opposite. |"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "400c209e",
"metadata": {},
"outputs": [],
"source": [
"tab.isnull()"
]
},
{
"cell_type": "markdown",
"id": "fb3164d5",
"metadata": {},
"source": [
"### Table.notna()\n",
"\n",
"```\n",
"Table.notna()\n",
"```\n",
"\n",
"Boolean inverse of Table.isna().\n",
"\n",
"Detects non-null values on a Table object.\n",
"\n",
"**Parameters:**\n",
"\n",
"\n",
"**Returns:**\n",
"\n",
"| Type | Description |\n",
"| :----------------: | :------------------------------------------------------------------- |\n",
"| Table | A Table with the same shape as the original but containing boolean values. 1b represents a non null value was on its place and 0b represents the opposite. |"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4206eec3",
"metadata": {},
"outputs": [],
"source": [
"tab.notna()"
]
},
{
"cell_type": "markdown",
"id": "4e8e5c07",
"metadata": {},
"source": [
"### Table.notnull()\n",
"\n",
"```\n",
"Table.notna()\n",
"```\n",
"\n",
"Boolean inverse of Table.isnull(). Alias of Table.isna()\n",
"\n",
"Detects non-null values on a Table object.\n",
"\n",
"**Parameters:**\n",
"\n",
"\n",
"**Returns:**\n",
"\n",
"| Type | Description |\n",
"| :----------------: | :------------------------------------------------------------------- |\n",
"| Table | A Table with the same shape as the original but containing boolean values. 1b represents a non null value was on its place and 0b represents the opposite. |"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3138d21a",
"metadata": {},
"outputs": [],
"source": [
"tab.notnull()"
]
},
{
"cell_type": "markdown",
"id": "a3c3fccd",
Expand All @@ -2360,7 +2498,7 @@
"\n",
"| Type | Description |\n",
"| :----------------: | :------------------------------------------------------------------- |\n",
"| Dictionary | A dictionary where the key represent the column name / row number and the values are the result of calling `max` on that column / row. |"
"| Dictionary | A dictionary where the key represent the column name / row number and the values are the result of calling `max` on that column / row. |"
]
},
{
Expand Down
16 changes: 16 additions & 0 deletions src/pykx/pandas_api/pandas_meta.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,3 +311,19 @@ def agg(self, func, axis=0, *args, **kwargs): # noqa: C901
return data
else:
return (q('{(flip enlist[`function]!enlist x)!y}', keyname, data))

@api_return
def isna(self):
return q.null(self)

@api_return
def isnull(self):
return self.isna()

@api_return
def notna(self):
return q('not', self.isna())

@api_return
def notnull(self):
return self.notna()
31 changes: 31 additions & 0 deletions tests/test_pandas_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -2029,3 +2029,34 @@ def test_keyed_loc_fixes(q):
mkt[['k1', 'y']]
with pytest.raises(KeyError):
mkt['k1']


def test_isnull(q):
tab = q('''([]
g:1#0Ng; h:1#0Nh; i1:1#0Ni; j:1#0Nj;
e:1#0Ne; f:1#0Nf; s:1#` ; p:1#0Np;
m:1#0Nm; d:1#0Nd; n:1#0Nn; u:1#0Nu;
v:1#0Nv; t:1#0Nt; c:1#" ";
g2:1?0Ng; h2:1?0Wh; i2:1?10i; j2:1?10j;
e2:1?10e; f2:1?10f; s2:1#`foo;p2:1?10p;
m2:1?"m"$10;d2:1?"d"$10;n2:1?10n; u2:1?10u;
v2:1?10v; t2:1?10t; c2:1?" ")
''')

cols = ["g", "h", "i1", "j",
"e", "f", "s", "p",
"m", "d", "n", "u",
"v", "t", "c",
"g2", "h2", "i2", "j2",
"e2", "f2", "s2", "p2",
"m2", "d2", "n2", "u2",
"v2", "t2", "c2"]

expected = pd.DataFrame.from_dict({c: [True] if i < 15 else [False]
for i, c in enumerate(cols)})
expected_inv = ~expected

pd.testing.assert_frame_equal(tab.isna().pd(), expected)
pd.testing.assert_frame_equal(tab.isnull().pd(), expected)
pd.testing.assert_frame_equal(tab.notna().pd(), expected_inv)
pd.testing.assert_frame_equal(tab.notnull().pd(), expected_inv)

0 comments on commit 460a218

Please sign in to comment.