Structural Pattern Matching

It would be cool if kajiki could generate code using PEP622 pattern matching when running on python3.10+

Benefits
Speed

Motivation

I wanted to do something easy
Directive Lookup

py:match was used by genshi, but it's free within kajiki
py:case is used together with py:switch (and py:else)
Directive Proposal

py:match (with on attribute) and py:case (with match attribute instead of value)

so kajiki have always been ahead of time :D

running the switch example
I see the code generated is

In [5]: print(Template.py_text)
class template:
    @kajiki.expose
    def __main__():
        yield '<div>\n'
        yield self.__kj__.escape(i)
        yield local.__kj__.gettext(' is')
        yield ' '
        local.__kj__.push_switch(i % 2)
        if False: pass
        elif local.__kj__.case(0):
            yield local.__kj__.gettext('even')
        else:
            yield local.__kj__.gettext('odd')
        local.__kj__.pop_switch()
        yield '</div>'
template = kajiki.Template(template)

I see switches are pushed to a list to allow them to be nested.
and case does the equality check

But of course due to the concepts behind it being different, the two approaches are incompatible.

I'm doing it.

I noticed after the dom is parsed with sax, it is reconstructed due to some optimization, i was loosing the attribute tough, so i added support to define tuples of attributes in the directives.

I also noticied the flattener does not flatten my deep stuff.
But since the backtrace was just crazy, instead of fixing the flattener I made a bad patch for generate_python

ok, I got the loose of the attribute was caused because py:case can be used even within other tags.
patched.

This is a squash of 14 commits:

tests and doc added too
structural pattern matching
added some tests for SPM
patched py:case directive used as attribute, added doc for pep622
test fixes
skip pep622 tests on python<3.10
one more test
black reformatting, flake8 decected F401
i don't like isort
follow @jackrosenthal review
comment py:match on genshi migration guide
rephrasing docs, adding newline to gitignore
support for python 3.11.0b3 fixed?
docs: Add table with summary of the how XML directives can be used
docs: add py:match to directives summary added in PR #71

Closes #76, #74

.gitignore

@@ -46,3 +46,9 @@ pypyenv3
 .mr.developer.cfg
 .project
 .pydevproject
+
+
+# emacs
+\#*\#
+.\#*
+flycheck_*

docs/migrating_from_genshi.rst

@@ -18,9 +18,10 @@ identical to those of Genshi_.
  * ``py:strip``
  * ``xi:include`` -- renamed ``py:include``
 
-Note that, in particular, ``py:match`` is not supported.  But Kajiki
-supports the following additional directives:
+Note that, in particular, py:match in Kajiki differs from Genshi, implementing `PEP634 <https://peps.python.org/pep-0636/>`_.
 
+Kajiki also supports the following additional directives not in Genshi:
+
  * ``py:extends`` - indicates that this is an extension template.  The parent
    template will be read in and used for layout, with any ``py:block`` directives in
    the child template overriding the ``py:block`` directives defined in the parent.

docs/xml-templates.rst

@@ -171,6 +171,32 @@ Perform multiple tests to render one of several alternatives.  The first matchin
 <div>
 3 is odd</div>
 
+py:match, py:case
+^^^^^^^^^^^^^^^^^
+
+Similar to ``py:switch`` this makes use of `PEP622 <https://peps.python.org/pep-0622/>`_
+Structural Pattern Matching:
+
+>>> import sys, pytest
+>>> if sys.version_info < (3, 10): pytest.skip('pep622 unsupported')
+>>> Template = kajiki.XMLTemplate('''<div>
+... $i is <py:match on="i % 2">
+... <py:case match="0">even</py:case>
+... <py:case match="_">odd</py:case>
+... </py:match></div>''')
+>>> print(Template(dict(i=4)).render())
+<div>
+4 is even</div>
+>>> print(Template(dict(i=3)).render())
+<div>
+3 is odd</div>
+
+.. note::
+
+   ``py:match`` compiles directly to Python's ``match`` syntax, and will
+   therefore not work on versions less than 3.10.  Only use this syntax if your
+   project targets Python 3.10 or newer.
+
 py:for
 ^^^^^^^^^^^^^
 
@@ -465,7 +491,8 @@ Directive  Usable as an attribute  Usable as a separate element When used as a s
 py:if      ✅                       ✅                            test
 py:else    ✅                       ✅
 py:switch  ❌                       ✅                            test
-py:case    ✅                       ✅                            value
+py:match   ❌                       ✅                            on
+py:case    ✅                       ✅                            value or match (for usage with py:switch or py:match)
 py:for     ✅                       ✅                            each
 py:def     ✅                       ✅                            function
 py:call    ❌                       ✅                            args, function

kajiki/ir.py

@@ -12,7 +12,12 @@ def generate_python(ir):
         elif isinstance(node, DedentNode):
             cur_indent -= 4
         for line in node.py():
-            yield line.indent(cur_indent)
+            if isinstance(line, IndentNode):
+                cur_indent += 4
+            elif isinstance(line, DedentNode):
+                cur_indent -= 4
+            else:
+                yield line.indent(cur_indent)
 
 
 class Node:
@@ -252,6 +257,40 @@ def py(self):
         yield self.line(f"elif local.__kj__.case({self.decl}):")
 
 
+class MatchNode(HierNode):
+    """Structural Pattern Matching Node"""
+
+    def __init__(self, decl, *body):
+        super().__init__(body)
+        self.decl = decl
+
+    def py(self):
+        yield self.line(f"match ({self.decl}):")
+        yield IndentNode()
+
+    def __iter__(self):
+        yield self
+        yield from self.body_iter()
+        yield DedentNode()
+
+
+class MatchCaseNode(HierNode):
+    """Structural Pattern Matching Case Node"""
+
+    def __init__(self, decl, *body):
+        super().__init__(body)
+        self.decl = decl
+
+    def py(self):
+        yield self.line(f"case {self.decl}:")
+        yield IndentNode()
+
+    def __iter__(self):
+        yield self
+        yield from self.body_iter()
+        yield DedentNode()
+
+
 class IfNode(HierNode):
     def __init__(self, decl, *body):
         super().__init__(body)

kajiki/markup_template.py

@@ -1,7 +1,7 @@
 DIRECTIVES = [
     ("def", "function"),
     ("call", "function"),
-    ("case", "value"),
+    ("case", ("value", "match")),
     ("else", ""),
     ("for", "each"),
     ("if", "test"),

kajiki/xml_template.py

@@ -4,6 +4,7 @@
 import io
 import re
 from codecs import open
+from sys import version_info
 from xml import sax
 from xml.dom import minidom as dom
 from xml.sax import SAXParseException
@@ -431,10 +432,50 @@ def _compile_switch(self, node):
 
         yield ir.SwitchNode(node.getAttribute("test"), *body)
 
+    @annotate
+    def _compile_match(self, node):
+        """Convert py:match nodes to their IR."""
+        if version_info < (3, 10):
+            msg = "At least Python 3.10 is required to use the py:match directive"
+            raise XMLTemplateCompileError(
+                msg,
+                doc=self.doc,
+                filename=self.filename,
+                linen=node.lineno,
+            )
+        body = []
+
+        # Filter out empty text nodes and report unsupported nodes
+        for n in self._compile_nop(node):
+            if isinstance(n, ir.TextNode) and not n.text.strip():
+                continue
+            elif not isinstance(n, ir.MatchCaseNode):
+                msg = "py:match directive can only contain py:case nodes and cannot be placed on a tag."
+                raise XMLTemplateCompileError(
+                    msg,
+                    doc=self.doc,
+                    filename=self.filename,
+                    linen=node.lineno,
+                )
+            body.append(n)
+
+        yield ir.MatchNode(node.getAttribute("on"), *body)
+
     @annotate
     def _compile_case(self, node):
         """Convert py:case nodes to their intermediate representation."""
-        yield ir.CaseNode(node.getAttribute("value"), *list(self._compile_nop(node)))
+        if node.getAttribute("value"):
+            yield ir.CaseNode(node.getAttribute("value"), *list(self._compile_nop(node)))
+        elif node.getAttribute("match"):
+            yield ir.MatchCaseNode(node.getAttribute("match"), *list(self._compile_nop(node)))
+        else:
+            msg = "case must have either value or match attribute, the former for py:switch, the latter for py:match"
+            raise XMLTemplateCompileError(
+                msg,
+                doc=self.doc,
+                filename=self.filename,
+                linen=node.lineno,
+            )
 
     @annotate
     def _compile_if(self, node):
@@ -905,7 +946,7 @@ def _expand_directives(cls, tree, parent=None):
             </py:if>
 
         This ensures that whenever a template is processed there is no
-        different between the two formats as the Compiler will always
+        difference between the two formats as the Compiler will always
         receive the latter.
         """
         if isinstance(tree, dom.Document):
@@ -914,7 +955,11 @@ def _expand_directives(cls, tree, parent=None):
         if not isinstance(getattr(tree, "tagName", None), str):
             return tree
         if tree.tagName in QDIRECTIVES_DICT:
-            tree.setAttribute(tree.tagName, tree.getAttribute(QDIRECTIVES_DICT[tree.tagName]))
+            attrs = QDIRECTIVES_DICT[tree.tagName]
+            if not isinstance(attrs, tuple):
+                attrs = [attrs]
+            for attr in attrs:
+                tree.setAttribute(tree.tagName, tree.getAttribute(attr))
             tree.tagName = "py:nop"
         if tree.tagName != "py:nop" and tree.hasAttribute("py:extends"):
             value = tree.getAttribute("py:extends")
@@ -931,7 +976,17 @@ def _expand_directives(cls, tree, parent=None):
             # nsmap = (parent is not None) and parent.nsmap or tree.nsmap
             el = tree.ownerDocument.createElement(directive)
             el.lineno = tree.lineno
-            if attr:
+            if isinstance(attr, tuple):
+                # eg: handle bare py:case tags
+                for at in attr:
+                    el.setAttribute(at, dict(tree.attributes.items()).get(at))
+                if directive == "py:case" and tree.nodeName != "py:case":
+                    if tree.parentNode.nodeName == "py:match" or "py:match" in tree.parentNode.attributes:
+                        at = "on"
+                    else:
+                        at = "value"
+                    el.setAttribute(at, value)
+            elif attr:
                 el.setAttribute(attr, value)
             # el.setsourceline = tree.sourceline
             parent.replaceChild(newChild=el, oldChild=tree)

tests/test_ir.py

@@ -1,5 +1,8 @@
+import sys
 from unittest import TestCase
 
+import pytest
+
 import kajiki
 from kajiki import ir
 
@@ -49,6 +52,35 @@ def test_basic(self):
         assert rsp == "0 is even\n1 is odd\n", rsp
 
 
+class TestMatch:
+    def setup_class(self):
+        if sys.version_info < (3, 10):
+            pytest.skip("pep622 unavailable before python3.10")
+
+        self.tpl = ir.TemplateNode(
+            defs=[
+                ir.DefNode(
+                    "__main__()",
+                    ir.ForNode(
+                        "i in range(2)",
+                        ir.ExprNode("i"),
+                        ir.TextNode(" is "),
+                        ir.MatchNode(
+                            "i % 2",
+                            ir.MatchCaseNode("0", ir.TextNode("even\n")),
+                            ir.MatchCaseNode("_", ir.TextNode("odd\n")),
+                        ),
+                    ),
+                )
+            ]
+        )
+
+    def test_basic(self):
+        tpl = kajiki.template.from_ir(self.tpl)
+        rsp = tpl({}).render()
+        assert rsp == "0 is even\n1 is odd\n", rsp
+
+
 class TestFunction(TestCase):
     def setUp(self):
         self.tpl = ir.TemplateNode(

tests/test_xml.py

@@ -1,4 +1,5 @@
 import os
+import sys
 import traceback
 import xml.dom.minidom
 from io import BytesIO
@@ -65,9 +66,12 @@ def test_expand(self):
                 continue
             assert node.tagName == tagname, f"{node.tagName} != {tagname}"
             if attr:
-                assert len(node.attributes) == 1
-                assert node.hasAttribute(attr)
-                assert node.getAttribute(attr) == tagname.split(":")[-1]
+                if node.tagName != "py:case":
+                    assert len(node.attributes) == 1, node.attributes.items()
+                    assert node.hasAttribute(attr)
+                    assert node.getAttribute(attr) == tagname.split(":")[-1]
+                else:
+                    assert len(node.attributes) == 2
             else:
                 assert len(node.attributes) == 0
             assert len(node.childNodes) == 1
@@ -300,6 +304,55 @@ def test_switch_div(self):
         assert "py:switch directive can only contain py:case and py:else nodes" in str(e)
 
 
+class TestMatch:
+    def setup_class(self):
+        if sys.version_info < (3, 10):
+            pytest.skip("pep622 unavailable before python3.10")
+
+    def test_match(self):
+        perform(
+            """<div py:for="i in range(2)">
+$i is <py:match on="i % 2">
+<py:case match="0">even</py:case>
+<py:case match="_">odd</py:case>
+</py:match></div>""",
+            """<div>
+0 is even</div><div>
+1 is odd</div>""",
+        )
+
+    def test_match_div(self):
+        with pytest.raises(
+            XMLTemplateCompileError,
+            match="case must have either value or match attribute, the former for py:switch, the latter for py:match",
+        ):
+            perform(
+                """
+        <div class="test" py:match="5 == 3">
+            <p py:case="True">True</p>
+            <p py:case="_">False</p>
+        </div>""",
+                "<div><div>False</div></div>",
+            )
+
+    def test_match_aliens(self):
+        with pytest.raises(
+            XMLTemplateCompileError,
+            match="py:match directive can only contain py:case",
+        ):
+            perform(
+                """<div py:for="i in range(2)">
+$i is <py:match on="i % 2">
+alien
+<py:case match="0">even</py:case>
+<py:case match="_">odd</py:case>
+</py:match></div>""",
+                """<div>
+0 is even</div><div>
+1 is odd</div>""",
+            )
+
+
 class TestElse(TestCase):
     def test_pyif_pyelse(self):
         with pytest.raises(XMLTemplateCompileError) as e: