Search bar enhancements and Help page (#823)

- Implement one idea described in #822, more specifically: add a leading wildcard to the query when the term contains a dot ".". - Fixes #819 - Move the NotFoundLinker to the linker.py module because t's now used by the help page renderer. Simplify the readthedocs theme and fix an issue that prevented to use the search bar from the module index page and other summary pages.
twisted · Nov 26, 2024 · 26cbc91 · 26cbc91
1 parent ad1fc12
commit 26cbc91
Show file tree

Hide file tree

Showing 15 changed files with 267 additions and 236 deletions.
diff --git a/.github/workflows/unit.yaml b/.github/workflows/unit.yaml
@@ -20,7 +20,7 @@ jobs:
     strategy:
       matrix:
         python-version: ['pypy-3.8', 'pypy-3.9', 'pypy-3.10', 
-                         '3.8', '3.9', '3.10', '3.11', '3.12', '3.13.0-rc.2']
+                         '3.8', '3.9', '3.10', '3.11', '3.12', '3.13']
         os: [ubuntu-latest, windows-latest, macos-latest]
 
     steps:

diff --git a/README.rst b/README.rst
@@ -91,6 +91,10 @@ in development
   from the ``pydoctor_url_path`` config option now includes a project name which defaults to 'main' (instead of putting None), 
   use mapping instead of a list to define your own project name.
 * Improve the themes so the adds injected by ReadTheDocs are rendered with the correct width and do not overlap too much with the main content.
+* Fix an issue in the readthedocs theme that prevented to use the search bar from the summary pages (like the class hierarchy).
+* The generated documentation now includes a help page under the path ``/apidocs-help.html``. 
+  This page is accessible by clicking on the information icon in the navbar (``ℹ``).
+* Improve the javascript searching code to better understand terms that contains a dot (``.``).
 
 pydoctor 24.3.3
 ^^^^^^^^^^^^^^^

diff --git a/pydoctor/linker.py b/pydoctor/linker.py
@@ -284,3 +284,16 @@ def link_xref(self, target: str, label: "Flattenable", lineno: int) -> Tag:
     def switch_context(self, ob:Optional['model.Documentable']) -> Iterator[None]:
         with self._scope_linker.switch_context(ob):
             yield
+
+class NotFoundLinker(DocstringLinker):
+    """A DocstringLinker implementation that cannot find any links."""
+
+    def link_to(self, target: str, label: "Flattenable") -> Tag:
+        return tags.transparent(label)
+
+    def link_xref(self, target: str, label: "Flattenable", lineno: int) -> Tag:
+        return tags.code(label)
+
+    @contextlib.contextmanager
+    def switch_context(self, ob: Optional[model.Documentable]) -> Iterator[None]:
+        yield
diff --git a/pydoctor/templatewriter/summary.py b/pydoctor/templatewriter/summary.py
@@ -2,6 +2,8 @@
 from __future__ import annotations
 
 from collections import defaultdict
+from string import Template
+from textwrap import dedent
 from typing import (
     TYPE_CHECKING, DefaultDict, Dict, Iterable, List, Mapping, MutableSet,
     Sequence, Tuple, Type, Union, cast
@@ -358,12 +360,141 @@ def stuff(self, request: object, tag: Tag) -> Tag:
                 ))
         return tag
 
+# TODO: The help page should dynamically include notes about the (source) code links. 
+class HelpPage(Page):
+
+    filename = 'apidocs-help.html'
+
+    RST_SOURCE_TEMPLATE = Template('''
+    Navigation
+    ----------
+
+    There is one page per class, module and package. 
+    Each page present summary table(s) which feature the members of the object.
+
+    Package or Module page
+    ~~~~~~~~~~~~~~~~~~~~~~~
+
+    Each of these pages has two main sections consisting of:
+
+    - summary tables submodules and subpackages and the members of the module or in the ``__init__.py`` file. 
+    - detailed descriptions of function and attribute members.
+
+    Class page
+    ~~~~~~~~~~
+
+    Each class has its own separate page. 
+    Each of these pages has three main sections consisting of:
+
+    - declaration, constructors, know subclasses and description
+    - summary tables of members, including inherited
+    - detailed descriptions of method and attribute members
+    
+    Entries in each of these sections are omitted if they are empty or not applicable.
+
+    Module Index
+    ~~~~~~~~~~~~
+    
+    Provides a high level overview of the packages and modules structure.
+    
+    Class Hierarchy
+    ~~~~~~~~~~~~~~~
+    
+    Provides a list of classes organized by inheritance structure. Note that ``object`` is ommited.
+
+    Index Of Names
+    ~~~~~~~~~~~~~~
+    
+    The Index contains an alphabetic index of all objects in the documentation.
+
+
+    Search
+    ------
+
+    You can search for definitions of modules, packages, classes, functions, methods and attributes. 
+    
+    These items can be searched using part or all of the name and/or from their docstrings if "search in docstrings" is enabled. 
+    Multiple search terms can be provided separated by whitespace. 
+
+    The search is powered by `lunrjs <https://lunrjs.com/>`_.
+
+    Indexing
+    ~~~~~~~~
+    
+    By default the search only matches on the name of the object. 
+    Enable the full text search in the docstrings with the checkbox option. 
+
+    You can instruct the search to look only in specific fields by passing the field name in the search like ``docstring:term``. 
+    
+    **Possible fields are**: 
+    
+    - ``name``, the name of the object (example: "MyClassAdapter" or "my_fmin_opti").
+    - ``qname``, the fully qualified name of the object (example: "lib.classses.MyClassAdapter").
+    - ``names``, the name splitted on camel case or snake case (example: "My Class Adapter" or "my fmin opti")
+    - ``docstring``, the docstring of the object (example: "This is an adapter for HTTP json requests that logs into a file...")
+    - ``kind``, can be one of: $kind_names
+        
+    Last two fields are only applicable if "search in docstrings" is enabled. 
+
+    Other search features
+    ~~~~~~~~~~~~~~~~~~~~~
+
+    Term presence. 
+        The default behaviour is to give a better ranking to object matching multiple terms of your query,
+        but still show entries that matches only one of the two terms. 
+        To change this behavour, you can use the sign ``+``.
+        
+        - To indicate a term must exactly match use the plus sing: ``+``. 
+        - To indicate a term must not match use the minus sing: ``-``.
+        
+    
+    Wildcards
+        A trailling wildcard is automatically added to each term of your query if they don't contain an explicit term presence (``+`` or ``-``). 
+        Searching for ``foo`` is the same as searching for ``foo*``. 
+        
+        If the query include a dot (``.``), a leading wildcard will to also added, 
+        searching for ``model.`` is the same as ``*model.*`` and ``.model`` is the same as ``*.model*``.
+
+        In addition to this automatic feature, you can manually add a wildcard anywhere else in the query.
+
+
+    Query examples
+    ~~~~~~~~~~~~~~
+
+    - "doc" matches "pydoctor.model.Documentable" and "pydoctor.model.DocLocation".
+    - "+doc" matches "pydoctor.model.DocLocation" but won't match "pydoctor.model.Documentable".
+    - "ensure doc" matches "pydoctor.epydoc2stan.ensure_parsed_docstring" and other object whose matches either "doc" or "ensure".
+    - "inp str" matches "java.io.InputStream" and other object whose matches either "in" or "str".
+    - "model." matches everything in the pydoctor.model module.
+    - ".web.*tag" matches "twisted.web.teplate.Tag" and related.
+    - "docstring:ansi" matches object whose docstring matches "ansi".
+    ''')
+
+    def title(self) -> str:
+        return 'Help'
+
+    @renderer
+    def heading(self, request: object, tag: Tag) -> Tag:
+        return tag.clear()("Help")
+
+    @renderer
+    def helpcontent(self, request: object, tag: Tag) -> Tag:
+        from pydoctor.epydoc.markup import restructuredtext, ParseError
+        from pydoctor.linker import NotFoundLinker
+        errs: list[ParseError] = []
+        parsed = restructuredtext.parse_docstring(dedent(self.RST_SOURCE_TEMPLATE.substitute(
+            kind_names=', '.join(f'"{k.name}"' for k in model.DocumentableKind)
+        )), errs)
+        assert not errs
+        return parsed.to_stan(NotFoundLinker())
+
 def summaryPages(system: model.System) -> Iterable[Type[Page]]:
-    pages = [
+    pages: list[type[Page]] = [
         ModuleIndexPage,
         ClassIndexPage,
         NameIndexPage,
         UndocumentedSummaryPage,
+        HelpPage, 
     ]
     if len(system.root_names) > 1:
         pages.append(IndexPage)

diff --git a/pydoctor/test/__init__.py b/pydoctor/test/__init__.py
@@ -1,24 +1,18 @@
 """PyDoctor's test suite."""
 
-import contextlib
 from logging import LogRecord
-from typing import Iterable, TYPE_CHECKING, Iterator, Optional, Sequence
+from typing import Iterable, TYPE_CHECKING, Sequence
 import sys
 import pytest
 from pathlib import Path
 
-from twisted.web.template import Tag, tags
-
 from pydoctor import epydoc2stan, model
 from pydoctor.templatewriter import IWriter, TemplateLookup
-from pydoctor.epydoc.markup import DocstringLinker
-
-if TYPE_CHECKING:
-    from twisted.web.template import Flattenable
+from pydoctor.linker import NotFoundLinker
 
 posonlyargs = pytest.mark.skipif(sys.version_info < (3, 8), reason="requires python 3.8")
 typecomment = pytest.mark.skipif(sys.version_info < (3, 8), reason="requires python 3.8")
-
+NotFoundLinker = NotFoundLinker
 
 # Because pytest 6.1 does not yet export types for fixtures, we define
 # approximations that are good enough for our test cases:
@@ -87,18 +81,4 @@ def _writeDocsFor(self, ob: model.Documentable) -> None:
 
         for o in ob.contents.values():
             self._writeDocsFor(o)
-
-
-class NotFoundLinker(DocstringLinker):
-    """A DocstringLinker implementation that cannot find any links."""
-
-    def link_to(self, target: str, label: "Flattenable") -> Tag:
-        return tags.transparent(label)
-
-    def link_xref(self, target: str, label: "Flattenable", lineno: int) -> Tag:
-        return tags.code(label)
-
-    @contextlib.contextmanager
-    def switch_context(self, ob: Optional[model.Documentable]) -> Iterator[None]:
-        yield
 
diff --git a/pydoctor/test/test_commandline.py b/pydoctor/test/test_commandline.py
@@ -304,6 +304,16 @@ def test_index_hardlink(tmp_path: Path) -> None:
     assert not (tmp_path / 'basic.html').is_symlink()
     assert (tmp_path / 'basic.html').is_file()
 
+
+def test_apidocs_help(tmp_path: Path) -> None:
+    """
+    Checks that the help page is well generated.
+    """
+    exit_code = driver.main(args=['--html-output', str(tmp_path), 'pydoctor/test/testpackages/basic/'])
+    assert exit_code == 0
+    help_page = (tmp_path / 'apidocs-help.html').read_text()
+    assert '>Search</h2>' in help_page
+
 def test_htmlbaseurl_option_all_pages(tmp_path: Path) -> None:
     """
     Check that the canonical link is included in all html pages, including summary pages.
@@ -319,4 +329,5 @@ def test_htmlbaseurl_option_all_pages(tmp_path: Path) -> None:
         if t.stem == 'basic':
             filename = 'index.html' # since we have only one module it's linked as index.html
         assert f'<link rel="canonical" href="https://example.com.abcde/{filename}"' in t.read_text(encoding='utf-8')
-
+
+
diff --git a/pydoctor/test/testcustomtemplates/allok/nav.html b/pydoctor/test/testcustomtemplates/allok/nav.html
@@ -1,5 +1,5 @@
 <nav class="navbar navbar-default" xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1">
-  <meta name="pydoctor-template-version" content="3" />
+  <meta name="pydoctor-template-version" content="4" />
 
     <div class="container">
 

diff --git a/pydoctor/themes/base/apidocs-help.html b/pydoctor/themes/base/apidocs-help.html
@@ -0,0 +1,39 @@
+<!DOCTYPE html>
+
+<html xmlns:t="http://twistedmatrix.com/ns/twisted.web.template/0.1" lang="EN">
+  <meta name="pydoctor-template-version" content="1" />
+
+  <t:transparent t:render="head">Head</t:transparent>
+
+  <body>
+
+    <div t:render="header" />
+
+    <nav class="navbar navbar-default mainnavbar">
+      <t:transparent t:render="nav" /></nav>
+
+    <div class="container-fluid">
+
+      <div class="page-header">
+        <h1 t:render="heading">Heading</h1>
+        <div id="showPrivate">
+          <button class="btn btn-link" onclick="togglePrivate()">Toggle Private API</button>
+        </div>
+      </div>
+
+    <div t:render="helpcontent" />
+
+    <hr />
+
+    <p>
+      <a href="https://github.com/twisted/pydoctor/issues">Report a bug or suggest an enhancement</a>
+    </p>
+
+    </div>
+    <footer class="container-fluid" t:render="footer" />
+
+    <script src="pydoctor.js" type='text/javascript'></script>
+
+  </body>
+
+</html>
diff --git a/pydoctor/themes/base/apidocs.css b/pydoctor/themes/base/apidocs.css
@@ -60,7 +60,6 @@ nav.navbar .navbar-header {
     margin-bottom: 3px;
     border-bottom: 0;
     box-shadow: 0 0 8px 8px #fff;
-    z-index: 99;
 }
 
 .navbar-brand {
@@ -1064,19 +1063,15 @@ input[type="search"]::-webkit-search-results-decoration { display: none; }
     padding: 5px 0 0 8px;
 }
 
-.search-help-hidden #search-help-box{
-    display: none!important;
-}
-
-#search-help-button{
+#apidocs-help-button{
     background-color: #e6e6e6;
 }
 
-.search-help-hidden #search-help-button{
+.search-help-hidden #apidocs-help-button{
     background-color: rgb(255, 255, 255);
 }
 
-.search-help-hidden #search-help-button:hover {
+.search-help-hidden #apidocs-help-button:hover {
     background-color: #e6e6e6;
 }