Help Wanted: Learn how to replace existing Sphinx Search

[cwarnermm]

Learn how to replace the existing search functionality with a third-party search technology, such as SwiftType.

The benefits of replacing the default Sphinx search include better capacity for cross-tool integration, more relevant search results, easier to navigate search results.

You can find information about the default Sphinx search here: https://stackoverflow.com/questions/44154226/how-do-i-customize-sphinx-rtd-theme-default-search-settings. Information about Sphinx configuration options can be found here: https://www.sphinx-doc.org/en/master/usage/configuration.html.

[Sayanta66]

Hi, @cwarnermm can you assign this issue to me? I will work on it. And, can you also point out the document that needs to be fixed?

[cwarnermm]

Hi @Sayanta66. This ticket is a bit different from the tickets requesting content updates. This technology ticket is to change the default search engine in the product documentation (available at docs.mattermost.com) to another engine entirely -- something more robust and provides better search results to users. The Mattermost Support team is using an instance of SwiftType, and that may be a good engine to try out.

This ticket is ideal for someone interested in cloning an instance of the docs repository in a local development environment, and learning how the default search engine works, how it's configured, and then learn how it can be replaced with another engine entirely. There will be considerable investigation and technical learning involved with this one, which is why it's labelled as a Medium Difficulty ticket.

With this context in mind, would you like me to assign this ticket to you?

[Sayanta66]

Okay so for a beginner like me probably it is not a good fit, right?

[cwarnermm]

This ticket is a good fit for any contributor at any level who would like to:

• learn how the default Sphinx search works natively out-of-the-box
• identify how the default search engine can be replaced with another service entirely that offers better search capabilities

However, I wouldn't classify this as a good first issue ticket because of the complexity and technical nature of the work involved.

That being said, if you haven't done so already, I highly encourage you to clone a copy of the docs repo locally, and try building the docs yourself locally to get a better sense of how the docs repository works, how the content is generated, and what source files make up the various components of what you see on docs.mattermost.com.

Working with the docs repository locally requires a local development environment, and requires specific development dependencies installed locally on your machine.

See the docs repository Readme for details.

[TomerPacific]

@cwarnermm - What options are you guys looking for as a replacement for Sphinx? Or is this just to configure Sphinx in a different way?

[cwarnermm]

Good question, @TomerPacific. Our native Sphinx search implementation returns results of varying quality/relevancy, and some results are duplicated. I'm 0/5 on whether we want to continue to develop and enhance the native Sphinx search, or consider a third-party search extension instead to achieve better results.

[TomerPacific]

@cwarnermm - So what, in your opinion, is the requirement from this issue?

[cwarnermm]

@TomerPacific - For this issue, I'm looking for someone who has significant experience with search engines to help evaluate our current deployed state of Sphinx search against other third-party search engines available that are Sphinx-friendly.

If there are ways to improve the current native search engine to return higher quality search results to users, I'm very much in favour of improving Sphinx search over introducing a different engine.

At an absolute minimum, I'm looking for help to learn how to:

• Control the order of search results based on relevancy (where weighted pages return higher in the list than unweighted pages)
• Return a matching page once instead of repeatedly based on section headers on the page
• Understand why some search criteria fails to return page results I know exist
• Learn how to return search results based on keywords not necessarily present on the docs page
• Split the search results into 2 tabs: one for configuration setting results and one for page-based results. Currently, configuration setting results are returned first, followed by page results, which isn't ideal.

[TomerPacific]

@cwarnermm - This is out of my scope of expertise, so I don't believe I can deliver the results you are after for this issue.