Research ways to improve SEO

[Mariatta]

How to improve docs.python.org's SEO ranking?

Come up with action items on how the SEO can be improved.

[Mariatta]

I did a quick reading on a few blogs about SEO.
Here are some things I learned:

• Analyze with Google search console https://search.google.com/search-console
• Add meta tags https://ahrefs.com/blog/seo-meta-tags/
• Fix broken links
• Optimize site for mobile devices
• Encourage sharing on social media
• Write click-worthy titles

[hugovk]

Google Chrome has something in devtools called Lighthouse that runs an audit on a webpage:

• Right click page > Inspect > Lighthouse > can uncheck the non-SEO categories for speed > Analyze page load

There's also an online version: https://web.dev/measure/

---

Results for an arbitrary page (https://docs.python.org/3/library/csv.html) are pretty good to begin with:

• https://web.dev/measure/?url=https%3A%2F%2Fdocs.python.org%2F3%2Flibrary%2Fcsv.html

Links to those recommendations:

• https://web.dev/meta-description/
• https://developers.google.com/search/docs/crawling-indexing/links-crawlable
• https://developers.google.com/search/mobile-sites/get-started

The first one should be pretty easy, and we should the second. The last is bigger theme work, and we may get a good enough bump from the first one or two.

[hugovk]

For the first one, https://pypi.org/project/sphinxext-opengraph/ is a Sphinx extension to automatically add OpenGraph metadata.

It doesn't add <meta name="description"> (although there's a feature request: wpilibsuite/sphinxext-opengraph#65), but it would be good to add this anyway and could also help with SEO. (And it improves the previews when sharing on Facebook.)

However, we will need to know the version number of the actual docs build (e.g. (it could be /3/, /3.9/ or /3.9.2/. See python/docsbuild-scripts#137

[ezio-melotti]

During the last call someone mentioned that "searching for specific functions doesn't bring docs.p.o up as first result".

In the chat I suggested to add the entries that recently got added to the docs ToC in a <meta name="keyword">, but I'm not sure whether that would belong to built-in Sphinx or if it would be better as an extension (@AA-Turner suggested to create a Sphinx issue, so I could start from there).

However It's also been a few years since I last looked into SEO-related stuff, and I have the impression that nowadays the meta keywords are not as important anymore (I might be mistaken, or there might be something equivalent/better).

Some concern was also brought up about search engines indexing version of the docs that shouldn't be indexed, e.g. docs PR previews or outdated versions. This could be solved by adding an explicit <meta robots="noindex; nofollow">, especially to the former group -- Google got much better with the search result not including outdated versions, and there might be old deprecated/removed features that should still be indexed.

[hugovk]

Good suggestions.

However It's also been a few years since I last looked into SEO-related stuff, and I have the impression that nowadays the meta keywords are not as important anymore (I might be mistaken, or there might be something equivalent/better).

Maybe, although I think a good rule of thumb for good Google results is to follow the suggestions by Google's SEO tool :)

[hugovk]

For the first one, pypi.org/project/sphinxext-opengraph is a Sphinx extension to automatically add OpenGraph metadata.

It doesn't add <meta name="description"> (although there's a feature request: wpilibsuite/sphinxext-opengraph#65), but it would be good to add this anyway and could also help with SEO. (And it improves the previews when sharing on Facebook.)

I contributed some improvements to this extension, including adding <meta name="description">.

We've added it to the devguide, and the SEO score has increased from 91 to 100. It also improves the previews on Facebook, Slack, Discord, etc.

• Use sphinxext-opengraph to generate OpenGraph metadata devguide#953 (comment)
• Use sphinxext-opengraph to generate OpenGraph metadata devguide#953 (comment)

We can look into adding it to the main docs next.

[hugovk]

Next up! PR to add the same thing to https://docs.python.org:

python/cpython#99931

[hugovk]

This is now on /3/ and /3.12/! 🚀

For example:

• https://docs.python.org/3/library/csv.html (3.11 branch)
• https://docs.python.org/3.12/library/csv.html (main branch)

(Note: It's not on the index.html or download.html pages, because they're html_additional_pages and not under control of the Sphinx extension.)

Google's Pagespeed Insights, SEO score

Increased from 81 to 88 on mobile, and 83 to 92 on desktop. As expected the meta description now passes. ✅

Details

Old /3.10/	New /3/
SEO mobile: 81, desktop: 83	SEO mobile: 88, desktop: 92

---

Facebook

Here's how the new page looks on Facebook's sharing debugger:

Details

Interestingly, because old pages like https://docs.python.org/3.10/library/csv.html declare their canonical URL to be the /3/ version, we also get that metadata for free!

<link rel="canonical" href="https://docs.python.org/3/library/csv.html" />

So here's 3.10:

Details

We have to go back to 2.7 to see how it was before:

Details

Discord

/3/ is expanded:

Slack

Also /3/ is expanded: