fix unsaved buffers pyright incompatibility

[rchl]

The sublime part in buffer://sublime/[view_id] URIs that we were using for unsaved buffers triggered an infinite loop in pyrights file traversing code. While I don't pretend to understand its code fully, the sublime in our made-up URIs is not really necessary or makes a lot of sense even.

Fixes sublimelsp/LSP-pyright#281

[predragnikolic]

Tested ✔️

Can confirm that this fixes pyright.

[jfcherng]

Not really working as expected.

Right after I type

import os

os.

the server goes to 100% CPU again and won't repsonse anymore.

[rwols]

The “sublime” path was meant to signify to langservers that it was an in-memory buffer from sublime text. But I don’t think any maintainer cares.

[rchl]

Not really working as expected.

Right after I type

import os

os.

the server goes to 100% CPU again and won't repsonse anymore.

Make sure that you test on a newly opened buffer because the one you had open before has the previous uri saved already.

[rchl]

The “sublime” path was meant to signify to langservers that it was an in-memory buffer from sublime text. But I don’t think any maintainer cares.

Nor should they have a sublime specific logic, probably...

The protocol itself ("buffer:" in this case) is already enough imo. As long as it's a protocol that it doesn't know how to handle, it should be treated more or less like a non-file and not assume that it's accessible on disk.

[jfcherng]

Make sure that you test on a newly opened buffer because the one you had open before has the previous uri saved already.

That's what I did.

>>> view.settings().get('lsp_uri')
'buffer://37'

Weirdly if I kill the server by Windows Task Manager, a new server process is auto created and then it works in the unsaved buffer.

[rchl]

I will have a look if you find a way to reproduce.

[rwols]

It's working here for in-memory buffers for Ubuntu 22. Maybe it's a Windows-only issue? Maybe pyright has special case handling for Windows to account for drive letters? Just thinking out loud here.

[predragnikolic]

Not really working as expected.
Right after I type import os...

I managed to reproduce your issue on both Linux and Windows.

1. Press ctrl+shift+n to open a new sublime window, (this is important to reproduce the issue)
2. Create a buffer
3. Set syntax to Python
4. Wait for pyright to be initialized
5. Set the following content:

import os

// then type anything and you will see that things do not work

It is enough to just import os
document highlight will not work in that example, nor will completion.

Screencast.from.2023-11-10.23-14-38.webm

[jwortmann]

My observations (on Windows):

• buffer://sublime/123 - Server becomes unresponsive immediately
• buffer://123 - Server becomes unresponsive after typing import
• buffer:/123 - Server becomes unresponsive immediately
• buffer:123 - Server is not even started by LSP (probably because scheme is empty here according to urllib.parse.urlparse, so it doesn't match the enabled schemes from the client config)

Python 3.3>>> from urllib.parse import urlparse
Python 3.3>>> urlparse('buffer://sublime/123')
ParseResult(scheme='buffer', netloc='sublime', path='/123', params='', query='', fragment='')
Python 3.3>>> urlparse('buffer://123')
ParseResult(scheme='buffer', netloc='123', path='', params='', query='', fragment='')
Python 3.3>>> urlparse('buffer:/123')
ParseResult(scheme='buffer', netloc='', path='/123', params='', query='', fragment='')
Python 3.3>>> urlparse('buffer:123')
ParseResult(scheme='', netloc='', path='buffer:123', params='', query='', fragment='')

[rchl]

Python 3.3>>> urlparse('buffer:123')
ParseResult(scheme='', netloc='', path='buffer:123', params='', query='', fragment='')

Yeah, I don't get why it doesn't parse it correctly in this case. Probably there is a good reason though...

[jwortmann]

Yeah, I don't get why it doesn't parse it correctly in this case. Probably there is a good reason though...

Looks like it's just a bug in the urllib library. It works as expected in recent Python version(s):

Python 3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlparse
>>> urlparse('buffer:123')
ParseResult(scheme='buffer', netloc='', path='123', params='', query='', fragment='')

[rchl]

I have been thinking for a while to just port https://github.com/microsoft/vscode-uri to python. The main benefit being that it would also handle path normalization (the slashes and the Windows drive letter case which have been problematic for us before).

Doesn't seem like a big lift.

[jwortmann]

I tried to fix the urllib bug for the parse_uri function (and in some other places), and then the server gets started by LSP again, but it still becomes unresponsive for buffer:123 (and also for untitled:123) when typing import. So I think this is simply nothing we can fix on our side and it probably requires a fix in the server instead.

--- a/plugin/core/url.py
+++ b/plugin/core/url.py
@@ -60,6 +60,9 @@ def parse_uri(uri: str) -> Tuple[str, str]:
             else:
                 return parsed.scheme, path
         return parsed.scheme, path
+    elif parsed.scheme == '' and ':' in parsed.path:
+        # workaround for bug in urllib.parse.urlparse
+        return parsed.path.split(':')[0], uri
     return parsed.scheme, uri

[jwortmann]

I have been thinking for a while to just port https://github.com/microsoft/vscode-uri to python.

Note that VS Code seems to normalize drive letters to lowercase, but we need to use uppercase drive letters in the ST API functions for them to work correctly. So I don't think we could simply use a ported version of what VS Code does. See

LSP/plugin/core/url.py

Lines 83 to 86 in d5be23a

	def _uppercase_driveletter(match: Any) -> str:
	"""
	For compatibility with Sublime's VCS status in the status bar.
	"""

[rchl]

Yeah, I don't get why it doesn't parse it correctly in this case. Probably there is a good reason though...

Looks like it's just a bug in the urllib library. It works as expected in recent Python version(s):

Python 3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlparse
>>> urlparse('buffer:123')
ParseResult(scheme='buffer', netloc='', path='123', params='', query='', fragment='')

Hmm...

Python 3.3>>> urlparse('buffer:a123')
ParseResult(scheme='buffer', netloc='', path='a123', params='', query='', fragment='')

So it works if the path starts with a letter...
Might be better to change the format to buffer:view-<id> or something like that rather than adding a workaround for urllib.