Redesign Mozc to not rely on synchronous edit sessions in Windows

[yukawa]

Description

Spun off from

• Mozc for Windows stops responding to key events on Microsoft Word  #819

There are several places where Mozc requests synchronous edit sessions with explicitly setting TF_ES_SYNC. However, as discussed in #819, it'd be better to redesign Mozc to not rely on synchronous edit sessions for application compatibility reasons (perhaps solely because of Microsoft Word though).

Environment

• OS: Windows 11 22H2
• Related Applications: Microsoft Word for Microsoft 365 MSO (Version 2309 Build 16.0.16827.20166) 64 bit

References

Rules of Text Services - TSF Aware

https://learn.microsoft.com/en-us/archive/blogs/tsfaware/rules-of-text-services

The second rule of writing a text service is: Avoid Synchronous edit sessions. There are a number of text stores (namely, the 500 pound gorilla of text stores - Microsoft Word) that never grant synchronous edit sessions. (The only exception is when Microsoft Word asks for reconversions.) If you start your design with this assumption, you will avoid a painful redesign when you start testing with Microsoft Word, and you find that nothing works.

The third rule of writing a text service is: Text service code is hard to maintain. Because document modifications have to be done in an edit session, which may not run synchronously, you can't write code that expects to, for example, grab some text, inspect it, and do something from (for example) an event handler, or a keystroke handler. Instead, your handlers consist of a lot of edit sessions, and the edit sessions usually end by firing a callback (or expecting some other event handler to fire as a result of the change in the edit session).

The end result is a big pile of spaghetti code. The only way to avoid this would be to write your text service in a language other than C, which has the following feature set:

• supports closures and anonymous functions (to simplify the inherently asynchronous nature of text services)
• doesn't have a large runtime (because text services get loaded into a lot of processes),
• compiles to efficient native code (because text services run inside the event loop, and have to be fast)
• supports COM efficiently (because TSF is COM-based)

As far as I know, no language has that feature set. The last three features are much more important than the first one; if your text service slows down the system by 50%, or increases the working set of the typical application by 30%, people aren't going to leave your text service running for very long.

Luckily, C++ has become much more powerful since the original article was written 17 years ago and the first feature is already available.

As a side note, Microsoft has been working on making Text Services Framework out-of-process, which is expected to mitigate these issues a lot when it becomes available to 3rd party IME developers.

Support Dictation With Text Services Framework - MSDN Magazine 2007 July

https://learn.microsoft.com/en-us/archive/msdn-magazine/2007/july/speak-up-support-dictation-with-text-services-framework

There are only a few instances where your application must grant a synchronous lock request. Your application must grant a synchronous lock if your application (or control) asks a text service to do something such as correcting text (via ITfFunctionProvider, the use of which is outside the scope of this article). Your application must also grant a synchronous lock whenever you call any method on the ITfKeystrokeMgr interface. Your application must also grant a synchronous lock when you do not use ITfKeystrokeMgr, and the lock request comes from within GetMessage or PeekMessage.

Aside from these exceptions, your application is not required to grant a synchronous lock request; it is perfectly allowable to only grant asynchronous lock requests. It is also perfectly allowable to treat asynchronous lock requests as synchronous lock requests.

From our observation in #819 and the previous quote in MSDN Magazine, it looks to be safer to assume the following rule can never be guaranteed in Microsoft Word.

Your application must also grant a synchronous lock whenever you call any method on the ITfKeystrokeMgr interface. Your application must also grant a synchronous lock when you do not use ITfKeystrokeMgr, and the lock request comes from within GetMessage or PeekMessage

ITextStoreACP::RequestLock method (textstor.h)

https://learn.microsoft.com/en-us/windows/win32/api/textstor/nf-textstor-itextstoreacp-requestlock

This method uses the ITextStoreACPSink::OnLockGranted method to lock the document. Applications must never modify the document or send change notifications using the ITextStoreACPSink::OnTextChange method from within the ITextStoreACP::RequestLock method. If the application has pending changes to report, the application can only respond to the asynchronous lock request.

Applications should not attempt to queue multiple ITextStoreACP::RequestLock method calls, because the application requires only a single callback. If the caller makes several read requests and one or more write requests, however, the callback should be for write access.

Successful requests for synchronous locks supersede requests for asynchronous locks. Unsuccessful requests for synchronous locks do not supersede requests for asynchronous locks. The implementation must still serve the outstanding asynchronous request, if one exists.

If the lock is granted before the ITextStoreACP::RequestLock method returns, the phrSession parameter will receive the HRESULT returned by the ITextStoreACPSink::OnLockGranted method. If the call is successful, but the lock will be granted later, the phrSession parameter receives the TS_S_ASYNC flag. The phrSession parameter should be ignored if ITextStoreACP::RequestLock returns anything other than S_OK.

A caller should never call this method reentrantly, except in the case that the caller holds a read-only lock. In this case the method can be called reentrantly to ask for an asynchronous write lock. The write lock will be granted later, after the read-only lock ends.

For more information about document locks, see Document Locks.