8341670: [Text,TextFlow] Public API for Text Layout Info #1596

andy-goryachev-oracle · 2024-10-08T16:07:54Z

Please refer to

https://github.com/andy-goryachev-oracle/Test/blob/main/doc/Text/LayoutInfo.md

The RichTextArea control (JDK-8301121), or any custom control that needs non-trivial navigation within complex or wrapped text needs a public API to get information about text layout.

This change fixes the missing functionality by adding a new public method to the Text and TextFlow classes.:

    /**
     * Obtains the snapshot of the current text layout information.
     * @return the layout information
     * @since 24
     */
    public final LayoutInfo getLayoutInfo()

The LayoutInfo provides a view into the text layout within Text/TextFlow nodes such as:

caret information
text lines: offsets and bounds
overall layout bounds
text selection geometry
strike-through geometry
underline geometry

This PR also adds the missing strikeThroughShape() method to complement the existing underlineShape() and rangeShape() for consistency sake:

    /**
     * Returns the shape for the strike-through in local coordinates.
     *
     * @param start the beginning character index for the range
     * @param end the end character index (non-inclusive) for the range
     * @return an array of {@code PathElement} which can be used to create a {@code Shape}
     * @since 24
     */
    public final PathElement[] strikeThroughShape(int start, int end)

Discussion Points

it is unclear whether CaretInfo.getPartAt() should return a simple Rectangle2D or a more complex object similar to TextLineInfo, to be able to add more information, such as the nature of text, text direction, etc.

WARNING

Presently, information obtained via certain Text/TextField methods is incorrect with non-zero padding and borders, see JDK-8341438.

This PR provides correct answers in the new API, leaving the behavior of the existing methods unchanged (there is a compatibility risk associated with trying to fix JDK-8341438 ).

Testing

The new APIs can be visually tested using the Monkey Tester on this branch:
https://github.com/andy-goryachev-oracle/MonkeyTest/tree/text.layout.api

in the Text and TextFlow pages:

Two very basic headful tests have been added.

Why an interface ? It is easier to evolve a class when you want to add more info.
Also it should be sealed unless you can show why it is important that
applications can create instances.

It is being exposed for the benefit of RT but the class doc itself doesn't say when you might want to get one and why.
It might also benefit from a succinct description of what "the text layout" means and encompasses.

Oh and how do you know when it is no longer valid ?

good points, thanks! please let me know if the update addressed your concerns.

a couple of notes:

converted to an abstract class. I feel weird referring to an internal implementation at the API level (public sealed abstract class LayoutInfo permits com.sun.javafx.text.PrismLayoutInfo, yikes!)

why we get it: we don't explain why we need HitInfo or caret shapes or range shapes, do we?

prrace · 2024-10-09T19:30:39Z

modules/javafx.graphics/src/main/java/javafx/scene/text/LayoutInfo.java

+/**
+ * Represents an immutable snapshot of certain aspects of the text layout
+ * in a {@code Text} or {@code TextFlow} node,
+ * such as break up of the text into lines and the their bounds.


"the their"

prrace · 2024-10-09T19:44:30Z

modules/javafx.graphics/src/main/java/javafx/scene/text/LayoutInfo.java

+ * such as break up of the text into lines and the their bounds.
+ * <p>
+ * The snapshot is valid until the layout changes due to any change that
+ * triggers that, such as resizing of the container or modification of properties.


"until the layout changes due to any change that triggers that"

it's not wrong, but writing change twice reads a bit off.
How about

The layout snapshot is no longer valid after actions such as resizing of the container, or modification of certain properties.
For example updating the text or the font would invalidate the layout snapshot, but a change of color would not.

I still don't see how applications can KNOW this has happened.
There's nothing on the (immutable) layout which says its invalidated.
There's no property to monitor.
So people will in effect never be able to do anything but use it and toss it and get a new one every time.
Perhaps you can make it 'cheap' to do this.
Since it is immutable, the Text or TextFlow can cache it.
If it is invalidated, the Text or TextFlow should know and can flush its cached version.
So the SAME immutable instance can be returned over and over again until that happens.

People can then also use "==" or equals() to check this and save themselves validation too.
I notice you don't have equals() so probably that is needed anyway to save people from manual comparison .. also an evolution issue if the class is updated.

A consequence of that is then the text about invalidation can be updated with advice on doing a simple equals() call if they need to know if it changed. And equals will start with "==" so will be really fast to return true ..

thank you!

I am struggling with how to explain that it should neither be cached, nor compared to another snapshot.

Maybe an example would help. Use case is an editable rich text component based on TextFlow. The user presses HOME and the caret needs to go to the beginning of the line. We get the LayoutInfo, determine which line the caret is currently at and determine the start offset of that line to move the caret to.

Another example: in Text, to determine whether the mouse is clicked inside of the text or beyond the last character (see JDK-8091012 ).

We are not expecting the application to monitor the text layout, so no notifications or observable properties or equals() are needed.

(I've added some words to that effect in Text/TextFlow.getLayoutInfo() )

Clients will usually obtain a LayoutInfo in order to respond to some user action or an update of the target node.
LayoutInfo should be considered an ephemeral snapshot, to be used and immediately discarded.
Clients should always obtain a new one when they need to respond again as likely the trigger to respond again means the target node has changed.

Although I don't see the harm in being more efficient.

I hear a suggestion to create a think wrapper/proxy instead of the snapshot, right?

This might, in fact, be a better idea. The API stays the same.
Let me think.

Thanks!

andy-goryachev-oracle · 2024-10-10T17:16:57Z

@Jugen do you think this PR addresses the issue https://bugs.openjdk.org/browse/JDK-8091012 ?

andy-goryachev-oracle · 2024-10-10T19:23:47Z

One additional thought/question:

There is a need to extract a few more data points from the text layout in addition to the text lines, specifically:

bounds of the (selection) range, as rectangles
strikethrough range shape (as PathElement[] and/or rectangles)
underline range shape (as PathElement[] and/or rectangles)

We currently have Text/TextFlow.underlineShape(int,int), .rangeShape(int,int) which return PathElement[] in an undocumented way.

There is no corresponding .strikeThroughShape() method, or a way to retrieve the geometric bounds as lines/rectangles.

Granted, one can attempt to analyze the PathElement[] and convert it to geometry, but it seems we should avoid asking the clients to do that.

So the question are:

should we add these data points to LayoutInfo
if so, in what form? i.e. as individual methods or as an array of Line/Rectangle objects,
should we add the .strikeThroughShape() to Text/TextFlow, or simply adding it here should be sufficient?

Jugen · 2024-10-11T13:57:02Z

@andy-goryachev-oracle First off thanks for all your work on the richtext control you're working on.

@Jugen do you think this PR addresses the issue https://bugs.openjdk.org/browse/JDK-8091012 ?

Unfortunately I don't think this helps for the hit problem Tomas was having, see hit(x,y) in RichTextFX's TextFlowExt for the relevant code.

However I was able to use all the methods in the LayoutInfo interface of this PR in RichTextFX to replace the custom layout info code, which is great.

Jugen · 2024-10-11T14:22:27Z

Wrt to rangeShape & underlineShape: RichTextFX quite successfully uses the PathElement[] returned to it's advantage and so wouldn't gain from Rectangles.

.strikeThroughShape() should be added to Text/TextFlow for consistency sake.

andy-goryachev-oracle · 2024-10-11T17:03:58Z

thank you so much @Jugen for checking and the feedback!

I see that a lot of what is being done in TextFlowExt should become unnecessary with the new API (or maybe even the whole thing might become unnecessary).

Also, while the new API do not address JDK-8091012 directly, that is, there is no field added to HitInfo, it looks like the information provided by the LayoutInfo should be sufficient to implement what you guys need as a utility, since you will have the geometry of text lines in the layout, right?

Anyway, thank you. I agree with you that we should add T/TF.strikeThroughShape() for consistency sake.

andy-goryachev-oracle · 2024-10-11T17:15:43Z

Thank you you all @prrace @Jugen @kevinrushforth for the feedback!
I am going to take this PR back to draft to make the suggested changes.

andy-goryachev-oracle · 2024-10-11T22:27:46Z

/issue add JDK-8341672

openjdk · 2024-10-11T22:28:11Z

@andy-goryachev-oracle
Adding additional issue to issue list: 8341672: [Text/TextFlow] getRangeInfo.

kevinrushforth

I took a quick initial pass of the API. It looks like what I would expect. I left a couple questions inline. Also, you might want to document whether the newly added methods can return null (maybe in the LayoutInfo class docs, if none of them can return null).

kevinrushforth · 2024-10-23T18:48:47Z

modules/javafx.graphics/src/main/java/javafx/scene/text/CaretInfo.java

+ *
+ * @since 24
+ */
+public sealed abstract class CaretInfo permits PrismCaretInfo {


Given the API you define, is there a reason not to make this an interface?

Originally it was an interface. Then @prrace said "Why an interface ? It is easier to evolve a class when you want to add more info."

Since this thing is sealed, it probably does not matter, and access to class methods is two nanoseconds faster (not that's important).

The only good thing about interface is the ease of mocking in testing, but we don't do that.

I had forgotten about that (and it doesn't matter, as you say). Carry on.

kevinrushforth · 2024-10-23T18:50:12Z

modules/javafx.graphics/src/main/java/javafx/scene/text/LayoutInfo.java

+ *
+ * @since 24
+ */
+public sealed abstract class LayoutInfo permits com.sun.javafx.text.PrismLayoutInfo {


Is there a reason to not make this an interface?

kevinrushforth · 2024-10-23T18:51:27Z

modules/javafx.graphics/src/main/java/javafx/scene/text/LayoutInfo.java

+     *
+     * @return the list of {@code TextLineInfo} objects
+     */
+    public abstract List<TextLineInfo> getTextLines();


Is the list immutable? If so (which I think it should be), can you specify that?

A new instance gets created for each caller, we don't care what they do to it. Maybe we should say that?

From the API purity perspective, it could be, but an immutable list would incur a copy overhead, do we really need that?

It also looks like there is no easy way of creating an immutable wrapper similar to JDK's List12, so we'd need to invent one.

(Note: the original design called for an array, but List was recommended because of stream/etc)

You can use Collections.unmodifiableList(List) to wrap the list without copying.

kevinrushforth · 2024-10-23T18:54:46Z

modules/javafx.graphics/src/main/java/javafx/scene/text/LayoutInfo.java

+    public abstract TextLineInfo getTextLine(int index);
+
+    /**
+     * Returns the geometry of the text selection, as an array of {@code Rectangle2D} objects,


"as an array ..." --> "as a list ...".

Is the list immutable?

kevinrushforth

Finished the first pass through the API, with a few more questions.

kevinrushforth · 2024-10-25T14:08:40Z

modules/javafx.graphics/src/main/java/javafx/scene/text/CaretInfo.java

+     * @param index the line index
+     * @return the array of [x, ymin, ymax] values
+     */
+    public abstract double[] getLineAt(int index);


Have you considered returning a record for this?

If you did that, it might be easier for apps to consume. It would also make it easy for you to add a convenient List<CaretInfo.Line> getLines() method.

well, we could have returned Line2D but we have no such public class!
We have javafx.geometry.Rectangle2D but no corresponding Line2D.

I could use Rectangle2D here, which might also take care of the vertical text support which will have been added in the year 252525.

kevinrushforth · 2024-10-25T14:12:09Z

modules/javafx.graphics/src/main/java/javafx/scene/text/CaretInfo.java

+    }
+
+    /**
+     * Returns the number of lines representing the caret.


Since this is in the text class, the term "lines" is ambiguous. You mean geometric lines, not text lines, right?

modules/javafx.graphics/src/main/java/javafx/scene/text/CaretInfo.java

kevinrushforth · 2024-10-25T14:14:08Z

modules/javafx.graphics/src/main/java/javafx/scene/text/CaretInfo.java

+     * The geometry is encoded in an array of [x, ymin, ymax] values which
+     * represent a line drawn from (x, ymin) to (x, ymax).


This seems overly restrictive, unless I don't understand what this does. How would this work for caret that is "caret" shaped -- ^ -- or shaped line an "I" with serifs? Presuming that specifying it with a series of geometric lines is sufficient, should it be defined as [xmin, xmax, ymin, ymax] instead?

We are talking about "geometry", assuming two things:

the caret "geometry" represents a boundary between two characters and therefore is a line (or two lines in the case of a split caret)

the text is horizontal

the latter poses a problem - on one hand, FX currently provides no support whatsoever for the vertical text, but that might change in the (distant) future. So perhaps Rectangle2D should work with the width of 0.

kevinrushforth · 2024-10-25T14:27:25Z

modules/javafx.graphics/src/main/java/javafx/scene/text/LayoutInfo.java

+     * <li>{@code minX} is always zero
+     * <li>{@code minY} is the ascent of the first line (negative)
+     * <li>{@code width} the width of the widest line
+     * <li>{@code height} the sum of all lines height


Is this correct ? What about spacing between lines, or is that included in the line height of each line?

There is an existing related issues
JDK-8317120 RFE: TextFlow.rangeShape() ignores lineSpacing
JDK-8317122 RFE: TextFlow.preferredHeight ignores lineSpacing

so perhaps we need to add a boolean argument to qualify whether the return value should include the last line's lineSpacing or not.

kevinrushforth · 2024-10-25T14:29:27Z

modules/javafx.graphics/src/main/java/javafx/scene/text/LayoutInfo.java

+    public abstract List<Rectangle2D> selectionShape(int start, int end);
+
+    /**
+     * Returns the geometry of the strike-through shape, as an array of {@code Rectangle2D} objects,


Same comment here (and below) as above: this is a list not an array; it should ideally be immutable.

bridgekeeper · 2024-11-22T21:43:20Z

@andy-goryachev-oracle This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

andy-goryachev-oracle · 2024-11-22T21:45:48Z

@prrace please take a look

draft api

285db28

openjdk bot added the csr Need approved CSR to integrate pull request label Oct 8, 2024

bounds

43ac0e1

andy-goryachev-oracle marked this pull request as ready for review October 8, 2024 18:16

openjdk bot added the rfr Ready for review label Oct 8, 2024

prrace reviewed Oct 9, 2024

View reviewed changes

andy-goryachev-oracle added 2 commits October 9, 2024 12:13

review comments

21dbe6c

javadoc

04c4abb

prrace reviewed Oct 9, 2024

View reviewed changes

andy-goryachev-oracle added 2 commits October 9, 2024 13:41

clarify

6869c76

convert to wrapper

2009a7e

andy-goryachev-oracle marked this pull request as draft October 11, 2024 17:15

openjdk bot removed the rfr Ready for review label Oct 11, 2024

andy-goryachev-oracle added 2 commits October 11, 2024 10:33

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

6b7b079

text line info

c2e26d5

andy-goryachev-oracle marked this pull request as ready for review October 11, 2024 22:29

openjdk bot added the rfr Ready for review label Oct 11, 2024

kevinrushforth reviewed Oct 23, 2024

View reviewed changes

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

eb99008

kevinrushforth reviewed Oct 25, 2024

View reviewed changes

andy-goryachev-oracle added 2 commits October 25, 2024 13:47

review comments

6776d97

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

9b1f99a

andy-goryachev-oracle marked this pull request as draft October 30, 2024 20:08

openjdk bot removed the rfr Ready for review label Oct 30, 2024

andy-goryachev-oracle added 16 commits October 31, 2024 07:48

include line spacing

4f4e659

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

dd34810

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

a716e57

caret and range

75b171f

for text

029bbe6

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

3925c02

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

e139484

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

8a5b904

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

a0e59fb

line spacing

3c2976b

shorter array

5076f36

text flow test

4479f0b

text layout test

c05c790

note

7c86978

coordinates

366ae4b

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

d31eb2b

andy-goryachev-oracle marked this pull request as ready for review November 19, 2024 21:51

openjdk bot added the rfr Ready for review label Nov 19, 2024

andy-goryachev-oracle mentioned this pull request Nov 20, 2024

Q: LayoutInfo API? gluonhq/rich-text-area#381

Open

andy-goryachev-oracle added 2 commits November 22, 2024 07:56

Merge remote-tracking branch 'origin/master' into ag.text.layout.api

7ea4e3b

segments

944bd0d

		* The geometry is encoded in an array of [x, ymin, ymax] values which
		* represent a line drawn from (x, ymin) to (x, ymax).

8341670: [Text,TextFlow] Public API for Text Layout Info #1596

Are you sure you want to change the base?

8341670: [Text,TextFlow] Public API for Text Layout Info #1596

Conversation

andy-goryachev-oracle commented Oct 8, 2024 • edited by openjdk bot Loading

Discussion Points

WARNING

Testing

See Also

Progress

Issues

Reviewing

bridgekeeper bot commented Oct 8, 2024

openjdk bot commented Oct 8, 2024

openjdk bot commented Oct 8, 2024

openjdk bot commented Oct 8, 2024

mlbridge bot commented Oct 8, 2024 • edited Loading

Webrevs

prrace Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prrace Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andy-goryachev-oracle commented Oct 10, 2024

andy-goryachev-oracle commented Oct 10, 2024

Jugen commented Oct 11, 2024

Jugen commented Oct 11, 2024

andy-goryachev-oracle commented Oct 11, 2024

andy-goryachev-oracle commented Oct 11, 2024

andy-goryachev-oracle commented Oct 11, 2024

openjdk bot commented Oct 11, 2024

kevinrushforth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kevinrushforth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bridgekeeper bot commented Nov 22, 2024

andy-goryachev-oracle commented Nov 22, 2024

andy-goryachev-oracle commented Oct 8, 2024 •

edited by openjdk bot

Loading

mlbridge bot commented Oct 8, 2024 •

edited

Loading

prrace Oct 9, 2024 •

edited

Loading

prrace Oct 9, 2024 •

edited

Loading