Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LaTeX] Math scopes #3608

Merged
merged 3 commits into from
Nov 1, 2023
Merged

[LaTeX] Math scopes #3608

merged 3 commits into from
Nov 1, 2023

Conversation

ngc92
Copy link
Contributor

@ngc92 ngc92 commented Dec 6, 2022

This PR adapts the scoping for maths in TeX/LaTeX

  • use same scopes as in markdown
  • = sign in maths is rarely assingment
  • indice is not an English word. Changed to subscript Q: should ^ be scoped superscript for consistency, or do we keep its mathematical meaning of exponentiation?
  • scope braces \{\} more consistent with other (math-)grouping mechanisms () and []

Q: How should we handle scoping regarding the surrounding construct and the actual contents, i.e.
$x$, here only x is actually maths. This is currently somewhat inconsistent between the long begin and end forms, where these commands are not part of the meta environment, but for short forms the begin/end delimiters are taken into account.

@ngc92
Copy link
Contributor Author

ngc92 commented Dec 7, 2022

One more question, should we consider math commands that produce such as \alpha to be (named) escape sequences and scope them like that, at least with an additional scope?

@jwortmann
Copy link
Contributor

I guess removing the string.other.math scope for the math delimiters would "break" popular LaTeX plugins and probably a lot of color schemes too.

https://github.com/SublimeText/LaTeXTools/search?q=%22string.other.math%22
https://github.com/LaTeXing/LaTeXing/search?q=%22string.other.math%22

The string scope doesn't make sense semantically, but it might be something that is difficult / infeasable to change now, due to historic reasons and backwards compatibility.

Q: should ^ be scoped superscript for consistency, or do we keep its mathematical meaning of exponentiation?

Yes, imo it should use a similar scope like _. It is keyword.operator.math on the master branch, and seems to have been adjusted to keyword.operator.arithmetic in one of the recent PRs, but this is wrong because it doesn't necessarily mean exponentation, for example consider $A^T$. Okay, "operator" would still fit in this case, but there are a lot of other notations in math or physics where an index is used in form of a superscript.

  • scope braces \{\} more consistent with other (math-)grouping mechanisms () and []

\{ is escaped and I think therefore it should be highlighted with the constant.character.escape color by default, while ( or [ are not escaped. I'm also not sure whether the punctuation.section scopes which can be found in this diff (not on master branch) can really be applied to the parentheses or braces like they are in other programming languages, because in LaTeX they are just printed characters and opening & closing parentheses don't necessarily need to match. But the constant.character.{parenthesis|bracket} on master is also a bit strange from the highlighting perspective (not sure if this was already discussed in one of the recent PRs).

One more question, should we consider math commands that produce such as \alpha to be (named) escape sequences and scope them like that, at least with an additional scope?

I would say no, assuming you refer to the constant.character.escape scope. Otherwise, following that logic, you would need to do the same for all other macros which print as a single character. For example \partial, \nabla, \infty etc.

@deathaxe
Copy link
Collaborator

deathaxe commented Dec 7, 2022

I'm also not sure whether the punctuation.section ... can really be applied

If they are to be treated as normally printed text, they shouldn't receive any scope, neither string, character nor punctuation. punctuation.section was choosen to provide a highlighting experience which is more inline with other syntaxes.

The argument about being printed would also result in removing any kind of scopes for basic math operators. Not sure if plain text is what we want in the end?

@ngc92
Copy link
Contributor Author

ngc92 commented Dec 7, 2022

@jwortmann
Thanks for the detailed comments.

I think you're right that removing string.other.math is too optimistic. It would be nice to have a mechanism so that these could be fixed in the syntax file, i.e. have an additional scope-remapping that would add the backward-compatible but semantically questionable scopes based on the actual, sensible scopes. That would also be very helpful in fixing the inconsistencies in noted in #3607.

I think highlighting parenthesis, and especially matching parenthesis, is a valuable help when typing math formulas. There are certain cases where things can go wrong, e.g. if you denote open intervals by ]a, b[ the highlighter will be very confused. Also, \{\} currently receives two scopes, one for the escape and one for the grouping mechanism. I think it makes sense to switch the order of the two scopes, though.

@ngc92 ngc92 force-pushed the scopes branch 3 times, most recently from 7699ec3 to 8baa6df Compare December 8, 2022 21:28
@jwortmann jwortmann mentioned this pull request Mar 9, 2023
@ngc92
Copy link
Contributor Author

ngc92 commented Jun 4, 2023

I rebased this on the other recent changes, #3787, and also reverted the change to the string scope name

@deathaxe deathaxe merged commit 66dccdf into sublimehq:develop/latex Nov 1, 2023
deathaxe added a commit that referenced this pull request Mar 13, 2024
* [LaTeX] Inherit LaTeX from TeX

* [Haskell] Fix Lterate Haskell

This commit extends LaTeX's `begin-end-commands` context to support
Haskell's `\begin{code} ... \end{code}` blocks.

Formerly used `plain-tex` does no longer exist.

* [LaTeX] Remove macro-braces

The same context with identical content is already defined in TeX.

* [TeX] Move most basic prototype like contexts to the end

Apply an top-down principle to have most basic contexts at the end of a
document and the organizational ones at the beginning - just like a
tree.

Readability may be improved by grouping contexts visually via delimiter
comments. See also: Java, PHP, ...

* added a brace-pop utility context

* renamed

* [LaTeX] Refactor braces contexts

Illustrate how `braces` are implemented in both TeX & LaTeX with same
content but different sub-scopes. Re-implementing a context without
meta_prepend/meta_append overrides it completely.

* [TeX] Add more named groups

* [LaTeX] Organize math expressions in named contexts

* [LaTeX] Re-arrange math contexts

Re-arrange contexts to implement the top-down principle.

* [TeX] Reorganize macro braces

Group macro related contexts in a MACROS section

* [Markdown] Adjust syntax tests

Some final sub scopes changed from .latex to tex due to inheritance.
Hence make Markdown syntax tests a bit more lazy.

* [LaTeX] Rename remaining `inside-...` context

* [TeX] Improved highlighting in macro definition bodies (#3576)


Co-authored-by: deathaxe <[email protected]>

* [TeX] improved handling of (cat)code family of commands (#3577)

Co-authored-by: deathaxe <[email protected]>
Co-authored-by: Michael <[email protected]>

* [TeX] Fix scoping error (#3582)

* [TeX] added scoping of "constants" (#3581)

This commit adds scoping/highlighting for the built-in constant-like objects in TeX.

The decision which control sequences to include here is based more on expected
usage, than on how this is defined in the underlying language. I think for a 
macro-processing system like TeX, this is probably the best approach. 

Thus I've considered as constants in this sense:
 - The underlying parameters of the TeX typesetting algorithm. 
 - Registers with pre-defined meaning
 - simple macros (i.e. without parameters) that insert a fixed element into
   the typesetting
 
In particular, a parameterless macro that 
a) assigns to some internal value, or 
b) changes how subsequent input tokens will be interpreted are not considered 
   constants here.

The typesetting parameters and registers can be assigned-to, though for most of
these this would typically only happen once in the beginning of the document
(usually hidden from the user, as part of the TeX format that is used).
However, it is possible to change these values in the middle of the document
(though some **really** should not be changed), so in that sense these are not
all technically constants. 

For the assignment, I've opted to not try to distinguish constants that are
dimensions from constants that are pure numbers, so both get the same scoping.
In extremely weird corner cases, this might lead to erroneous scoping, e.g. if
you write
```
\hyphenpenalty=5pt
```
then the current scoping would not know that the penalty
is a number, so the `pt` are actually letters to typeset, and not part of the
unit. Such cases should be very rare, though.

Similarly, since macros can hide part of the syntax, there are situations 
where we have no chance of knowing which scoping is correct. For example, in
```
\abovedisplayskip\value plus 5pt
```
if `\value` evaluates to some measurement, then the `plus 5pt` would be part 
of the measurement, otherwise, it is normal text.

Co-authored-by: deathaxe <[email protected]>

* [Tex] add \relax command (#3586)

This commit adds the \relax command to control sequence. 
As its usage is similar to python's `pass`, if used the same scoping, 
i.e. as a special  `keyword.control.flow`

* [Tex] Fix \relax test case

* [TeX] refactor LaTeX Log syntax (#3557)

* [TeX] Small formal tweaks for LaTeX Log syntax

* [LaTeX] xparse command definition support (#3585)

This commit adds support for xparse package.

see: https://mirror.informatik.hs-fulda.de/tex-archive/macros/latex/contrib/l3packages/xparse.pdf

Co-authored-by: deathaxe <[email protected]>

* [LaTeX] Adjust latex-newcommand context structure

1. converts anonymous into named contexts
2. make use of `{{letter}}` variable
3. bring contexts inline with xparese-newcommand

* [LaTeX] Fixups for command names (#3594)

* [TeX] Modifiers for \def's (#3593)

* [TeX] improve register assignments (#3587)

This commit ...

1. uses `tex-dimension-value` to implement register assignments
2. reorganizes some contexts logically and groups them into sections

Co-authored-by: deathaxe <[email protected]>

* [TeX] Reorganize comments

* [TeX] Reorganize braces

This commit groups all code block like contexts, which are used to
nest `main` context in a `BLOCKS` sections.

* [TeX] Reorganize command and control contexts

* [TeX] Reorganize main keys

* [TeX] Move test files into test/ sub directory

* [LaTeX] Scope all braced groups .tex (#3595)

* [TeX] Apply scope naming guidelines to math operators

* [TeX] Apply scope naming guidelines to math numbers

* [TeX] rename math variables context

* [TeX] Apply scope naming guidelines to math brackets

* [LaTeX] Enhance math syntax tests

* [LaTeX] Fix register id scope

* fixed a scoping bug (#3609)

* [TeX] Improved integer handling (#3604)

This commit does a first step at generalizing and unifying the handling of integers.
It provides a general context that matches decimal, octal, hexadecimal, and 
"based on character" integers. I've not yet used this for the definition of dimensions, 
because here we currently enter different scopes based on whether we've matched
an integer or a float. The syntax specified here assumes that everything will be
written on one line. It also does not handle the case of multiple leading signs.

* [LaTeX] Text formatting (#3618)

This commit reworks the handling of text formatting commands in LaTeX. 
In particular, in cases where there are commands within formatted text, 
e.g. `\textbf{\TeX in bold}`, we no longer format the command names. 

One challenge is that there are several font properties that can be changed
independently:

- Font family (roman, sans serif, typewriter), 
- font weight (bold, medium, (possibly) light), 
- and font shape (upright, italic, slanted, small-caps). 

Most of these do not have any corresponding `markup` scopes and highlighting
rules (at least with default color schemes). At this point, only a subset of
nesting of these properties is actually handled: Bold/italic combination, 
and nested \emph emphasize, as well as ensuring that all the commands that
do not change one of these properties still typeset their arguments according
to the outer command.

No provision has been made for any form of triple-nesting, 
i.e. `\textbf{\textit{\textup{a}}}` would not result in bold `a` right now. 
Not sure if supporting this would be worth it.

Finally, there is also an \underline command that could be combined with bold/italic. 
So far, I've not included this, because in LaTeX underline comes with some 
caveats, so that package commands like `\uline` or `\ul` are often recommended.

* [TeX.DocStrip] Basic DocStrip support (#3555)

This PR adds support for the `DocStrip` format of TeX, used by many LaTeX packages to specify their installation. 
https://www.texlive.info/CTAN/macros/latex/base/docstrip.pdf

I've inherited from the base `TeX` syntax, and just prepended the new commands to the `main` context. This is not quite accurate, in the sense that many `plainTeX` constructs would simply be invalid in a docstrip file -- as there is no typesetting to be done, typesetting and math commands don't really make sense. But I think it is OK to highlight these commands, and probably better to handle things like this, because if we manually select the subset that is supported, and later on add something new in `TeX` that also works in `DocStrip`, we might forget to change `DocStrips` `main` context.

In general, `DocStrip` files are usually rather short and simple, and typically look like the examples given in https://texdoc.org/serve/dtxtut.pdf/0 (appendix A).

Finally, since the goal is not to produce typeset output, but instead to describe how files are generated, I've used a `source` main scope.

* Revert to support.function scope (#3787)

* [LaTeX] Math scopes (#3608)

* [LaTeX] sublime syntax version 2

* [LaTeX] Replace some ugly anonymous multi-push contexts

---------

Co-authored-by: ngc92 <[email protected]>
Co-authored-by: Michael <[email protected]>
Co-authored-by: Johannes Rappen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants