Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LaTeX] Improve LaTeX #3575

Merged
merged 49 commits into from
Mar 13, 2024
Merged

[LaTeX] Improve LaTeX #3575

merged 49 commits into from
Mar 13, 2024

Conversation

deathaxe
Copy link
Collaborator

This PR is introduced to track changes on develop/latex branch, which is intented to be used to collect and review various improvements implemented by @ngc92 without creating too much noise on master branch.

The idea is to base all work/refactoring on this branch and merge them into this one using PRs.

Once a milestone with reasonable changes is hit, we can merge it to master branch as "Rewrite LaTeX" or whatever name/category is suitable.

The starting point is #3546 but with the opportunity to use this repo as working area instead of my "private" one.

deathaxe and others added 16 commits November 12, 2022 20:46
This commit extends LaTeX's `begin-end-commands` context to support
Haskell's `\begin{code} ... \end{code}` blocks.

Formerly used `plain-tex` does no longer exist.
The same context with identical content is already defined in TeX.
Apply an top-down principle to have most basic contexts at the end of a
document and the organizational ones at the beginning - just like a
tree.

Readability may be improved by grouping contexts visually via delimiter
comments. See also: Java, PHP, ...
added a brace-pop utility context
Illustrate how `braces` are implemented in both TeX & LaTeX with same
content but different sub-scopes. Re-implementing a context without
meta_prepend/meta_append overrides it completely.
Re-arrange contexts to implement the top-down principle.
Group macro related contexts in a MACROS section
Some final sub scopes changed from .latex to tex due to inheritance.
Hence make Markdown syntax tests a bit more lazy.
@deathaxe deathaxe marked this pull request as draft November 19, 2022 11:27
ngc92 and others added 11 commits November 19, 2022 16:05
This commit adds scoping/highlighting for the built-in constant-like objects in TeX.

The decision which control sequences to include here is based more on expected
usage, than on how this is defined in the underlying language. I think for a 
macro-processing system like TeX, this is probably the best approach. 

Thus I've considered as constants in this sense:
 - The underlying parameters of the TeX typesetting algorithm. 
 - Registers with pre-defined meaning
 - simple macros (i.e. without parameters) that insert a fixed element into
   the typesetting
 
In particular, a parameterless macro that 
a) assigns to some internal value, or 
b) changes how subsequent input tokens will be interpreted are not considered 
   constants here.

The typesetting parameters and registers can be assigned-to, though for most of
these this would typically only happen once in the beginning of the document
(usually hidden from the user, as part of the TeX format that is used).
However, it is possible to change these values in the middle of the document
(though some **really** should not be changed), so in that sense these are not
all technically constants. 

For the assignment, I've opted to not try to distinguish constants that are
dimensions from constants that are pure numbers, so both get the same scoping.
In extremely weird corner cases, this might lead to erroneous scoping, e.g. if
you write
```
\hyphenpenalty=5pt
```
then the current scoping would not know that the penalty
is a number, so the `pt` are actually letters to typeset, and not part of the
unit. Such cases should be very rare, though.

Similarly, since macros can hide part of the syntax, there are situations 
where we have no chance of knowing which scoping is correct. For example, in
```
\abovedisplayskip\value plus 5pt
```
if `\value` evaluates to some measurement, then the `plus 5pt` would be part 
of the measurement, otherwise, it is normal text.

Co-authored-by: deathaxe <[email protected]>
This commit adds the \relax command to control sequence. 
As its usage is similar to python's `pass`, if used the same scoping, 
i.e. as a special  `keyword.control.flow`
1. converts anonymous into named contexts
2. make use of `{{letter}}` variable
3. bring contexts inline with xparese-newcommand
This commit reworks the handling of text formatting commands in LaTeX. 
In particular, in cases where there are commands within formatted text, 
e.g. `\textbf{\TeX in bold}`, we no longer format the command names. 

One challenge is that there are several font properties that can be changed
independently:

- Font family (roman, sans serif, typewriter), 
- font weight (bold, medium, (possibly) light), 
- and font shape (upright, italic, slanted, small-caps). 

Most of these do not have any corresponding `markup` scopes and highlighting
rules (at least with default color schemes). At this point, only a subset of
nesting of these properties is actually handled: Bold/italic combination, 
and nested \emph emphasize, as well as ensuring that all the commands that
do not change one of these properties still typeset their arguments according
to the outer command.

No provision has been made for any form of triple-nesting, 
i.e. `\textbf{\textit{\textup{a}}}` would not result in bold `a` right now. 
Not sure if supporting this would be worth it.

Finally, there is also an \underline command that could be combined with bold/italic. 
So far, I've not included this, because in LaTeX underline comes with some 
caveats, so that package commands like `\uline` or `\ul` are often recommended.
@deathaxe deathaxe marked this pull request as ready for review February 28, 2023 17:39
@deathaxe
Copy link
Collaborator Author

This PR includes countless improvements for TeX and LaTeX, which might be worth releasing anytime soon.

There are propably more things to do, but those could be done in separate PR or dev branch.

@jwortmann
Copy link
Contributor

Just a few comments from looking at the syntax tests and comparing with the previous highlighting after opening a few files:

  • [LaTeX] Math scopes #3608 is not included yet, which makes ^ and _ look quite strange for one being highlighted as keyword.operator and the other as punctuation.separator. I would suggest to include the other PR first, before merging this one.
  • Some plain TeX macros like \centering or \thinspace are now constant.language, what makes them appear quite different from LaTeX macros, e.g. \newline, in most color schemes, so that they stand out very much in a document. There is a big wall of reasoning text in the commit message 700ed59, but I wonder whether this scope change is really useful, and I also don't really get the classification as "constant-like objects" (i.e. the constant scope seems pretty arbitrary to me, why not e.g. keyword, or variable.language, ...). I wouldn't consider this as a blocking issue, but if you would ask me, I'd keep using the current support.function scope and I don't think that the differentiation between plain TeX and LaTeX macros is important to the user.
  • The content of \textsc{} is now markup.raw, which seems also a bit surprising and arbitrary to me, especially because this scope gets a different background color in some color schemes. There is no particular scope name in the guidelines for smallcaps, but it mentions markup.other for "other" markup. So maybe markup.other.smallcaps would be a better choice here.

@deathaxe
Copy link
Collaborator Author

About #3608

I hasitated to merge this in due to comments about changed string scopes probably causing issues with LaTeXTools.

In general I can moderate only and/or give some technical advices about syntax/context structure as I am not involved with LaTeX in any way. I know what it is, understand some very basics, but probably won't use or spend time learning its details anytime soon. It's just no priority at the moment.

I am however happy to discuss or merge in proposals for further improvements.

This PR adds support for the `DocStrip` format of TeX, used by many LaTeX packages to specify their installation. 
https://www.texlive.info/CTAN/macros/latex/base/docstrip.pdf

I've inherited from the base `TeX` syntax, and just prepended the new commands to the `main` context. This is not quite accurate, in the sense that many `plainTeX` constructs would simply be invalid in a docstrip file -- as there is no typesetting to be done, typesetting and math commands don't really make sense. But I think it is OK to highlight these commands, and probably better to handle things like this, because if we manually select the subset that is supported, and later on add something new in `TeX` that also works in `DocStrip`, we might forget to change `DocStrips` `main` context.

In general, `DocStrip` files are usually rather short and simple, and typically look like the examples given in https://texdoc.org/serve/dtxtut.pdf/0 (appendix A).

Finally, since the goal is not to produce typeset output, but instead to describe how files are generated, I've used a `source` main scope.
@jrappen
Copy link
Contributor

jrappen commented May 13, 2023

@ngc92 @deathaxe Any plans to have this reviewed and merged soon(ish)?

@ngc92
Copy link
Contributor

ngc92 commented May 17, 2023

I may have some time end of this/beginning of next month to look into this again.

michaelblyons
michaelblyons previously approved these changes Jul 21, 2023
michaelblyons
michaelblyons previously approved these changes Nov 1, 2023
Copy link
Collaborator

@michaelblyons michaelblyons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jwortmann 's comments are valid, but I'm in favor of "Merge and then iterate." Or someone can open a PR to this PR.

@deathaxe
Copy link
Collaborator Author

deathaxe commented Nov 2, 2023

I'd at least vote for updateing to sublime-syntax version 2 before.

@FichteFoll
Copy link
Collaborator

I'll test out the branch on some of my old LaTeX files later when I find the time, though I expect no need for further changes in order to get this merged. Holding off my approval until then.

@jrappen
Copy link
Contributor

jrappen commented Mar 11, 2024

@deathaxe if you have a minute, maybe you could update the text in the opening comment (commit message).

Copy link
Collaborator

@FichteFoll FichteFoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I obviously forgot about this PR, but I just loaded it to look at a couple of my LaTeX files and they all looked normal, so here goes.

@deathaxe deathaxe merged commit 07488b7 into master Mar 13, 2024
2 checks passed
@deathaxe deathaxe deleted the develop/latex branch March 13, 2024 07:39
deathaxe added a commit to SublimeText-Markdown/MarkdownEditing that referenced this pull request May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants