Skip to content

Commit

Permalink
Merge branch 'master' into download-progress
Browse files Browse the repository at this point in the history
  • Loading branch information
StefanVukovic99 committed Jul 17, 2024
2 parents fdfd901 + f2632ef commit 689fa65
Show file tree
Hide file tree
Showing 25 changed files with 643 additions and 498 deletions.
90 changes: 85 additions & 5 deletions docs/development/language-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,18 +139,26 @@ Transforms files should export a `LanguageTransformDescriptor`, which is then im

```js
// from language-transformer.d.ts
export type LanguageTransformDescriptor = {
export type LanguageTransformDescriptor<TCondition extends string = string> = {
language: string;
conditions: ConditionMapObject;
transforms: {
[name: string]: Transform;
};
conditions: ConditionMapObject<TCondition>;
transforms: TransformMapObject<TCondition>;
};

export type ConditionMapObject<TCondition extends string> = {
[type in TCondition]: Condition;
};

export type TransformMapObject<TCondition> = {
[name: string]: Transform<TCondition>;
};

```

- `language` is the ISO code of the language
- `conditions` are an object containing parts of speech and grammatical forms that are used to check which deinflections make sense. They are referenced by the deinflection rules.
- `transforms` are the actual deinflection rules
- `TCondition` is an optional generic parameter that can be passed to `LanguageTransformDescriptor`. You can learn more about it at the end of this section.

Let's try and write a bit of deinflection for English, from scratch.

Expand Down Expand Up @@ -320,6 +328,78 @@ Here, by setting `valid` to `false`, we are telling the test function to fail th

You can also optionally pass a `preprocess` helper function to `testLanguageTransformer`. Refer to the language transforms test files for its specific use case.

#### Opting in autocompletion

If you want additional type-checking and autocompletion when writing your deinflection rules, you can add them with just a few extra lines of code. Due to the limitations of TypeScript and JSDoc annotations, we will have to perform some type magic in our transformations file, but you don't need to understand what they mean in detail.

Your `english-transforms.js` file should look like this:

```js
// english-transforms.js
import { suffixInflection } from "../language-transforms.js";

/** @type {import('language-transformer').LanguageTransformDescriptor} */
export const englishTransforms = {
language: "en",
conditions: {
n: {
name: "Noun",
isDictionaryForm: true,
subConditions: ["np", "ns"],
},
np: {
name: "Noun plural",
isDictionaryForm: true,
},
ns: {
name: "Noun singular",
isDictionaryForm: true,
},
},
transforms: {
// omitted
},
};
```

To gain type-safety, we have to pass an additional `TCondition` type parameter to `LanguageTransformDescriptor`. (You can revisit its definition [at the top of this section](#deinflection-rules-aka-language-transforms))

The passed type value should be the union type of all conditions in our transforms. To find this value, we first need to move the `conditions` object outside of `englishTransforms` and extract its type by adding a `/** @typedef {keyof typeof conditions} Condition */` comment at the start of the file. Then, you just need to pass it to the `LanguageTransformDescriptor` type declaration like so:

```js
// english-transforms.js
import { suffixInflection } from "../language-transforms.js";

/** @typedef {keyof typeof conditions} Condition */

const conditions = {
n: {
name: "Noun",
isDictionaryForm: true,
subConditions: ["np", "ns"],
},
np: {
name: "Noun plural",
isDictionaryForm: true,
},
ns: {
name: "Noun singular",
isDictionaryForm: true,
},
};

/** @type {import('language-transformer').LanguageTransformDescriptor<Condition>} */
export const englishTransforms = {
language: "en",
conditions,
transforms: {
// omitted
},
};
```

Now you should be able to check for types whenever writing a deinflection rule.

### Text Postprocessors

In special cases, text may need to be modified after deinflection. These work exactly like text preprocessors, but are applied after deinflection. Currently, this is only used for Korean, where the Hangul text is disassembled into jamo during preprocessing, and so must be reassembled after deinflection.
Expand Down
5 changes: 0 additions & 5 deletions ext/css/display.css
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,6 @@

/* Variables */
:root {
/* Fonts */
--font-family: 'sans-serif';
/* Strings */
--compact-list-separator: ' | ';

Expand Down Expand Up @@ -250,9 +248,6 @@


/* Fonts */
* {
font-family: var(--font-family);
}
@font-face {
font-family: kanji-stroke-orders;
src: url('/data/fonts/kanji-stroke-orders.ttf');
Expand Down
2 changes: 1 addition & 1 deletion ext/css/material.css
Original file line number Diff line number Diff line change
Expand Up @@ -1241,7 +1241,7 @@ button.icon-button>.icon-button-inner {
top: 0;
right: 0;
bottom: 0;
z-index: 101;
z-index: 1001;
outline: none;
overflow: hidden;
}
Expand Down
2 changes: 1 addition & 1 deletion ext/data/schemas/options-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@
},
"fontFamily": {
"type": "string",
"default": "sans-serif"
"default": ""
},
"fontSize": {
"type": "number",
Expand Down
15 changes: 14 additions & 1 deletion ext/js/data/options-util.js
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ export class OptionsUtil {
resultOutputMode: 'group',
debugInfo: false,
maxResults: 32,
fontFamily: 'sans-serif',
fontFamily: '',
fontSize: 14,
lineHeight: '1.5',
showAdvanced: false,
Expand Down Expand Up @@ -557,6 +557,7 @@ export class OptionsUtil {
this._updateVersion43,
this._updateVersion44,
this._updateVersion45,
this._updateVersion46,
];
/* eslint-enable @typescript-eslint/unbound-method */
if (typeof targetVersion === 'number' && targetVersion < result.length) {
Expand Down Expand Up @@ -1410,6 +1411,18 @@ export class OptionsUtil {
}
}

/**
* - Set default font to empty
* @type {import('options-util').UpdateFunction}
*/
async _updateVersion46(options) {
for (const profile of options.profiles) {
if (profile.options.general.fontFamily === 'sans-serif') {
profile.options.general.fontFamily = '';
}
}
}

/**
* @param {string} url
* @returns {Promise<chrome.tabs.Tab>}
Expand Down
2 changes: 1 addition & 1 deletion ext/js/display/display.js
Original file line number Diff line number Diff line change
Expand Up @@ -535,9 +535,9 @@ export class Display extends EventDispatcher {
* @param {string} lineHeight
*/
setFontOptions(fontFamily, fontSize, lineHeight) {
document.documentElement.style.setProperty('--font-family', fontFamily);
// Setting these directly rather than using the existing CSS variables
// minimizes problems and ensures everything scales correctly
document.documentElement.style.fontFamily = fontFamily;
document.documentElement.style.fontSize = `${fontSize}px`;
document.documentElement.style.lineHeight = lineHeight;
}
Expand Down
38 changes: 21 additions & 17 deletions ext/js/language/de/german-transforms.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,16 @@

import {prefixInflection, suffixInflection} from '../language-transforms.js';

/** @typedef {keyof typeof conditions} Condition */

// https://www.dartmouth.edu/~deutsch/Grammatik/Wortbildung/Separables.html
const separablePrefixes = ['ab', 'an', 'auf', 'aus', 'auseinander', 'bei', 'da', 'dabei', 'dar', 'daran', 'dazwischen', 'durch', 'ein', 'empor', 'entgegen', 'entlang', 'entzwei', 'fehl', 'fern', 'fest', 'fort', 'frei', 'gegenüber', 'gleich', 'heim', 'her', 'herab', 'heran', 'herauf', 'heraus', 'herbei', 'herein', 'herüber', 'herum', 'herunter', 'hervor', 'hin', 'hinab', 'hinauf', 'hinaus', 'hinein', 'hinterher', 'hinunter', 'hinweg', 'hinzu', 'hoch', 'los', 'mit', 'nach', 'nebenher', 'nieder', 'statt', 'um', 'vor', 'voran', 'voraus', 'vorbei', 'vorüber', 'vorweg', 'weg', 'weiter', 'wieder', 'zu', 'zurecht', 'zurück', 'zusammen'];

/**
* @param {string} prefix
* @param {string[]} conditionsIn
* @param {string[]} conditionsOut
* @returns {import('language-transformer').Rule}
* @param {Condition[]} conditionsIn
* @param {Condition[]} conditionsOut
* @returns {import('language-transformer').Rule<Condition>}
*/
function separatedPrefix(prefix, conditionsIn, conditionsOut) {
const germanLetters = 'a-zA-ZäöüßÄÖÜẞ';
Expand All @@ -48,22 +50,24 @@ const zuInfinitiveInflections = separablePrefixes.map((prefix) => {
return prefixInflection(prefix + 'zu', prefix, [], ['v']);
});

const conditions = {
v: {
name: 'Verb',
isDictionaryForm: true,
},
n: {
name: 'Noun',
isDictionaryForm: true,
},
adj: {
name: 'Adjective',
isDictionaryForm: true,
},
};

export const germanTransforms = {
language: 'de',
conditions: {
v: {
name: 'Verb',
isDictionaryForm: true,
},
n: {
name: 'Noun',
isDictionaryForm: true,
},
adj: {
name: 'Adjective',
isDictionaryForm: true,
},
},
conditions,
transforms: {
'nominalization': {
name: 'nominalization',
Expand Down
86 changes: 46 additions & 40 deletions ext/js/language/en/english-transforms.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,14 @@

import {prefixInflection, suffixInflection} from '../language-transforms.js';

/** @typedef {keyof typeof conditions} Condition */

/**
* @param {string} consonants
* @param {string} suffix
* @param {string[]} conditionsIn
* @param {string[]} conditionsOut
* @returns {import('language-transformer').SuffixRule[]}
* @param {Condition[]} conditionsIn
* @param {Condition[]} conditionsOut
* @returns {import('language-transformer').SuffixRule<Condition>[]}
*/
function doubledConsonantInflection(consonants, suffix, conditionsIn, conditionsOut) {
const inflections = [];
Expand Down Expand Up @@ -64,7 +66,9 @@ const phrasalVerbPrepositions = ['aback', 'about', 'above', 'across', 'after', '
const particlesDisjunction = phrasalVerbParticles.join('|');
const phrasalVerbWordSet = new Set([...phrasalVerbParticles, ...phrasalVerbPrepositions]);
const phrasalVerbWordDisjunction = [...phrasalVerbWordSet].join('|');
/** @type {import('language-transformer').Rule} */
/**
* @type {import('language-transformer').Rule<Condition>}
*/
const phrasalVerbInterposedObjectRule = {
type: 'other',
isInflected: new RegExp(`^\\w* (?:(?!\\b(${phrasalVerbWordDisjunction})\\b).)+ (?:${particlesDisjunction})`),
Expand All @@ -78,7 +82,7 @@ const phrasalVerbInterposedObjectRule = {
/**
* @param {string} inflected
* @param {string} deinflected
* @returns {import('language-transformer').Rule}
* @returns {import('language-transformer').Rule<Condition>}
*/
function createPhrasalVerbInflection(inflected, deinflected) {
return {
Expand All @@ -93,8 +97,8 @@ function createPhrasalVerbInflection(inflected, deinflected) {
}

/**
* @param {import('language-transformer').SuffixRule[]} sourceRules
* @returns {import('language-transformer').Rule[]}
* @param {import('language-transformer').SuffixRule<Condition>[]} sourceRules
* @returns {import('language-transformer').Rule<Condition>[]}
*/
function createPhrasalVerbInflectionsFromSuffixInflections(sourceRules) {
return sourceRules.flatMap(({isInflected, deinflected}) => {
Expand All @@ -105,41 +109,43 @@ function createPhrasalVerbInflectionsFromSuffixInflections(sourceRules) {
});
}

/** @type {import('language-transformer').LanguageTransformDescriptor} */
const conditions = {
v: {
name: 'Verb',
isDictionaryForm: true,
subConditions: ['v_phr'],
},
v_phr: {
name: 'Phrasal verb',
isDictionaryForm: true,
},
n: {
name: 'Noun',
isDictionaryForm: true,
subConditions: ['np', 'ns'],
},
np: {
name: 'Noun plural',
isDictionaryForm: true,
},
ns: {
name: 'Noun singular',
isDictionaryForm: true,
},
adj: {
name: 'Adjective',
isDictionaryForm: true,
},
adv: {
name: 'Adverb',
isDictionaryForm: true,
},
};

/** @type {import('language-transformer').LanguageTransformDescriptor<Condition>} */
export const englishTransforms = {
language: 'en',
conditions: {
v: {
name: 'Verb',
isDictionaryForm: true,
subConditions: ['v_phr'],
},
v_phr: {
name: 'Phrasal verb',
isDictionaryForm: true,
},
n: {
name: 'Noun',
isDictionaryForm: true,
subConditions: ['np', 'ns'],
},
np: {
name: 'Noun plural',
isDictionaryForm: true,
},
ns: {
name: 'Noun singular',
isDictionaryForm: true,
},
adj: {
name: 'Adjective',
isDictionaryForm: true,
},
adv: {
name: 'Adverb',
isDictionaryForm: true,
},
},
conditions,
transforms: {
'plural': {
name: 'plural',
Expand Down
Loading

0 comments on commit 689fa65

Please sign in to comment.