Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle local type definitions in Monaco #2883

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Handle local type definitions in Monaco #2883

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7f70acf
editor: handle @local typedefs
josephjclark Jan 30, 2025
3ae97aa
tidy
josephjclark Jan 30, 2025
fc441af
editor: fix type tracking issue
josephjclark Feb 26, 2025
9393daa
remove log line
josephjclark Feb 26, 2025
1a04c2b
standardise regex
josephjclark Feb 26, 2025
8dc2f5e
slightly improve URL handling
josephjclark Feb 26, 2025
79a66d5
changelog
josephjclark Feb 26, 2025
c6478d8
revert lib stuff
josephjclark Mar 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ and this project adheres to

- Allow workflow and project concurrency progress windows
[#2995](https://github.com/OpenFn/lightning/issues/2995)
- Support for loading local type definitions when using the adaptors monorepo.

### Changed

Expand Down
75 changes: 57 additions & 18 deletions assets/js/editor/Editor.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,39 @@ import type { editor } from 'monaco-editor';
import type { EditorProps as MonacoProps } from '@monaco-editor/react';

import { MonacoEditor, type Monaco } from '../monaco';
import { fetchDTSListing, fetchFile } from '@openfn/describe-package';
import * as describe from '@openfn/describe-package';
import createCompletionProvider from './magic-completion';
import { initiateSaveAndRun } from '../common';

const LOCAL_ADAPTORS_ROOT = 'http://localhost:5000';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: update this constant when the static fileserver is set up


async function* fetchDTSListing(specifier: string) {
if (specifier.endsWith('@local')) {
const lang = specifier
.replace('@local', '')
.replace('@openfn/language-', '');
const url = `${LOCAL_ADAPTORS_ROOT}/packages/${lang}/types`;
const r = await fetch(url, { headers: { Accept: 'application/json' } });
const json = await r.json();
for (const f of json.files) {
yield `/types/${f.base}`;
}
} else {
return describe.fetchDTSListing(specifier);
}
}
const fetchFile = async (path: string) => {
if (path.includes('@local')) {
path = path.replace('@openfn/language-', '').replace('@local', '');
const url = `${LOCAL_ADAPTORS_ROOT}/packages/${path}`;
return fetch(url, { headers: { Accept: 'text/plain' } }).then(r =>
r.text()
);
} else {
return describe.fetchFile(path);
}
};

// static imports for core lib
import dts_es5 from './lib/es5.min.dts';

Expand Down Expand Up @@ -95,6 +124,8 @@ type Lib = {
};

async function loadDTS(specifier: string): Promise<Lib[]> {
const useLocal = specifier.endsWith('@local');

// Work out the module name from the specifier
// (his gets a bit tricky with @openfn/ module names)
const nameParts = specifier.split('@');
Expand All @@ -107,16 +138,18 @@ async function loadDTS(specifier: string): Promise<Lib[]> {
// TODO maybe we need other dependencies too? collections?
if (name !== '@openfn/language-common') {
const pkg = await fetchFile(`${specifier}/package.json`);
const commonVersion = JSON.parse(pkg || '{}').dependencies?.[
'@openfn/language-common'
];

// jsDeliver doesn't appear to support semver range syntax (^1.0.0, 1.x, ~1.1.0)
const commonVersionMatch = commonVersion?.match(/^\d+\.\d+\.\d+/);
if (!commonVersionMatch) {
console.warn(
`@openfn/language-common@${commonVersion} contains semver range syntax.`
);
const commonVersion = useLocal
? 'local'
: JSON.parse(pkg || '{}').dependencies?.['@openfn/language-common'];

if (!useLocal) {
// jsDeliver doesn't appear to support semver range syntax (^1.0.0, 1.x, ~1.1.0)
const commonVersionMatch = commonVersion?.match(/^\d+\.\d+\.\d+/);
if (!commonVersionMatch) {
console.warn(
`@openfn/language-common@${commonVersion} contains semver range syntax.`
);
}
}

const commonSpecifier = `@openfn/language-common@${commonVersion.replace(
Expand All @@ -127,10 +160,12 @@ async function loadDTS(specifier: string): Promise<Lib[]> {
if (!filePath.startsWith('node_modules')) {
// Load every common typedef into the common module
let content = await fetchFile(`${commonSpecifier}${filePath}`);
content = content.replace(/\* +@(.+?)\*\//gs, '*/');
results.push({
content: `declare module '@openfn/language-common' { ${content} }`,
});
if (!content.match(/<!doctype html>/i)) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO check this doctype stuff - this is from npm serve returning folders, I think. When the plug is enabled I'll come back and have another look.

content = content.replace(/\* +@(.+?)\*\//gs, '*/');
results.push({
content: `declare module '@openfn/language-common' { ${content} }`,
});
}
}
}
}
Expand All @@ -144,6 +179,13 @@ async function loadDTS(specifier: string): Promise<Lib[]> {
for await (const filePath of fetchDTSListing(specifier)) {
if (!filePath.startsWith('node_modules')) {
let content = await fetchFile(`${specifier}${filePath}`);
if (content.match(/<!doctype html>/i)) {
continue;
}
// Convert relative paths
content = content
.replace(/from '\.\//g, `from '${name}/`)
.replace(/import '\.\//g, `import '${name}/`);

// Remove js doc annotations
// this regex means: find a * then an @ (with 1+ space in between), then match everything up to a closing comment */
Expand All @@ -154,9 +196,6 @@ async function loadDTS(specifier: string): Promise<Lib[]> {

// Import the index as the global namespace - but take care to convert all paths to absolute
if (fileName === 'index' || fileName === 'Adaptor') {
content = content.replace(/from '\.\//g, `from '${name}/`);
content = content.replace(/import '\.\//g, `import '${name}/`);

// It turns out that "export * as " seems to straight up not work in Monaco
// So this little hack will refactor import statements in a way that works
content = content.replace(
Expand Down