Skip to content

Commit 5cc30c2

Browse files
benfdkingbenfdking
committed
opening extension in right place
registered logging [ci skip] format works making progress making progress sharing node modules creating the docs progress getting react server to show making progress with the server temp get calling api now need to add lineage tab making progress on vscode lineage temp progress: showing the lineage graph [ci skip] cleaning up lineage [ci skip] added ability to open files temp [ci skip] trying to implement lsp [ci skip] temp [ci skip] test [ci skip] temp [ci skip] temp [ci skip] temp [ci skip]
1 parent 633fe09 commit 5cc30c2

File tree

109 files changed

+34160
-6
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

109 files changed

+34160
-6
lines changed

‎Makefile

+4
Original file line numberDiff line numberDiff line change
@@ -178,3 +178,7 @@ athena-test: guard-AWS_ACCESS_KEY_ID guard-AWS_SECRET_ACCESS_KEY guard-ATHENA_S3
178178
vscode_settings:
179179
mkdir -p .vscode
180180
cp -r ./tooling/vscode/*.json .vscode/
181+
182+
vscode-generate-openapi:
183+
python3 web/server/openapi.py --output vscode/extension/openapi.json
184+
cd vscode/react && npm run generate:api

‎docs/development.md

+86
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,11 @@ SQLMesh is licensed under [Apache 2.0](https://github.com/TobikoData/sqlmesh/blo
44

55
## Prerequisites
66

7+
<<<<<<< HEAD
78
Before you begin, ensure you have the following installed on your machine. Exactly how to install these is dependent on your operating system.
9+
=======
10+
Before you begin, ensure you have the following installed on your machine. Exacltly how to install these is dependent on your operating system.
11+
>>>>>>> afd4b2c1 (opening extension in right place)
812
913
* Docker
1014
* Docker Compose V2
@@ -25,9 +29,47 @@ Once you have activated your virtual environment, you can install the dependenci
2529
```bash
2630
make install-dev
2731
```
32+
<<<<<<< HEAD
2833

2934
Optionally, you can use pre-commit to automatically run linters/formatters:
3035

36+
=======
37+
<<<<<<< HEAD
38+
Run linters and formatters:
39+
```bash
40+
make style
41+
```
42+
Run faster tests for quicker local feedback:
43+
```bash
44+
make fast-test
45+
```
46+
Run more comprehensive tests that run on each commit:
47+
```bash
48+
make slow-test
49+
```
50+
Install docs dependencies:
51+
```bash
52+
make install-doc
53+
```
54+
Run docs server:
55+
```bash
56+
make docs-serve
57+
```
58+
Run docs tests:
59+
```bash
60+
make doc-test
61+
```
62+
Run ide:
63+
```bash
64+
make ui-up
65+
```
66+
(Optional) Use pre-commit to automatically run linters/formatters:
67+
=======
68+
69+
Optionally, you can use pre-commit to automatically run linters/formatters:
70+
71+
>>>>>>> 61520e5a (creating the docs)
72+
>>>>>>> afd4b2c1 (opening extension in right place)
3173
```bash
3274
make install-pre-commit
3375
```
@@ -52,6 +94,15 @@ Run more comprehensive tests that run on each commit:
5294
make slow-test
5395
```
5496

97+
<<<<<<< HEAD
98+
=======
99+
Run Airflow tests that will run when PR is merged to main:
100+
101+
```bash
102+
make airflow-docker-test-with-env
103+
```
104+
105+
>>>>>>> afd4b2c1 (opening extension in right place)
55106
## Documentation
56107

57108
In order to run the documentation server, you will need to install the dependencies by running the following command.
@@ -90,6 +141,7 @@ make ui-up
90141

91142
## Developing the VSCode extension
92143

144+
<<<<<<< HEAD
93145
Similar to UI development, you can also develop the VSCode extension. To do so, make sure you have the dependencies installed by running the following command inside the `vscode/extension` directory.
94146

95147
```bash
@@ -101,3 +153,37 @@ Once that is done, developing the VSCode extension is most easily done by launch
101153
```bash
102154
make vscode_settings
103155
```
156+
=======
157+
Developing the VSCode extension is most easily done by launching the debug process from a visual studio code workspace.
158+
159+
By default, the VSCode extension will run the SQLMesh server locally and open a new visual studio code window that allows you to try out the SQLMesh IDE. It by default opens the `examples/sushi` project.
160+
161+
Please see the below diagram for a high level overview of the UI.
162+
163+
```mermaid
164+
graph TD
165+
A[VSCode Extension] --> |start server| B[SQLMesh Server]
166+
A --> |creates webviews| C[Webviews]
167+
C --> |reads react webpages from| B
168+
A --> |calls lsp| B
169+
React[React App] --> |embedded in| B
170+
```
171+
172+
For development purposes, the React app is not embedded into the python server. Instead a separate instance of the React app is run. This allows you to make changes to the UI and see them immediately.
173+
174+
This makes the architecture diagram at development time look like the following.
175+
176+
```mermaid
177+
graph TD
178+
A[VSCode Extension] --> |start server| B[SQLMesh Server]
179+
A --> |creates webviews| C[Webviews]
180+
React [React Server] --> |passes on api requests| B
181+
C --> |reads react webpages from| React
182+
```
183+
184+
In development mode, the React app is not embedded into the python server. Instead a separate instance of the React app is run. This allows you to make changes to the UI and see them immediately.
185+
186+
The React App runs on the port 5173. The VSCode extension will automatically open the React app in a new tab.
187+
The python server will run on the port 5174.
188+
189+
>>>>>>> afd4b2c1 (opening extension in right place)

‎examples/sushi/.vscode/settings.json

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"editor.formatOnSave": true
3+
}

‎examples/sushi/requirements.txt

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
sqlmesh-utils
2+
sqlmesh
3+
psycopg2

‎examples/sushi/vscode_setup.sh

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
#!/bin/bash
2+
set -euo pipefail
3+
4+
create_sqlmesh_venv() {
5+
local venv_dir=".venv"
6+
local requirements_file="requirements.txt"
7+
local custom_mats_dir="../custom_materializations"
8+
9+
echo "Creating Python virtual environment in $venv_dir..."
10+
if ! python -m venv "$venv_dir"; then
11+
echo "ERROR: Failed to create virtual environment" >&2
12+
return 1
13+
fi
14+
15+
echo "Activating virtual environment..."
16+
source "$venv_dir/bin/activate" || {
17+
echo "ERROR: Failed to activate virtual environment" >&2
18+
return 1
19+
}
20+
21+
if [ -f "$requirements_file" ]; then
22+
echo "Installing requirements from $requirements_file..."
23+
if ! pip install -r "$requirements_file"; then
24+
echo "ERROR: Failed to install requirements" >&2
25+
deactivate
26+
return 1
27+
fi
28+
else
29+
echo "WARNING: $requirements_file not found" >&2
30+
fi
31+
32+
if [ -d "$custom_mats_dir" ]; then
33+
echo "Installing custom materializations from $custom_mats_dir..."
34+
if ! pip install -e "$custom_mats_dir"; then
35+
echo "ERROR: Failed to install custom materializations" >&2
36+
deactivate
37+
return 1
38+
fi
39+
else
40+
echo "ERROR: Custom materializations directory not found at $custom_mats_dir" >&2
41+
deactivate
42+
return 1
43+
fi
44+
45+
echo "Setup completed successfully. Virtual environment is active."
46+
return 0
47+
}
48+
49+
# Check if the script is being run in a terminal
50+
if [ -t 1 ]; then
51+
create_sqlmesh_venv
52+
else
53+
echo "This script should be run in a terminal." >&2
54+
exit 1
55+
fi

‎tooling/vscode/tasks.json

+35-4
Original file line numberDiff line numberDiff line change
@@ -18,16 +18,47 @@
1818
},
1919
"options": {
2020
"cwd": "${workspaceFolder}/vscode/extension"
21+
},
22+
"dependsOn": [
23+
"react-dev"
24+
],
25+
"dependsOrder": "parallel"
26+
},
27+
{
28+
"label": "react-dev",
29+
"script": "dev",
30+
"type": "npm",
31+
"options": {
32+
"cwd": "${workspaceFolder}/vscode/react"
33+
},
34+
"group": {
35+
"kind": "build",
36+
},
37+
"isBackground": true,
38+
"problemMatcher": {
39+
"owner": "npm",
40+
"pattern": {
41+
"regexp": "."
42+
},
43+
"background": {
44+
"activeOnStart": true,
45+
"beginsPattern": ".",
46+
"endsPattern": "."
47+
}
48+
},
49+
"presentation": {
50+
"reveal": "never",
51+
"group": "watchers"
2152
}
2253
},
23-
{
54+
{
2455
"label": "extension-watch-develop",
2556
"group": {
2657
"kind": "build",
2758
"isDefault": true
2859
},
29-
"dependsOn": ["extension-watch"],
30-
"dependsOrder": "parallel"
60+
"dependsOn": ["react-dev", "extension-watch"],
61+
"dependsOrder": "parallel",
3162
},
3263
{
3364
"type": "npm",
@@ -40,7 +71,7 @@
4071
},
4172
"group": "build",
4273
"options": {
43-
"cwd": "${workspaceFolder}/vscode/extension"
74+
"cwd": "${workspaceFolder}/vscode"
4475
}
4576
},
4677
{

‎vscode/bus/.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
dist
2+
node_modules

‎vscode/bus/package.json

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"name": "sqlmesh-extension-bus",
3+
"private": true,
4+
"version": "0.0.1",
5+
"type": "module",
6+
"scripts": {
7+
"build": "tsc",
8+
"dev": "tsc -w",
9+
"test": "jest"
10+
},
11+
"files": [
12+
"/dist"
13+
],
14+
"main": "dist/index.js",
15+
"types": "dist/index.d.ts",
16+
"dependencies": {
17+
"@types/vscode": "^1.92.0"
18+
},
19+
"devDependencies": {
20+
"@jest/globals": "^29.7.0",
21+
"@types/jest": "^29.5.12",
22+
"@types/vscode-webview": "^1.57.5",
23+
"jest": "^29.7.0",
24+
"ts-jest": "^29.2.5",
25+
"typescript": "^5.5.4"
26+
}
27+
}

‎vscode/bus/src/callbacks.ts

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
export type Callback = {
2+
openFile: {
3+
path: string
4+
}
5+
formatProject: {}
6+
}
7+
8+
/**
9+
* A tuple type representing a callback event with its associated payload.
10+
* The first element is the callback key (e.g., 'openFile', 'formatProject').
11+
* The second element is the payload type associated with that key.
12+
*
13+
* Example:
14+
* const openFileEvent: CallbackEvent<'openFile'> = ['openFile', { path: '/path/to/file' }];
15+
* const formatProjectEvent: CallbackEvent<'formatProject'> = ['formatProject', {}];
16+
*/
17+
export type CallbackEvent = {
18+
[K in keyof Callback]: { key: K; payload: Callback[K] }
19+
}[keyof Callback];

‎vscode/bus/tsconfig.json

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"compilerOptions": {
3+
"target": "ESNext",
4+
"module": "ESNext",
5+
"declaration": true,
6+
"outDir": "./dist",
7+
"strict": true,
8+
"noEmit": true,
9+
"moduleResolution": "node",
10+
"baseUrl": "./",
11+
},
12+
"include": ["src/**/*"]
13+
}

‎vscode/extension/.gitignore

+4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
node_modules
22
dist
33
out
4+
<<<<<<< HEAD
45
.vscode-test
56
*.vsix
67
LICENSE
8+
=======
9+
.vscode-test
10+
>>>>>>> c5ee2535 (progress)

‎vscode/extension/.vscodeignore

+3
Original file line numberDiff line numberDiff line change
@@ -12,5 +12,8 @@ vsc-extension-quickstart.md
1212
**/*.map
1313
**/*.ts
1414
**/.vscode-test.*
15+
<<<<<<< HEAD
1516
assets/logo.svg
1617
esbuild.js
18+
=======
19+
>>>>>>> c5ee2535 (progress)

0 commit comments

Comments
 (0)