Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix puml not syncing Resolve #1617 #2345

Merged
merged 44 commits into from
Aug 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
f493979
Fix `puml` not syncing.
SPWwj Jul 23, 2023
eae18c8
Fix lint error
SPWwj Jul 24, 2023
7fa3dbc
Revert plantuml.test.ts
SPWwj Jul 24, 2023
16f3edd
Fix file not found on wait
SPWwj Jul 26, 2023
6e7cddb
Update to LockManager
SPWwj Jul 31, 2023
67c4fc6
Merged index.ts
SPWwj Aug 2, 2023
9c52f52
Update index.ts
SPWwj Aug 2, 2023
4d66abc
Update index.ts
SPWwj Aug 2, 2023
c7dfa89
Fix lint
SPWwj Aug 2, 2023
fff34f0
Fix diagram not waiting on serve -d
SPWwj Aug 4, 2023
da44bf0
Revert .gitignore
SPWwj Aug 4, 2023
e849998
Update createLock to accept optional parameter
SPWwj Aug 5, 2023
cc6f30f
Merge branch 'master' into pr-puml
SPWwj Aug 5, 2023
ca27621
Update LockManager.ts doc comment
SPWwj Aug 5, 2023
5c1870d
Merge branch 'pr-puml' of https://github.com/SPWwj/markbind into pr-puml
SPWwj Aug 5, 2023
094d49e
Introduce Lock Service for Async Functions
SPWwj Aug 5, 2023
92806c9
Merge branch 'pr-puml' of https://github.com/SPWwj/markbind into pr-puml
SPWwj Aug 12, 2023
2b747c9
Introduce Lock Service for Async Functions
SPWwj Aug 12, 2023
cbd5e96
Merge branch 'pr-puml' of https://github.com/SPWwj/markbind into pr-puml
SPWwj Aug 12, 2023
0f6fd66
Add support to page content
SPWwj Aug 12, 2023
2b6f1f3
Revert Site index.ts
SPWwj Aug 12, 2023
1022c75
Remove processedDiagrams.clear()
SPWwj Aug 12, 2023
03263ee
Fix puml to reprocess all distinct images
SPWwj Aug 13, 2023
81a8037
Fix lint error
SPWwj Aug 13, 2023
a32e38d
Clear dead diagram
SPWwj Aug 14, 2023
e23d92b
Apply suggestions from code review
tlylt Aug 19, 2023
760b04e
Merge branch 'master' into pr-puml
tlylt Aug 19, 2023
2b79e8a
Remove redundant assets
SPWwj Aug 19, 2023
f4636b4
Update setTimeout to 100
SPWwj Aug 19, 2023
d0d1338
Add test
SPWwj Aug 19, 2023
02a805e
refactor
tlylt Aug 19, 2023
ca312d6
Add comment for stale tracking
SPWwj Aug 19, 2023
9ad306f
Rename isStale to isFresh
SPWwj Aug 19, 2023
5e029a3
Update comments for processedDiagrams
SPWwj Aug 19, 2023
e30f5d5
Fix lint
SPWwj Aug 19, 2023
d0ff17d
Update clearDeadDiagrams to remove unused diagrams
SPWwj Aug 20, 2023
e7d5a9e
minor refactor
tlylt Aug 21, 2023
a46f136
Merge branch 'master' into pr-puml
SPWwj Aug 25, 2023
89e3de5
Remove delete stale diagram feature
SPWwj Aug 26, 2023
ced74fb
Fix .puml file not update
SPWwj Aug 26, 2023
da5bef0
refactor import and comment
tlylt Aug 27, 2023
4727492
update ug inline puml
tlylt Aug 27, 2023
938d125
cleanup docs
tlylt Aug 27, 2023
ee49d91
update comment
tlylt Aug 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 1 addition & 9 deletions docs/devGuide/development/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,15 +375,7 @@ To update PlantUML to a newer version:

1. Download the JAR file from [PlantUML's website](https://plantuml.com/download).
1. Rename the file to `plantuml.jar` (if required), and replace the existing JAR file located in `packages/core/src/plugins/default`.
1. Generate the image files for the `.puml` files listed in `docs/userGuide/diagrams`.

<box type="tip" seamless header="Here are the recommended steps to generate the image files:">

1. Add a new `.md` file in `userGuide`, e.g. `plantuml.md`, containing `<puml>` tags of all diagrams to be generated.
1. Serve the documentation site using `markbind serve -d`.
1. Access the corresponding HTML page with the generated diagrams, i.e. `/userGuide/plantuml.html`.
1. Right-click on each image and save the image in `docs/userGuide/diagrams`.
</box>
1. Check the HTML pages that contain PlantUML diagrams, i.e. `/userGuide/components/imagesAndDiagrams.html`.

### Updating Bootstrap and Bootswatch

Expand Down
Binary file removed docs/userGuide/diagrams/activity.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/archimate.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/class.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/component.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/ditaa.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/entityrelation.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/gantt.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/object.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/sequence.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/state.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/usecase.png
Binary file not shown.
44 changes: 19 additions & 25 deletions docs/userGuide/syntax/diagrams.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,9 @@ See [Deploying via Github Actions](../deployingTheSite.html#deploying-via-github
</box>

<div id="main-example">
<include src="outputBox.md" boilerplate>
<include src="codeAndOutput.md" boilerplate>
<variable name="code">

```
<puml width="300">
@startuml
alice -> bob ++ : hello
Expand All @@ -52,11 +51,6 @@ bob -> george !! : delete
return success
@enduml
</puml>
```
</variable>

<variable name="output">
<pic src="../diagrams/sequence.png" width="300" />
</variable>

</include>
Expand Down Expand Up @@ -88,7 +82,7 @@ in another file:
</variable>

<variable id="output">
<pic src="../diagrams/sequence.png" width="300" />
<puml src="../diagrams/sequence.puml" width=300 />
</variable>

</include>
Expand All @@ -103,50 +97,50 @@ The full PlantUML syntax reference can be found at plantuml.com/guide
<div id="puml-examples">

**Sequence Diagram**:<br>
<pic src="../diagrams/sequence.png" />
<puml src="../diagrams/sequence.puml" />

**Use Case Diagram**:<br>
<pic src="../diagrams/usecase.png" />
<puml src="../diagrams/usecase.puml" />

**Class Diagram**:<br>
<pic src="../diagrams/class.png" />
<puml src="../diagrams/class.puml" />

**Activity Diagram**:<br>
<pic src="../diagrams/activity.png" />
<puml src="../diagrams/activity.puml" />

**Component Diagram**:<br>
<pic src="../diagrams/component.png" />
<puml src="../diagrams/component.puml" />

**State Diagram**:<br>
<pic src="../diagrams/state.png" />
<puml src="../diagrams/state.puml" />

**Object Diagram**:<br>
<pic src="../diagrams/object.png" />
<puml src="../diagrams/object.puml" />

**Gantt Diagram**:<br>
<pic src="../diagrams/gantt.png" />
<puml src="../diagrams/gantt.puml" />

**Entity Relation Diagram**:<br>
<pic src="../diagrams/entityrelation.png" />
<puml src="../diagrams/entityrelation.puml" />

**Ditaa Diagram**:<br>
<pic src="../diagrams/ditaa.png" />
<puml src="../diagrams/ditaa.puml" />

**Archimate Diagram**:<br>
<pic src="../diagrams/archimate.png" />
<puml src="../diagrams/archimate.puml" />

</div>
</panel>
<p/>

****Options****
Name | Type | Description
--- | --- | ---
alt | `string` | The alternative text of the diagram.
Name | Type | Description
-----|----------|-------------------------------------
alt | `string` | The alternative text of the diagram.
height | `string` | The height of the diagram in pixels.
name | `string` | The name of the output file.
src | `string` | The URL of the diagram if your diagram is in another `.puml` file.<br>The URL can be specified as absolute or relative references. More info in: _[Intra-Site Links]({{baseUrl}}/userGuide/formattingContents.html#intraSiteLinks)_
width | `string` | The width of the diagram in pixels.<br>If both width and height are specified, width takes priority over height. It is to maintain the diagram's aspect ratio.
name | `string` | The name of the output file.<br>Avoid using the same name for different diagrams to prevent overwriting.
src | `string` | The URL of the diagram if your diagram is in another `.puml` file.<br>The URL can be specified as absolute or relative references. More info in: _[Intra-Site Links]({{baseUrl}}/userGuide/formattingContents.html#intraSiteLinks)_
width | `string` | The width of the diagram in pixels.<br>If both width and height are specified, width takes priority over height. It is to maintain the diagram's aspect ratio.

<div id="short" class="d-none">

Expand Down
5 changes: 5 additions & 0 deletions packages/core/src/Page/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@

const _ = { cloneDeep, isObject, isArray };

const LockManager = require('../utils/LockManager');

const PACKAGE_VERSION = require('../../package.json').version;

const {
Expand Down Expand Up @@ -534,6 +536,9 @@
// Each source path will only contain 1 copy of build/re-build page (the latest one)
pageVueServerRenderer.pageEntries[this.pageConfig.sourcePath] = builtPage;

// Wait for all pages resources to be generated before writing to disk
await LockManager.waitForLockRelease();

Check warning on line 540 in packages/core/src/Page/index.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/Page/index.ts#L540

Added line #L540 was not covered by tests

/*
* Server-side render Vue page app into actual html.
*
Expand Down
39 changes: 31 additions & 8 deletions packages/core/src/plugins/default/markbind-plugin-plantuml.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,24 @@
import { PluginContext } from '../Plugin';
import { NodeProcessorConfig } from '../../html/NodeProcessor';
import { MbNode } from '../../utils/node';
import LockManager from '../../utils/LockManager';

interface DiagramStatus {
hashKey: string;
}

const JAR_PATH = path.resolve(__dirname, 'plantuml.jar');

const processedDiagrams = new Set();
const PUML_EXT = '.png';

/**
* This Map maintains a record of processed diagrams. When a diagram is generated or regenerated,
* it's added to this map. Subsequently, if a PUML or non-PUML file is edited, leading to a hot reload,
* the generateDiagram function can avoid redundant regeneration by checking this map.
* If the diagram's identifier is present in the map,
* the generation process is bypassed, thus preventing duplicates.
*/
const processedDiagrams = new Map<string, DiagramStatus>();

let graphvizCheckCompleted = false;

Expand All @@ -28,15 +42,22 @@
* @param content puml dsl used to generate the puml diagram
*/
function generateDiagram(imageOutputPath: string, content: string) {
const hashKey = cryptoJS.MD5(imageOutputPath + content).toString();

// Avoid generating twice
if (processedDiagrams.has(imageOutputPath)) { return; }
processedDiagrams.add(imageOutputPath);
if (processedDiagrams.has(imageOutputPath) && processedDiagrams.get(imageOutputPath)?.hashKey === hashKey) {
return;

Check warning on line 49 in packages/core/src/plugins/default/markbind-plugin-plantuml.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/plugins/default/markbind-plugin-plantuml.ts#L49

Added line #L49 was not covered by tests
}

// Creates output dir if it doesn't exist
const outputDir = path.dirname(imageOutputPath);
if (!fs.existsSync(outputDir)) {
fs.mkdirSync(outputDir, { recursive: true });
}
const lockId = LockManager.createLock();

// Add new diagram to the map
processedDiagrams.set(imageOutputPath, { hashKey });

// Java command to launch PlantUML jar
const cmd = `java -jar "${JAR_PATH}" -nometadata -pipe > "${imageOutputPath}"`;
Expand All @@ -59,6 +80,7 @@
childProcess.on('error', (error) => {
logger.debug(error as unknown as string);
logger.error(`Error generating ${imageOutputPath}`);
LockManager.deleteLock(lockId);

Check warning on line 83 in packages/core/src/plugins/default/markbind-plugin-plantuml.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/plugins/default/markbind-plugin-plantuml.ts#L83

Added line #L83 was not covered by tests
});

childProcess.stderr?.on('data', (errorMsg) => {
Expand All @@ -68,6 +90,7 @@
childProcess.on('exit', () => {
// This goes to the log file, but not shown on the console
logger.debug(errorLog);
LockManager.deleteLock(lockId);

Check warning on line 93 in packages/core/src/plugins/default/markbind-plugin-plantuml.ts

View check run for this annotation

Codecov / codecov/patch

packages/core/src/plugins/default/markbind-plugin-plantuml.ts#L93

Added line #L93 was not covered by tests
});
}

Expand All @@ -90,7 +113,6 @@
},

beforeSiteGenerate: () => {
processedDiagrams.clear();
graphvizCheckCompleted = false;
},

Expand All @@ -111,6 +133,7 @@

let pumlContent;
let pathFromRootToImage;

if (node.attribs.src) {
const srcWithoutBaseUrl = urlUtil.stripBaseUrl(node.attribs.src, config.baseUrl);
const srcWithoutLeadingSlash = srcWithoutBaseUrl.startsWith('/')
Expand All @@ -126,8 +149,8 @@
return;
}

pathFromRootToImage = fsUtil.setExtension(srcWithoutLeadingSlash, '.png');
node.attribs.src = fsUtil.ensurePosix(fsUtil.setExtension(node.attribs.src, '.png'));
pathFromRootToImage = fsUtil.setExtension(srcWithoutLeadingSlash, PUML_EXT);
node.attribs.src = fsUtil.ensurePosix(fsUtil.setExtension(node.attribs.src, PUML_EXT));
} else {
pumlContent = cheerio(node).text();

Expand All @@ -136,13 +159,13 @@
const nameWithoutLeadingSlash = nameWithoutBaseUrl.startsWith('/')
? nameWithoutBaseUrl.substring(1)
: nameWithoutBaseUrl;
pathFromRootToImage = fsUtil.ensurePosix(fsUtil.setExtension(nameWithoutLeadingSlash, '.png'));
pathFromRootToImage = fsUtil.ensurePosix(fsUtil.setExtension(nameWithoutLeadingSlash, PUML_EXT));

delete node.attribs.name;
} else {
const normalizedContent = pumlContent.replace(/\r\n/g, '\n');
const hashedContent = cryptoJS.MD5(normalizedContent).toString();
pathFromRootToImage = `${hashedContent}.png`;
pathFromRootToImage = `${hashedContent}${PUML_EXT}`;
}

node.attribs.src = `${config.baseUrl}/${pathFromRootToImage}`;
Expand Down
86 changes: 86 additions & 0 deletions packages/core/src/utils/LockManager.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import { v4 as uuidv4 } from 'uuid';

/**
* The `LockManager` is a singleton class designed to help wait for required async
* promised operations to complete
* before the page is generated. It provides functionalities to create, delete, and wait
* for the release of locks.
* The locks are stored in a Map with a unique ID (either provided or auto-generated) as
* the key.
* The class provides an instance property to get the singleton instance of `LockManager`.
*/

class LockManager {
tlylt marked this conversation as resolved.
Show resolved Hide resolved
// Holds the single instance of LockManager.
private static _instance: LockManager;

// A Map to keep track of the active locks.
private locks: Map<string, boolean>;

/**
* Private constructor to prevent direct instantiation from outside.
* Initializes the locks Map.
*/
private constructor() {
this.locks = new Map();
tlylt marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Provides a way to access the single instance of the LockManager.
* If it doesn't exist, it creates one.
* @returns {LockManager} The single instance of LockManager.
*/
public static get instance() {
tlylt marked this conversation as resolved.
Show resolved Hide resolved
if (!LockManager._instance) {
LockManager._instance = new LockManager();
}

return LockManager._instance;
}

/**
* Creates a new lock.
* @param {string} [id] - An optional ID to use for the lock. If not provided, a UUID will be generated.
* @returns {string} The ID of the created lock.
*/
createLock(id?: string): string {
const lockId = id ?? uuidv4();
this.locks.set(lockId, true);
return lockId;
}

/**
* Deletes a lock by its ID.
* @param {string} lockId - The ID of the lock to be deleted.
*/
deleteLock(lockId: string): void {
this.locks.delete(lockId);
}

/**
* Deletes all locks, clearing the locks Map.
*/
deleteAllLocks(): void {
this.locks.clear();
}

/**
* Waits until all locks are released and then resolves.
* @returns {Promise<void>} A promise that resolves when all locks are released.
*/
waitForLockRelease(): Promise<void> {
return new Promise((resolve) => {
const checkLocks = () => {
if (this.locks.size === 0) {
resolve();
} else {
setTimeout(checkLocks, 100);
tlylt marked this conversation as resolved.
Show resolved Hide resolved
}
};
checkLocks();
});
}
}

// Export the singleton instance of LockManager.
export = LockManager.instance;
42 changes: 42 additions & 0 deletions packages/core/test/unit/utils/LockManager.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import LockManager from '../../../src/utils/LockManager';

describe('LockManager', () => {
let lockManager: typeof LockManager;

beforeEach(() => {
lockManager = LockManager;
});

afterEach(() => {
lockManager.deleteAllLocks();
});

it('should create a new lock', () => {
const lockId = lockManager.createLock();
expect(lockId).toBeDefined();
});

it('should use the provided ID when creating a lock', () => {
const lockId = 'customId';
const createdLockId = lockManager.createLock(lockId);
expect(createdLockId).toEqual(lockId);
});

it('should delete all locks', () => {
lockManager.createLock();
lockManager.createLock();
lockManager.deleteAllLocks();
});

it('should wait until all locks are released and resolve', async () => {
const lockId1 = lockManager.createLock();
const lockId2 = lockManager.createLock();

const waitForLockReleasePromise = lockManager.waitForLockRelease();

setTimeout(() => lockManager.deleteLock(lockId1), 100);
setTimeout(() => lockManager.deleteLock(lockId2), 200);

await expect(waitForLockReleasePromise).resolves.toBeUndefined();
});
});