Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up scripts for old website #391

Merged
merged 2 commits into from
Apr 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 8 additions & 31 deletions scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,59 +15,36 @@ For example, to point all links to innersourcecommons.org:
node substitute_article_urls.js isc
```

## generate_jekyll_learning_path_markdown.js, generate_hugo_learning_path_markdown.js
## generate_learning_path_markdown.js

Node scripts to generate markdown files required for hosting Learning Path on innersourcecommons.org and innersourcecommons.net.
Node script to generate markdown files required for hosting Learning Path on innersourcecommons.org.

These scripts require a [GitHub access token](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token), as they use the GitHub API to get Learning Path contributors. Your token does not require any scopes, as the Learning Path is Open Source. To provide this, create a `.env` file in this directory in the following format:
This script requires a [GitHub access token](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token), as it uses the GitHub API to get Learning Path contributors. Your token does not require any scopes, as the Learning Path is Open Source. To provide this, create a `.env` file in this directory in the following format:
```
TOKEN=<your_github_token>
```

### Usage:
```
npm ci
node generate_jekyll_learning_path_markdown.js
node generate_hugo_learning_path_markdown.js
node generate_learning_path_markdown.js
```

## How to update innersourcecommons.org with new articles

Anytime there are _new_ articles written, you need to run the script that generates the new pages on the [innersourcecommons.org] site.
Anytime there are _any changes_, you need to run the script that generates the updated pages on the [innersourcecommons.org] site.

1. Clone the [InnerSourceCommons/InnerSourceLearningPath] and [InnerSourceCommons/innersourcecommons.org] repos.
1. Is this the first time that articles have been written for this section in this language?
If so, then do both of the following:

* update the ["sections" config](https://github.com/InnerSourceCommons/InnerSourceLearningPath/blob/master/scripts/section_data.json) with the language code of the articles for the appropriate section.
Open a pull request for the change.
* update the [Learning Path landing page](https://github.com/InnerSourceCommons/innersourcecommons.org/blob/master/resources/learningpath/index.md) with a link to your new language pages.

3. Run **generate_jekyll_learning_path_markdown.js** as described above.
3. `cp -r learningpath/* <path-to-innersourcecommons.org-repo>/resources/learningpath/`.
3. Open a pull request with the modified files in the [InnerSourceCommons/innersourcecommons.org] repo.

Note that these steps only needs to happen when there are new articles written.
Changes to existing articles will automatically show up on the [innersourcecommons.org] site.

## How to update innersourcecommons.net

Anytime there are _any changes_, you need to run the script that generates the updated pages on the [innersourcecommons.net] site.

1. Clone the [InnerSourceCommons/InnerSourceLearningPath] and [InnerSourceCommons/innersourcecommons.net] repos.
1. Is this the first time that articles have been written for this section in this language?
If so, then do both of the following:

* update the ["sections" config](https://github.com/InnerSourceCommons/InnerSourceLearningPath/blob/master/scripts/section_data.json) with the language code of the articles for the appropriate section and open a pull request for the change
* make sure this language appears in the [website config](https://github.com/InnerSourceCommons/innersourcecommons.net/blob/master/config.yaml) and there are the relevant [index pages](https://github.com/InnerSourceCommons/innersourcecommons.net/blob/master/content/learn/_index.ru.md)

3. Run `node substitute_article_urls.js isc.net` as described above. This changes the asciidoc source files' links to point at [innnersourcecommons.net] - do not commit this change!
3. Run **generate_hugo_learning_path_markdown.js** as described above.
3. `cp -r newsite/ <path-to-innersourcecommons.net-repo>/content/learn/learning-path/`.
3. Open a pull request with the modified files in the [InnerSourceCommons/innersourcecommons.net] repo.
3. Run **generate_learning_path_markdown.js** as described above.
3. `cp -r newsite/ <path-to-innersourcecommons.org-repo>/content/learn/learning-path/`.
3. Open a pull request with the modified files in the [InnerSourceCommons/innersourcecommons.org] repo.

[innersourcecommons.org]: https://innersourcecommons.org/
[innersourcecommons.net]: https://innersourcecommons.net/
[InnerSourceCommons/InnerSourceLearningPath]: https://github.com/InnerSourceCommons/InnerSourceLearningPath/
[InnerSourceCommons/innersourcecommons.org]: https://github.com/InnerSourceCommons/innersourcecommons.org
[InnerSourceCommons/innersourcecommons.net]: https://github.com/InnerSourceCommons/innersourcecommons.net
55 changes: 0 additions & 55 deletions scripts/generate_hugo_learning_path_markdown.js

This file was deleted.

43 changes: 0 additions & 43 deletions scripts/generate_jekyll_learning_path_markdown.js

This file was deleted.

94 changes: 55 additions & 39 deletions scripts/generate_learning_path_markdown.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
const { join, basename, relative } = require('path')
const fs = require('fs')
const yamlFront = require('yaml-front-matter')
const asciidoctor = require('asciidoctor')()
const YAML = require('yaml')

const getContributors = require('./get_contributors')
const mkdirSync = require('./mkdir_sync')
Expand All @@ -18,7 +20,17 @@ const languageMap = {
ru: 'Russian'
}

module.exports = async (writeDir, generatorFn, workbookFn, isJekyllSite) => {
const urls = YAML.parse(fs.readFileSync(join('..', 'config', 'urls.yaml'), 'utf-8'))

const getYouTubeCode = (section, articleNumber) => {
const firstEntryOfGroupIndex = urls.findIndex(entry => entry.section === section.toLowerCase())
const currentPageIndexOffset = articleNumber - 1
const youtubeUrl = urls[firstEntryOfGroupIndex + currentPageIndexOffset].video.youtube
return youtubeUrl.replace('https://www.youtube.com/watch?v=', '')
}

(async () => {
const writeDir = 'newsite'
mkdirSync(join('.', writeDir))

sections.forEach((section, sectionDataIndex) => {
Expand All @@ -29,51 +41,50 @@ module.exports = async (writeDir, generatorFn, workbookFn, isJekyllSite) => {

translations.concat('' /* The English original */).forEach(async (translation) => {
const isTranslation = translation !== ''
const writePath = join(baseWritePath, translation)
const readPath = join(baseReadPath, translation)
const articles = getArticleFiles(readPath)

// Bad separation of concerns (Hugo/Jekyll specific logic shouldn't be here) but is temporary until Jekyll site is decomissioned
if (isJekyllSite) {
mkdirSync(writePath)
} else {
let indexReadPath = join(readPath, 'index.md')
const indexTranslated = fs.existsSync(indexReadPath)
if (!indexTranslated) {
// Default to English index page content
indexReadPath = join(baseReadPath, 'index.md')
}
const indexWritePath = isTranslation ? join(baseWritePath, `_index.${translation}.md`) : join(baseWritePath, '_index.md')

let { indexContent, ...indexFrontMatter } = yamlFront.loadFront(fs.readFileSync(indexReadPath, 'utf-8'), { contentKeyName: 'indexContent' })
indexFrontMatter.image = section.image
indexFrontMatter.contributors = await getContributors(relative('..', readPath))
indexFrontMatter.weight = sectionDataIndex
let indexReadPath = join(readPath, 'index.md')
const indexTranslated = fs.existsSync(indexReadPath)
if (!indexTranslated) {
// Default to English index page content
indexReadPath = join(baseReadPath, 'index.md')
}
const indexWritePath = isTranslation ? join(baseWritePath, `_index.${translation}.md`) : join(baseWritePath, '_index.md')

if (!indexTranslated) {
// Add placeholder text if index is not translated
indexContent = `This text will be translated soon to ${languageMap[translation]}.${indexContent}`
}
let { indexContent, ...indexFrontMatter } = yamlFront.loadFront(fs.readFileSync(indexReadPath, 'utf-8'), { contentKeyName: 'indexContent' })
indexFrontMatter.image = section.image
indexFrontMatter.contributors = await getContributors(relative('..', readPath))
indexFrontMatter.weight = sectionDataIndex

writeMarkdownFile(indexWritePath, indexFrontMatter, indexContent)
if (!indexTranslated) {
// Add placeholder text if index is not translated
indexContent = `This text will be translated soon to ${languageMap[translation]}.${indexContent}`
}

writeMarkdownFile(indexWritePath, indexFrontMatter, indexContent)

articles.forEach(async (article) => {
const articleTitle = article.asciiDoc.match(/== (.*)/)[1]
const articleNumber = basename(article.filePath).split('-')[0]
const contributors = await getContributors(relative('..', article.filePath))
const fileName = isTranslation ? join(baseWritePath, [articleNumber, translation, 'md'].join('.')) : join(baseWritePath, [articleNumber, 'md'].join('.'))
const weight = parseInt(articleNumber)
const youtubeCode = getYouTubeCode(section.learning_path_group, weight)

generatorFn({
section,
articleTitle,
articleNumber,
isTranslation,
article,
translation,
const frontMatter = {
title: articleTitle,
contributors,
writePath,
baseWritePath
})
image: `https://img.youtube.com/vi/${youtubeCode}/mqdefault.jpg`,
featured: weight === 1,
weight,
youtubeCode
}

const titleStripped = article.asciiDoc.replace(/== (.*)/, '')
const body = section.renderArticles || isTranslation ? asciidoctor.convert(titleStripped) : ''

writeMarkdownFile(fileName, frontMatter, body)
})

// Workbooks not translated.
Expand All @@ -82,13 +93,18 @@ module.exports = async (writeDir, generatorFn, workbookFn, isJekyllSite) => {
const contributors = await getContributors(`workbook/${section.workbook}`)
const workbookPosition = articles.length + 1

workbookFn({
section,
workbookFileName,
const workbookFrontMatter = {
title: 'Workbook',
contributors,
workbookPosition
})
image: section.image,
weight: workbookPosition
}

const workbookReadPath = join('..', 'workbook', section.workbook)
const body = asciidoctor.convert(fs.readFileSync(workbookReadPath, 'utf-8'))

writeMarkdownFile(workbookFileName, workbookFrontMatter, body)
}
})
})
}
})()