From 202d1cc8f00dfbe61e083b47a54c5421472242a8 Mon Sep 17 00:00:00 2001 From: Kesara Rathnayake Date: Wed, 8 Sep 2021 09:38:30 +1200 Subject: [PATCH 01/25] Improves install.md instructions Adds additional instructions to getting a local instance up with a data dump. --- install.md | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/install.md b/install.md index cf195435..8632ab91 100644 --- a/install.md +++ b/install.md @@ -9,13 +9,36 @@ First, clone this repository. This project uses Docker to build and run the code, both frontend and backend. So the only requirement to run it locally is a recent version of Docker with docker-compose. -#### How to run +#### How to run (with a database dump) -1. (Optional) Obtain a recent database dump with name like `ietfa.torchbox.*.gz` and place in `docker/database/` directory. Otherwise, it will start with a fresh database. +1. Obtain a recent database dump with name like `ietfa.torchbox.*.gz` and place in `docker/database/` directory. Otherwise, it will start with a fresh database. 2. Obtain and unarchive media files into `media/` folder. 3. Run `docker-compose up`. It will build and start the frontend builder (`yarn run start`) and the backend (`python manage.py runserver` analog), along with a Postgresql database. The first run will take a while because the database dump needs to be restored. 4. After the frontend compilation finishes, the website should become available at http://localhost:8081 -5. To destroy everything (i.e. start the database from scratch) run `docker-compose down`. +5. Create a super user on **Python application** docker instance to access http://localhost:8081/admin +``` +docker exec -ti wagtail_website_application_1 python manage.py createsuperuser +``` +6. To destroy everything (i.e. start the database from scratch) run `docker-compose down`. + +#### How to run (without a database dump) + +1. Run `docker-compose up`. It will build and start the frontend builder (`yarn run start`) and the backend (`python manage.py runserver` analog), along with a Postgresql database. The first run will take a while because the database dump needs to be restored. +2. After **Python application** docker instance is started, run database migrations. +``` +docker exec -ti wagtail_website_application_1 python manage.py migrate +docker exec -ti wagtail_website_application_1 python manage.py createcachetable +``` +3. Create an admin user +``` +docker exec -ti wagtail_website_application_1 python manage.py createsuperuser +``` +4. Log into http://localhost:8081/admin +5. Create a new "Home Page" (page type must be `Home Page`) and **publish**. +6. Go to http://localhost:8001/admin/sites/ and select **localhost**. +7. Select the new "Home Page" as the **root page** and **save**. +8. The website should become available at http://localhost:8081 +9. To destroy everything (i.e. start the database from scratch) run `docker-compose down`. #### Backend details @@ -50,7 +73,7 @@ yarn run build yarn run dist ``` -Other available commands cand be viewed with +Other available commands can be viewed with ```sh yarn run From 3c89849f24e54b52ec46a90eb2bca9d1f77edc7d Mon Sep 17 00:00:00 2001 From: Kesara Rathnayake Date: Wed, 15 Sep 2021 13:20:05 +1200 Subject: [PATCH 02/25] Adds database migration steps to dev docker image --- docker/Dockerfile | 5 +++-- docker/init-dev.sh | 5 +++++ install.md | 23 +++++++++-------------- 3 files changed, 17 insertions(+), 16 deletions(-) create mode 100755 docker/init-dev.sh diff --git a/docker/Dockerfile b/docker/Dockerfile index 92b2cba8..00b60fd5 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -86,6 +86,7 @@ CMD ["/usr/local/bin/gunicorn", "--config", "/app/docker/gunicorn.py", "ietf.wsg # development stage FROM base as app-dev ENV DJANGO_SETTINGS_MODULE ietf.settings.docker.dev +COPY docker/init-dev.sh /app/docker/ ADD https://raw.githubusercontent.com/mrako/wait-for/d9699cb9fe8a4622f05c4ee32adf2fd93239d005/wait-for /usr/local/bin/ USER root @@ -93,6 +94,6 @@ RUN chmod +rx /usr/local/bin/wait-for USER www # wait-for script waits until the database is ready and reachable -# and then it runs the actual process +# and then it runs the migrations and actual process ENTRYPOINT ["wait-for", "database:5432", "--"] -CMD ["/usr/local/bin/gunicorn", "--config", "/app/docker/gunicorn.py", "--reload", "ietf.wsgi" ] +CMD ["/app/docker/init-dev.sh"] diff --git a/docker/init-dev.sh b/docker/init-dev.sh new file mode 100755 index 00000000..6dc01bbf --- /dev/null +++ b/docker/init-dev.sh @@ -0,0 +1,5 @@ +#!/bin/sh -e + +python /app/manage.py migrate --no-input +python /app/manage.py createcachetable +/usr/local/bin/gunicorn --config /app/docker/gunicorn.py --reload ietf.wsgi diff --git a/install.md b/install.md index 8632ab91..8e1f41d0 100644 --- a/install.md +++ b/install.md @@ -14,8 +14,8 @@ So the only requirement to run it locally is a recent version of Docker with doc 1. Obtain a recent database dump with name like `ietfa.torchbox.*.gz` and place in `docker/database/` directory. Otherwise, it will start with a fresh database. 2. Obtain and unarchive media files into `media/` folder. 3. Run `docker-compose up`. It will build and start the frontend builder (`yarn run start`) and the backend (`python manage.py runserver` analog), along with a Postgresql database. The first run will take a while because the database dump needs to be restored. -4. After the frontend compilation finishes, the website should become available at http://localhost:8081 -5. Create a super user on **Python application** docker instance to access http://localhost:8081/admin +4. After the frontend compilation finishes, the website should become available at http://localhost:8001 +5. Create a super user on **Python application** docker instance to access http://localhost:8001/admin ``` docker exec -ti wagtail_website_application_1 python manage.py createsuperuser ``` @@ -24,21 +24,16 @@ docker exec -ti wagtail_website_application_1 python manage.py createsuperuser #### How to run (without a database dump) 1. Run `docker-compose up`. It will build and start the frontend builder (`yarn run start`) and the backend (`python manage.py runserver` analog), along with a Postgresql database. The first run will take a while because the database dump needs to be restored. -2. After **Python application** docker instance is started, run database migrations. -``` -docker exec -ti wagtail_website_application_1 python manage.py migrate -docker exec -ti wagtail_website_application_1 python manage.py createcachetable -``` -3. Create an admin user +2. Create an admin user ``` docker exec -ti wagtail_website_application_1 python manage.py createsuperuser ``` -4. Log into http://localhost:8081/admin -5. Create a new "Home Page" (page type must be `Home Page`) and **publish**. -6. Go to http://localhost:8001/admin/sites/ and select **localhost**. -7. Select the new "Home Page" as the **root page** and **save**. -8. The website should become available at http://localhost:8081 -9. To destroy everything (i.e. start the database from scratch) run `docker-compose down`. +3. Log into http://localhost:8001/admin +4. Create a new "Home Page" (page type must be `Home Page`) and **publish**. +5. Go to http://localhost:8001/admin/sites/ and select **localhost**. +6. Select the new "Home Page" as the **root page** and **save**. +7. The website should become available at http://localhost:8001 +8. To destroy everything (i.e. start the database from scratch) run `docker-compose down`. #### Backend details From 8553682d0077816f958d23ad6549cf9251961959 Mon Sep 17 00:00:00 2001 From: Lars Eggert Date: Wed, 8 Sep 2021 09:46:58 +0300 Subject: [PATCH 03/25] Enable all editing tools This enables additional formatting options for RichTextField and RichTextBlock, such as preformatted/code sections or blockquotes. Also see: https://docs.wagtail.io/en/stable/reference/settings.html?highlight=RichTextBlock#id15 https://docs.wagtail.io/en/stable/advanced_topics/customisation/page_editing_interface.html#limiting-features-in-a-rich-text-field --- ietf/settings/base.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/ietf/settings/base.py b/ietf/settings/base.py index 5ee1ea9e..a4ec7a06 100644 --- a/ietf/settings/base.py +++ b/ietf/settings/base.py @@ -181,6 +181,21 @@ WAGTAIL_USAGE_COUNT_ENABLED = True WAGTAILIMAGES_IMAGE_MODEL = "images.IETFImage" WAGTAILDOCS_DOCUMENT_MODEL = "documents.IetfDocument" +WAGTAILADMIN_RICH_TEXT_EDITORS = { + 'default': { + 'WIDGET': 'wagtail.admin.rich_text.DraftailRichTextArea', + 'OPTIONS': { + 'features': [ + 'h2', 'h3', 'h4', 'h5', 'h6', + 'ol', 'ul', + 'bold', 'italic', + 'superscript', 'subscript', 'strikethrough', + 'hr', 'link', 'document-link', + 'image', 'embed', + 'code', 'blockquote'] + } + } +} # Application-wide settings DATATRACKER_URI = "https://datatracker.ietf.org" From 9e9f552f22d8ab945554303341e0ece5c37beaad Mon Sep 17 00:00:00 2001 From: Lars Eggert Date: Tue, 14 Sep 2021 13:24:41 +0300 Subject: [PATCH 04/25] Add wagtailmarkdown. Fixes #123. --- ietf/settings/base.py | 5 +++++ ietf/utils/blocks.py | 2 ++ package.json | 3 ++- requirements.txt | 2 ++ yarn.lock | 5 +++++ 5 files changed, 16 insertions(+), 1 deletion(-) diff --git a/ietf/settings/base.py b/ietf/settings/base.py index a4ec7a06..18ba7ea0 100644 --- a/ietf/settings/base.py +++ b/ietf/settings/base.py @@ -71,6 +71,7 @@ "django.contrib.admindocs", "mod_wsgi.server", "analytical", + "wagtailmarkdown", ) MIDDLEWARE = [ @@ -175,6 +176,10 @@ } +WAGTAILMARKDOWN = { + "autodownload_fontawesome": True, +} + # Wagtail settings WAGTAIL_SITE_NAME = "ietf" diff --git a/ietf/utils/blocks.py b/ietf/utils/blocks.py index a9b010e2..f5acc325 100644 --- a/ietf/utils/blocks.py +++ b/ietf/utils/blocks.py @@ -4,12 +4,14 @@ from wagtail.contrib.table_block.blocks import TableBlock from wagtail.images.blocks import ImageChooserBlock from wagtail.embeds.blocks import EmbedBlock +from wagtailmarkdown.blocks import MarkdownBlock class StandardBlock(StreamBlock): heading = CharBlock(icon="title") paragraph = RichTextBlock(icon="pilcrow") image = ImageChooserBlock(icon="image", template='includes/imageblock.html') + markdown = MarkdownBlock(icon="code") embed = EmbedBlock(icon="code") raw_html = RawHTMLBlock(icon="placeholder") table = TableBlock(table_options={'renderer': 'html'}, template="includes/tableblock.html") diff --git a/package.json b/package.json index cf47170f..c6e1eeb5 100644 --- a/package.json +++ b/package.json @@ -42,6 +42,7 @@ "dependencies": { "@axe-core/puppeteer": "^4.2.2", "@buzuli/json": "^2.1.0", + "@fortawesome/fontawesome-free": "^5.15.4", "autoprefixer": "^10.2.6", "babel-loader": "^8.2.2", "bootstrap": "^4.6.0", @@ -50,11 +51,11 @@ "ionicons": "^2", "jquery": "^3.6.0", "jquery-ui": "^1.12.1", - "sass": "^1.35.1", "normalize.css": "^8.0.1", "popper.js": "^1.16.1-LTS", "postcss-normalize": "^10.0.0", "puppeteer": "^10.1.0", + "sass": "^1.35.1", "ts-node": "^10.0.0", "typescript": "^4.3.4" } diff --git a/requirements.txt b/requirements.txt index 18b90426..b0023720 100644 --- a/requirements.txt +++ b/requirements.txt @@ -12,6 +12,8 @@ django_compressor>=2.2 mod_wsgi>=4.6 django_analytical>=2.5 tqdm>=3.7.0 +wagtail-markdown +Pygments # env var configuration typed-environment-configuration>=0.1.3,<0.2 diff --git a/yarn.lock b/yarn.lock index 960b45da..cdc97dce 100644 --- a/yarn.lock +++ b/yarn.lock @@ -947,6 +947,11 @@ minimatch "^3.0.4" strip-json-comments "^3.1.1" +"@fortawesome/fontawesome-free@^5.15.4": + version "5.15.4" + resolved "https://registry.yarnpkg.com/@fortawesome/fontawesome-free/-/fontawesome-free-5.15.4.tgz#ecda5712b61ac852c760d8b3c79c96adca5554e5" + integrity sha512-eYm8vijH/hpzr/6/1CJ/V/Eb1xQFW2nnUKArb3z+yUWv7HTwj6M7SP957oMjfZjAHU6qpoNc2wQvIxBLWYa/Jg== + "@nodelib/fs.scandir@2.1.5": version "2.1.5" resolved "https://registry.yarnpkg.com/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz#7619c2eb21b25483f6d167548b4cfd5a7488c3d5" From a9770f5fd914500b0622072c8af78fb04e107169 Mon Sep 17 00:00:00 2001 From: Lars Eggert Date: Tue, 14 Sep 2021 13:26:38 +0300 Subject: [PATCH 05/25] Don't need fontawesome, since we're autoloading it --- package.json | 1 - yarn.lock | 5 ----- 2 files changed, 6 deletions(-) diff --git a/package.json b/package.json index c6e1eeb5..515bcbea 100644 --- a/package.json +++ b/package.json @@ -42,7 +42,6 @@ "dependencies": { "@axe-core/puppeteer": "^4.2.2", "@buzuli/json": "^2.1.0", - "@fortawesome/fontawesome-free": "^5.15.4", "autoprefixer": "^10.2.6", "babel-loader": "^8.2.2", "bootstrap": "^4.6.0", diff --git a/yarn.lock b/yarn.lock index cdc97dce..960b45da 100644 --- a/yarn.lock +++ b/yarn.lock @@ -947,11 +947,6 @@ minimatch "^3.0.4" strip-json-comments "^3.1.1" -"@fortawesome/fontawesome-free@^5.15.4": - version "5.15.4" - resolved "https://registry.yarnpkg.com/@fortawesome/fontawesome-free/-/fontawesome-free-5.15.4.tgz#ecda5712b61ac852c760d8b3c79c96adca5554e5" - integrity sha512-eYm8vijH/hpzr/6/1CJ/V/Eb1xQFW2nnUKArb3z+yUWv7HTwj6M7SP957oMjfZjAHU6qpoNc2wQvIxBLWYa/Jg== - "@nodelib/fs.scandir@2.1.5": version "2.1.5" resolved "https://registry.yarnpkg.com/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz#7619c2eb21b25483f6d167548b4cfd5a7488c3d5" From f3f708687607157947a7607b6c334a3ed259dbc3 Mon Sep 17 00:00:00 2001 From: Lars Eggert Date: Tue, 14 Sep 2021 13:27:43 +0300 Subject: [PATCH 06/25] Add comment about wanting more customization --- ietf/settings/base.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ietf/settings/base.py b/ietf/settings/base.py index 18ba7ea0..bd8f0dfb 100644 --- a/ietf/settings/base.py +++ b/ietf/settings/base.py @@ -177,6 +177,8 @@ WAGTAILMARKDOWN = { + # We probably want to configure this more, see + # https://github.com/torchbox/wagtail-markdown "autodownload_fontawesome": True, } From e1ddd706c71a531b84afe4fa2dc586f31a69ece9 Mon Sep 17 00:00:00 2001 From: Emilia Zapata Date: Wed, 8 Sep 2021 17:33:02 +1200 Subject: [PATCH 07/25] chore(yarn): upgrade packages And ensure everything transpiles without a hitch as well. --- ietf/static_src/js/init.js | 2 +- package.json | 4 ++-- yarn.lock | 18 +++++++++--------- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/ietf/static_src/js/init.js b/ietf/static_src/js/init.js index f431c39d..7c5f0226 100644 --- a/ietf/static_src/js/init.js +++ b/ietf/static_src/js/init.js @@ -1,6 +1,6 @@ import $ from 'jquery'; import 'jquery-ui/ui/widgets/datepicker'; -import Popper from 'popper.js'; +import * as Popper from '@popperjs/core'; import 'bootstrap'; diff --git a/package.json b/package.json index 515bcbea..610be1a3 100644 --- a/package.json +++ b/package.json @@ -42,16 +42,16 @@ "dependencies": { "@axe-core/puppeteer": "^4.2.2", "@buzuli/json": "^2.1.0", + "@popperjs/core": "^2.10.1", "autoprefixer": "^10.2.6", "babel-loader": "^8.2.2", - "bootstrap": "^4.6.0", + "bootstrap": "^5.1.1", "color-json": "^2.0.1", "cssnano": "^5.0.6", "ionicons": "^2", "jquery": "^3.6.0", "jquery-ui": "^1.12.1", "normalize.css": "^8.0.1", - "popper.js": "^1.16.1-LTS", "postcss-normalize": "^10.0.0", "puppeteer": "^10.1.0", "sass": "^1.35.1", diff --git a/yarn.lock b/yarn.lock index 960b45da..fdb75e84 100644 --- a/yarn.lock +++ b/yarn.lock @@ -968,6 +968,11 @@ "@nodelib/fs.scandir" "2.1.5" fastq "^1.6.0" +"@popperjs/core@^2.10.1": + version "2.10.1" + resolved "https://registry.yarnpkg.com/@popperjs/core/-/core-2.10.1.tgz#728ecd95ab207aab8a9a4e421f0422db329232be" + integrity sha512-HnUhk1Sy9IuKrxEMdIRCxpIqPw6BFsbYSEUO9p/hNw5sMld/+3OLMWQP80F8/db9qsv3qUjs7ZR5bS/R+iinXw== + "@trysound/sax@0.1.1": version "0.1.1" resolved "https://registry.yarnpkg.com/@trysound/sax/-/sax-0.1.1.tgz#3348564048e7a2d7398c935d466c0414ebb6a669" @@ -1623,10 +1628,10 @@ boolbase@^1.0.0: resolved "https://registry.yarnpkg.com/boolbase/-/boolbase-1.0.0.tgz#68dff5fbe60c51eb37725ea9e3ed310dcc1e776e" integrity sha1-aN/1++YMUes3cl6p4+0xDcwed24= -bootstrap@^4.6.0: - version "4.6.0" - resolved "https://registry.yarnpkg.com/bootstrap/-/bootstrap-4.6.0.tgz#97b9f29ac98f98dfa43bf7468262d84392552fd7" - integrity sha512-Io55IuQY3kydzHtbGvQya3H+KorS/M9rSNyfCGCg9WZ4pyT/lCxIlpJgG1GXW/PswzC84Tr2fBYi+7+jFVQQBw== +bootstrap@^5.1.1: + version "5.1.1" + resolved "https://registry.yarnpkg.com/bootstrap/-/bootstrap-5.1.1.tgz#9d6eed81e08feaccedf3adaca51fe4b73a2871df" + integrity sha512-/jUa4sSuDZWlDLQ1gwQQR8uoYSvLJzDd8m5o6bPKh3asLAMYVZKdRCjb1joUd5WXf0WwCNzd2EjwQQhupou0dA== brace-expansion@^1.1.7: version "1.1.11" @@ -3678,11 +3683,6 @@ pkg-up@^2.0.0: dependencies: find-up "^2.1.0" -popper.js@^1.16.1-LTS: - version "1.16.1" - resolved "https://registry.yarnpkg.com/popper.js/-/popper.js-1.16.1.tgz#2a223cb3dc7b6213d740e40372be40de43e65b1b" - integrity sha512-Wb4p1J4zyFTbM+u6WuO4XstYx4Ky9Cewe4DWrel7B0w6VVICvPwdOpotjzcf6eD8TsckVnIMNONQyPIUFOUbCQ== - postcss-browser-comments@^4: version "4.0.0" resolved "https://registry.yarnpkg.com/postcss-browser-comments/-/postcss-browser-comments-4.0.0.tgz#bcfc86134df5807f5d3c0eefa191d42136b5e72a" From 57dec6ad1146a3b44074a57314832187363e0866 Mon Sep 17 00:00:00 2001 From: Emilia Zapata Date: Mon, 13 Sep 2021 13:21:40 +1200 Subject: [PATCH 08/25] fix(bs): rename some css classes and add some According to BS5 breaking changes --- ietf/blog/templates/blog/blog_page.html | 6 +++--- ietf/blog/templates/includes/blog_filters.html | 12 ++++++------ ietf/events/templates/events/event_listing_page.html | 2 +- ietf/events/templates/events/event_page.html | 2 +- ietf/forms/templates/includes/form_field.html | 2 +- ietf/glossary/templates/glossary/glossary_page.html | 2 +- ietf/home/templates/includes/home_event.html | 2 +- .../iesg_statement/iesg_statement_page.html | 6 +++--- ietf/search/templates/search/search.html | 2 +- ietf/snippets/templates/snippets/area_charter.html | 2 +- ietf/snippets/templates/snippets/working_group.html | 2 +- ietf/standard/templates/standard/standard_page.html | 2 +- ietf/templates/includes/highlight.html | 2 +- ietf/topics/templates/topics/primary_topic_page.html | 2 +- 14 files changed, 23 insertions(+), 23 deletions(-) diff --git a/ietf/blog/templates/blog/blog_page.html b/ietf/blog/templates/blog/blog_page.html index b12de2a6..34779038 100644 --- a/ietf/blog/templates/blog/blog_page.html +++ b/ietf/blog/templates/blog/blog_page.html @@ -10,7 +10,7 @@
-
+
-