Installation / Configuration / Production Deployment

My setting

This documentation explain how I setup my last wikibase configuration. Install, configure and migrate the data.

This documentation is a good start to put a wikibase in production with the docker-compile.yml.

• We have 1 Ip
• A 12G, 1 To HD and a Xeon with 4 cores
• A Debian 9.11
• A Docker 5:19.03.5
• Nginx 1.10.3
• My favorite text editor emacs
• I am also using Portainer to have a view on all docker container, volumes, images network ...
• Hostnames:
  • query.exemple.com (wdqs)
  • qs.exemple.com (QuickStatements)
  • wiki.exemple.com (wiki)
• All SSL certificates i need for those 3 hostnames.
• 1 sql dump of our previous installation and the content or /var/www/html/images on our previous Wikibase installation.

Basic installation

I am starting by the git clone of all the git folder of Wikibase.

git clone https://github.com/wmde/wikibase-docker.git
cd wikibase-docker
## My scripts and configuration files are on this git with script and config
git clone  https://github.com/lordcyfer/my-wikibase.git
cp -r wikibase-config/config ./
cp -r wikibase-config/script ./
mv docker-compose.yml docker-compose.yml-orig
cp config/docker-compose.yml ./
cp config/LocalSettings.php .

Now we have to do the setting of docker-compile.yml to match our hostnames and admin basic settings. In Lo calSettings.php we need to adjust the mail configuration and the setting for authentication. You can find this file in our git config/docker-compile.yml and config/LocalSettings.php

For the keys of our authentication between Wikibase and QuickStatements services we need 2 files and an entrance in our db that must have to match

Our qs-oauth.json and oauth.ini are the config folder and match our SQL dump entrance for the keys.

We are ready for the installation.

docker-compose pull
docker-compose up -d

We will now adjust the setting with our extensions and our settings.

./script/install_personal_data.sh

This script install a set of extensions that we are using.

• Widgets
• Gadgets
• Scibunto
• EmbedVideo
• MobileFrontend
• ConfirmEdit
• Nuke
• DeleteBatch
• SpamBlacklist
• ConfirmAccount
• Echo
• CodeEditor
• SyntaxHighlight
• WikibaseQualityConstraints

Basic query test:

SELECT * WHERE { ?s ?p ?o. }

Docker compose

You need to learn how to use and understand how docker work.

nginx configuration

We have 1 Ip and certificates via Let's encrypt. Our Setting is a proxy from our real network to the port we'll use in our docker configuration.

Our docker network is configure to :

• Provide the wikidata http service on port 8181
• Provide the wdqs service (wdqs-frontend) on port 8282
• Provide the quickstatements service on port 9191

This configuration open 443 with SSL and forward the request to the ports (8181, 8282 and 9191)

Here is the nginx configuration for wdqs you just need to add the setting for wikidata and quickstatements (You replace server_name and the port)

server {
       listen 80;
       server_name query.exemple.com;

       return 301 https://$server_name$request_uri;
       }

server {
       listen 443;
       server_name query.exemple.com;
       ssl on;
       ssl_certificate     /etc/letsencrypt/live/query.exemple.com/fullchain.pem;
       ssl_certificate_key /etc/letsencrypt/live/query.exemple.com/privkey.pem;
       ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
       ssl_ciphers         HIGH:!aNULL:!MD5;  
       access_log /var/log/nginx/query.exemple.com.log;
       location / {
        proxy_pass http://127.0.0.1:8282;
  	    proxy_set_header Host $host;
		    proxy_set_header X-Real-IP $remote_addr;
		    proxy_set_header X-forwarded-host $host;
		    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		    proxy_set_header X-Forwarded-Proto $scheme;
      }
}

Backup

I put my backup script is ./scripts/

cd data_backup
docker exec wikibase-docker_mysql_1 mysqldump -u wikiuser -psqlpass my_wiki > backup.sql
docker cp wikibase-docker_mysql_1:backup.sql ./
docker cp wikibase-docker_wikibase_1:/var/www/html/ ./
docker cp -a wikibase-docker_wdqs_1:/wdqs/data/data.jnl ./
docker cp

Restoring

TO TEST

cd data_backup
cat backup.sql | docker exec wikibase-docker_mysql_1 mysql -u wikiuser -psqlpass my_wiki
docker exec cp -a html wikibase-docker-dev_wikibase_1:/var/www/
docker cp ./data.jnl wikibase-docker_wdqs_1:/wdqs/data/data.jnl

New configuration

For Wikibase Repository and Wikibase Client

cd extensions/Wikibase
php repo/maintenance/rebuildItemsPerSite.php
php client/maintenance/populateInterwiki.php

Prefix test

SELECT ?p ?o WHERE {
   pdio:Q1 ?p ?o.
}

Bugs

wdqs-updater_1

Sometimes we reach a situation with a problems on date / wikibase / wdqs_pers :

$ docker-compose ps | grep Exit
wikibase-docker-dev_wdqs-updater_1  /entrypoint.sh /runUpdate.sh Exit 1

The only way to repair that is too reset EVERYTHING.

autocomplete in the query service uses wrong prefix

guzzlehttp/streams

on _wikibase_1 /var/www/html/composer.json change

"guzzlehttp/streams": "^3.0"

to

"guzzlehttp/streams": "3.0.0"

Useful links

https://www.mediawiki.org/wiki/Wikibase

https://phabricator.wikimedia.org/

and the real Bible :

https://addshore.com/tag/wikibase/

Update of wikibase configuration

docker cp wikibase-docker_wikibase_1:/var/www/html/w/LocalSettings.php ./
emacs LocalSettings.php
docker cp LocalSettings.php wikibase-docker_wikibase_1:/var/www/html/w/
docker exec -ti wikibase-docker_wikibase_1  php maintenance/update.php

wgDebugLogGroups

cp images/LogoPersonal.png /var/www/html/resources/assets/wiki.png