Wiki Upgrade

Eternal Wiki Upgrade Quest

This is the beginning of the wiki upgrade page....

Requirements

1. https by letsencrypt
2. debian 9
3. no snowflakes, all our (known) devops culture at nb is ansible
4. salt the passwords correctly

Chosen Technologies

1. Ansible
2. Debian 9
3. Caddy
4. Letsencrypt / Certbot

Ansible Roles

Noisebridge already has an ansible infrastructure repo, please join the Noisebridge github group and hack this: https://github.com/noisebridge/ansible/

1. Certbot - https://github.com/geerlingguy/ansible-role-certbot
2. Semantic Mediawiki - Mediawiki maintains some: https://phabricator.wikimedia.org/diffusion/1881/
   1. It may be useful to look at this one: https://github.com/yongxinL/ansible-mediawiki

Database Hacking

We need to upgrade a really old version of mediawiki and will likely need to futz with the database.

1. Migrate to postgresql?
   1. make a git repo to store migration sql and other sql used for massaging the database contents
2. Delete old passwords (md5 with no salt) and expire all user passwords?
   1. Write/publish public sql script to clean up the production database for public publishing
3. Clean the database of spam/cruft?
   1. publish these scripts to the same sql script repo

MediaWiki-Vagrant

This potentially is a good environment for testing and migrating the database: https://www.mediawiki.org/wiki/MediaWiki-Vagrant

(more OpenStack Semantic MediaWiki images: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS)

You'll need to follow this guide to get a dump of our database up and running: https://www.mediawiki.org/wiki/Manual:Moving_a_wiki

Password Rollover

Here is the mediawiki default user table data dictionary: https://www.mediawiki.org/wiki/Manual:User_table

We need to delete the password column and expire everyone's password. The old mediawiki only stores md5 hashed passwords with no salts, so we'll have to delete the whole thing

user_password_expires

MediaWiki version: ≥ 1.23

Date when user's password expires; null for no expiration date. Can also be set manually by calling User->expirePassword().

Noisebridge Wiki Database Fixing w/ Kevin 2017.12.23

This is a log of what we did to a local copy of Noisebridge wiki database. These changes were not propogated to the live/production Noisebridge MediaWiki instance.

Things We Deleted

1. 1. purged password hashes - noisebridge_mediawikiuser.user_password all set to (empty string)
      1. update set <column>=;
      2. purged user emails - noisebridge_mediawikiuser.user_email all set to (empty string)
         1. update
      set <column>=;

      
      

      1. 1. purged noisebridge_mediawikiipblocks.ipb_address - replaced the composite primary key column with a sequence as follows: https://stackoverflow.com/questions/2643371/how-to-renumber-primary-index
      1. 1. truncated user watchlists (these are private)
            1. truncate table noisebridge_mediawikiwatchlist;
      1. 1. truncated all protected tables (these might give private info on who protected the page?)
            1. truncate table noisebridge_mediawikiprotected_titles;

      Important How-tos and Gotchas

      • MediaWiki upgrade for NB
        • Follow directions for setting up local Vagrant instance on your development machine:
          • Install vagrant and virtualbox (OS X: http://sourabhbajaj.com/mac-setup/Vagrant/README.html)
          • $ cd vagrant
          • $ ./setup.sh
          • $ vagrant up
        • How to copy files into virtualbox instance
          • Find your eth1 ip address:
            • $ /sbin/ifconfig # and remember that as MW_IP_ADDR
            • $ scp -i .vagrant/machines/default/virtualbox/private_key YOUR_FILE_HERE vagrant@MW_IP_ADDR:
        • Noisebridge's MediaWiki database was configured with a tablename prefix of "noisebridge_mediawiki"
          • Change this in /vagrant/LocalSettings.php:

            $wgDBprefix = "noisebridge_mediawiki";

        • Use the default database name 'wiki' - do not be clever and configure a different name, because `wiki` is a baked-in default somewhere in vagrant's puppet scripts, or maybe in mediawiki's CLI scripts, but either way IT CANNOT BE OVERRIDDEN IN LocalSettings.php or DefaultSettings.php, or --conf, or the 'dbname-prefixname' CLI parameter
        • You may need to drop and create the MySQL database 'wiki' to eliminate the default MediaWiki 1.31 structure, to avoid problems overwriting it with our old MW 1.19 schema mysqldump

      $ mysql

      > drop database wiki;

      > create database wiki;

      > ^D

      $ mysql wiki < the-uncompressed-mysqldump-file-you-were-provided-with.sql

      You may safely ignore messages like "Warning (Code 1300): Invalid utf8 character string: 'BD5A6D'"

      • • Now run the upgrade script:

      $ cd /vagrant/mediawiki/maintenance/

      $ mwscript update.php

      If all goes as planned you'll see a bunch of advisory progress messages such as:

      Your composer.lock file is up to date with current dependencies!

      Going to run database updates for wiki-noisebridge_mediawiki

      Depending on the size of your database this may take a while!

      Abort with control-c in the next five seconds (skip this countdown with --quick) ... 0

      Turning off Content Handler DB fields for this part of upgrade.

      ...have ipb_id field in ipblocks table.

      ...have ipb_expiry field in ipblocks table.

      ...already have interwiki table

      ...indexes seem up to 20031107 standards.

      ...have rc_type field in recentchanges table.

      ...index new_name_timestamp already set on recentchanges table.

      ...have user_real_name field in user table.

      ...

      ...doing rev_id from 2001 to 2200

      ...doing rev_id from 2201 to 2400

      ...doing rev_id from 2401 to 2600

      ...doing rev_id from 2601 to 2800

      ...doing rev_id from 2801 to 3000

      ...checking 40 revisions for IP edits that need copying, between rev_ids 20100 and 20300

      ...checking 49 revisions for IP edits that need copying, between rev_ids 20301 and 20501

      ...checking 74 revisions for IP edits that need copying, between rev_ids 20502 and 20702

      ...checking 48 revisions for IP edits that need copying, between rev_ids 20703 and 20903

      ...checking 41 revisions for IP edits that need copying, between rev_ids 20904 and 21104

      ...et cetera.

      • • At this point you may be able to visit the Noisebridge site on your dev machine by going to:

      http://127.0.0.1:8080/wiki/Noisebridge

      • • If this doesn't work it's likely some step was omitted from this doc, in which case ask Kevin or Trent for support.

      FUTURE Database Hacking Plans

      FUTURE ACTION: After Upgrading we can expire everyone's password

      details here: https://noisebridge.net/wiki/Wiki_Upgrade and here: https://www.mediawiki.org/wiki/Manual:User_table#user_password_expires

      FUTURE ACTIONS

      1. reduce permissions for inactive moderators, admins, etc, to "user".
         1. really drop everyone and add current people back, to make it an intentional process
         2. be very restrictive about CURRENT active users