Refactor keywords, clusters, non-quotes, and short suspensions

1.4

• Massive refactor to bring keywords to scientific standard
  • Implements a correct Keyword Extraction Algorithm (KEA)
  • Note: Earlier results should be distrusted
  • Tests KEA to check results are identical to match results set by Paul Rayson
  • Documents use and origin of the KEA
• Refactors the clusters
• Updates User Interface to increase modularity and add total counts
  • Makes clusters and keywords tables 100% wide to fix word wrap bug
  • Renames js and GET variables in clusters and keywords to more sane defaults
  • Adds right values for the keywords table (use, p_value, LL)
  • Enables sorting of keywords and clusters on rank
  • Updates book modal dialogue with latest corpus counts
  • Clusters and keywords now include total cluster count
  • Implements descriptions of the corpora based on the user's input
  • Provides a summary of the search settings before giving clusters and keywords tables
• Starts building help pages, About -> Help
• Fixes API code that was unreadable
• Fixes bug that returned 1grams when 5grams were asked
• Enables caching for keywords and clusters
  • Updates beaker cache version to handle args
• Fixes non-quote counts that missed non-quotes before the first quote
  • Correct mistake (before my time) in create_bookcountsjson
  • Update book counts now including total counts/corpus
  • Add note about the wrong q+nq counts
  • Adds selector with help from Rob, Cat, and Matt, but does not use it yet
• Adds consultant to the home page
• Implements concordance searches in short suspensions
• Implements 3 and 4 grams in short suspensions (both keywords and clusters)

Increase usability and documentation

• adds a DOI and a citable reference
• updates the docs
• fixes issues in the KWICgrouper
• implements toggle for concordance view in User Annotation
• boosts export function of annotation to 10000 lines
• enable 1-gram keywords
• adds vodcast
• enables users to add each others tags
• disables the registration view
• updates docs of the User Annotation

Implement user annotation

• adds a user classification and annotation module that is only accessible for users with specific credentials
• allows users to create custom, user-specific tags and add these to subsets
• uses a postgreSQL database to store subsets, tags, notes, users, and roles
• documents the creation of tags, advanced filtering of subsets, phrases searches, and exporting
• allows for user-annotated csv-exports of up to 2000 rows
• adds advanced filtering to the subsets
  • including filtering on book name, abbreviation, user, text, etc.
  • using different filtering types as 'contains', 'not contains', 'in list', etc.
• adds a quick search box and documents its functionality
• enables sorting of subsets
• adds pagination
• enables user and role management
• hedges off the user annotation from the public site
• implements a different design to highlight that this module is not read-only

Feature: Add 'patterns' module for advanced analysis

• adds a pattern module that allows advance concordance analysis (aka KWICgrouper)
• adds documentation next to the form
• enables sorting
• allows for csv export
• generates a clickable collocation table
• adds a basic concordance for subsets executed in the background

Encourage user interaction and improve user interface

• populates Github releases page with our releases and allow users to download each release
• fixes bug that prevented the chapter view to load, before it was hard coded
• moves AHRC, UoB, and UoN logos up on large screens (and down on mobile devices)
• adds a tweet button to the about, documentation, news, and release pages
• renames the code repository
• redesigns corners from rounded to rectangular corners and moves loading icon up
• moves feedback widget into template block
• makes it possible to run debugging on the flask dev server
• adds news/blog page, a releases page, and a dropdown information menu for impact

Move to the University of Birmingham, Centre for Corpus Research

• adds logo of the University of Birmingham (the new home of the CLiC project)
• adds Google analytics for the Birmingham server
• changes the contact in the about page to [email protected]

Fix analytics and license

• fixes a bug that deleted the Google Analytics javascript
• adds a licence to the code
• adds this changelog

Add subsets as a tool to study the text

• adds subsets as a way to navigate and explore quotes, suspensions, short suspensions, etc.
• boosts number of uwsgi processes
• adds a profiler
• activates 1-grams in keywords and clusters

Improve clusters and keywords

• moves clusters and keywords to jinja templates (speeds them up massively because
  there is less client side rendering)
• boosts number of uwsgi processes

Add logo

• adds AHRC logo