GStudioLITE : Optimization of GStudio by backend caching with revalidation.

[ansuman-eng]

The entirety of our group's work in the Summer of 2018 as a part of BITS Pilani's Practice School-I. The project aimed at optimizing GStudio by setting up a backend Filesystem-based caching. The trade-off of dynamic nature was kept to as minimal as possible. A few utilitarian scripts have also been included.

We stood up the back-end cache by configuring the settings.py file as in the image below. The relative order of MIDDLEWARE is critical. Do not change it. To be on the safe side, the first MIDDLEWARE should be placed first in the list of actual MIDDLEWARES - as should the last one be kept at the end.

A path on File-system must be provided to the Cache where-in the responses will be cached. Do not use the same path as in the image.
MAX_ENTRIES has to be adjusted according to the size of the website. TIMEOUT is in seconds and doesn't really matter at this point since the max_age parameter of cache_control (see below) has a higher precedence. It has just been added for the sake of completion.

A Selenium-based web-crawler was designed to parse through all the pages of any web-site based on GStudio. It's aim is to request multiple resources so that the responses to them are cached in the File-System in a one-time manner. Dependencies for the same have been added in the requirements.txt file.

Cache-revalidation was also set in place by using cache_control decorators. Any view which rendered a response was provided with that decorator. This forces the cache to revalidate the stale responses before sending them over to the client side. It is a convenient and robust method of cache-revalidation which can scale efficiently. The mechanism of cache revalidation is as follows.

Finally there is a GStudioLITE_README under doc/developers which contains a more extensive documentation and synopsis of the project.

Please review the changes.

[kedar2a]

@ansuman1998 , can you specify how to go ahead after taking this pull locally?

1. Overriding one of the settings.py var in local_settings.py (if any)
2. Updating requirements?
3. Which script to run? How to run the script?

Example PR description for reference:

• Progress-Report and Group-Analytics reconfigured #1946
• Provision to calculate Time spent(in seconds) by user on activity-detail  #1932
• Added new script to logout all users along with their buddies #1797

[ansuman-eng]

1. Whatever differences in settings.py is found, can be put in localsettings for overriding. Include CACHES, some MIDDLEWARE (in proper order) and some variable for MIDDLEWARE arguments (ex, key-prefix)
2. requirements.txt has been updated
3. newparser.py can be run to cache the full site after setting up cache. Please use newparser.py in doc/deployer.Can be executed from terminal.