DataBase Changes

The database is in mongoDb and consist of the collector names as the db name. There are 2 DBs 1)Config DB 2)Tweets DB

Config DB

The Config db consist of 2 collection names 1)config 2)extra_parametersvalue

The config collection stores the name of all the collector terms(twitter names of the people whom we follow) of the website with the Tweets DB name and the name of the consumer key,error code Datetime stamp etc.The Config DB is created using the function def set_collector_detail in models.py.The unique term id of each and every term is stored in the config db,config collector

The extra_parametersvalue stores the details of all the collector term properties which includes computed values like total_urls,total_tweets and other parameters. The extra_parametersvalue collector is created using the function def Get_Set_OtherParameters in models.py.

Working Steps

1. User click on a term on the website
2. Ajax call is made to the views.py
3. From views.py the call is made to get_term_details in models.py
4. From get_term_details the user information is obtained from the project DB
5. To get the information about tweets we call the function Get_Set_OtherParameters
6. First a check is made to see that for that particular term does an collector called extra_parameter value exsist,if not it is created.
7. If the extra_parameter exist
   1. Check to see the total values count in extra_parameter is same as the count value in DB
   2. If values are same fetch values from extra_parameter and display on screen
   3. If values not same re-compute the values and re-store in DB
8. If the extra_parametervalue DOES NOT exist 8.1)Create the collection and compute the values and store in DB

If you add a new value to display or change the logic for counting the entries,you need not go to each and every term and drop the extra_parameter value,simply run this command in mongo db.timestamp.update({status:1},{timestamp:new Date(),status:1}),because this command will update the timestamp value in the create_timestamp db and now when you click to display stats the date of the timestamp is compared to the date stored in the db if not same values are updated and the recent timestamp is stored.

Tweets DB There are 2 kinds of tweets and we display the most recent 200 tweets based on the most recent dates.

1)Tweets tweeted by the person himself eg:Person A tweets "Hello world " on his twitter account

2)Tweets tweeted to a person from other eg: Person B tweets "@A bad day"

3)All the tweets have an id associated "https://twitter.com/RogueNASA/status/824693805110722560" here the id is "824693805110722560" this id has to be unique and tweets must unqiue id must only be stored in the mongo DB

4)To only add unique id tweets an index with property set for true ids has been created and added in the code whenever a new collector is created ,the code is present in models.py def set_collector_detail, dbnamevalue.tweets.create_index('id_str',unique=True)