Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database is locked (again) #113

Open
timronan opened this issue Jul 10, 2019 · 1 comment
Open

Database is locked (again) #113

timronan opened this issue Jul 10, 2019 · 1 comment

Comments

@timronan
Copy link
Contributor

timronan commented Jul 10, 2019

An instance of rover retrieve request.txt, attached to this issue, run on Python 3.6 has lead to a critical error sqlite3.OperationalError: data base is locked. The rover retrieve.log shows two stack traces that are leading to the data base is locked error.

 CRITICAL 2019-06-25 18:00:08,691: Traceback (most recent call last):
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/__init__.py", line 116, in main
     with ProcessManager(config):
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/process.py", line 48, in __enter__
     self._check_command(self._command)
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/process.py", line 87, in _check_command
     pid, command = self._current_command_inside_transaction()
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/process.py", line 69, in _current_command_inside_transaction
     self._db.execute('DELETE FROM rover_processes WHERE pid = ?', (pid,))
 sqlite3.OperationalError: database is locked
 CRITICAL 2019-06-25 19:47:33,962: Traceback (most recent call last):
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/__init__.py", line 117, in main
     execute(config.command, config)
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/__init__.py", line 96, in execute
     commands[command][0](config).run(config.args)
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/retrieve.py", line 213, in run
     return self.do_run(args, True, RETRIEVE)
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/retrieve.py", line 152, in do_run
     return self._fetch()
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/retrieve.py", line 180, in _fetch
     n_downloads = self._download_manager.download()
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/manager.py", line 882, in download
     self.step(quiet=False)
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/manager.py", line 854, in step
     self._update_stats()
   File "/.conda/envs/rover/lib/python3.6/site-packages/rover/manager.py", line 906, in _update_stats
     self._db.execute('DELETE FROM rover_download_stats', tuple())
 sqlite3.OperationalError: database is locked

Please note that both database is locked operations originate from a DELETE FROM sql command.
request.txt

@timronan
Copy link
Contributor Author

It seems like this database locking is associated with multiple open transactions that are all trying to write or read from the database at the same time. The busy timeout has been increased to 120 seconds. The busy timeout allows incoming SQL transactions to sleep while another transaction is interfacing with the database, the maximum amount of time that the transaction can sleep is set in seconds by the integer value above Busy Timeout.

A connection.commit() command also been added after the DELETE FROM statements. Committing the Delete FROM statements to the database should allow the database to unlock and be ready for another transaction commit .

The request.txt, above, has been run through this modified code with no locking database exceptions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant