supersede: duplicate package in rings can cause failure in middle of non-atomic operation

[nilxam]

llvm4 SR#532127(https://build.opensuse.org/request/show/532127) has been staged in Staging M, it was superseded by SR#533345 , staging-bot handles it in a strange way

• the meta is erased - osc rdiff -M -c 1127 openSUSE:Factory:Staging:M
• the package still was there in M - osc log openSUSE:Factory:Staging:M/llvm4

r1 | dimstar_suse | 2017-10-10 21:56:21 | 6bb682d548e680b266ea913306f19727 | unknown |

[jberry-suse]

From journal:

sudo journalctl -u osrt-staging-bot-regular@openSUSE:Factory --since '-18 hours' --until '-17 hours'

Traceback (most recent call last):
  File "/usr/bin/osc", line 41, in <module>
    r = babysitter.run(osccli)
  File "/usr/lib/python2.7/site-packages/osc/babysitter.py", line 61, in run
    return prg.main(argv)
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 344, in main
    return self.cmd(args)
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 367, in cmd
    retval = self.onecmd(argv)
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 501, in onecmd
    return self._dispatch_cmd(handler, argv)
  File "/usr/lib/python2.7/site-packages/osc/cmdln.py", line 1232, in _dispatch_cmd
    return handler(argv[0], opts, *args)
  File "/usr/lib/osc-plugins/osc-staging.py", line 592, in do_staging
    ListCommand(api).perform(supersede=opts.supersede)
  File "/usr/lib/osc-plugins/osclib/list_command.py", line 26, in perform
    SupersedeCommand(self.api).perform()
  File "/usr/lib/osc-plugins/osclib/supersede_command.py", line 15, in perform
    for stage_info, code, request in self.api.dispatch_open_requests(requests):
  File "/usr/lib/osc-plugins/osclib/stagingapi.py", line 684, in dispatch_open_requests
    stage_info, code = self.update_superseded_request(rq, target_requests)
  File "/usr/lib/osc-plugins/osclib/stagingapi.py", line 610, in update_superseded_request
    review='declined')
  File "/usr/lib/osc-plugins/osclib/stagingapi.py", line 845, in rm_from_prj
    project = self.map_ring_package_to_subject(project, package)
  File "/usr/lib/osc-plugins/osclib/stagingapi.py", line 1125, in map_ring_package_to_subject
    if self.ring_packages.get(pkg) and self.rings.index(self.ring_packages.get(pkg)) == 2 and not project.endswith(':DVD'):
  File "/usr/lib/osc-plugins/osclib/stagingapi.py", line 110, in ring_packages
    self._ring_packages = self._generate_ring_packages()
  File "/usr/lib/osc-plugins/osclib/stagingapi.py", line 203, in _generate_ring_packages
    raise Exception(msg.format(pkg, ret[pkg], prj))
Exception: python-flake8 is defined in two projects (openSUSE:Factory:Rings:1-MinimalX and openSUSE:Factory:Rings:2-TestDVD)

[jberry-suse]

Relevant lines seem to explain the behavior.

https://github.com/openSUSE/osc-plugin-factory/blob/a2ba667f262607826131cace758696c4751c40fd/osclib/stagingapi.py#L844-L845

Not sure what the proper way to handle this would be. Really we want the operation to be atomic, but no simple way of doing that. Could map before, but that seems somewhat arbitrary. Likely the best method is to ensure that when moving things between rings the package if first removed before added to new ring to ensure there are not duplicates.

I am inclined to close this since any of the code could crap out in the middle and no rollback of partial change...like API transactions with commit.

[jberry-suse]

Feel free to re-open if you believe something more should be done. The workflow should be reflect in #848.