Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make BlockingClient send thread safe #97

Merged
merged 2 commits into from
Jan 22, 2025

Conversation

beenje
Copy link
Contributor

@beenje beenje commented Jan 22, 2025

At MAX IV we use the BlockingClient in several sardana controllers.
The same client is used in different methods to send commands. Sardana automatically runs some of those methods in a separate threads. We sometimes get an exception raised from the pandablocks library:

Dummy-38       WARNING  2025-01-21 14:35:45,549 pandabox_ctrl: pandabox_ctrl.StopOne(1) has failed
Traceback (most recent call last):
  File "/opt/conda/envs/sardana/lib/python3.11/site-packages/sardana_finest/ctrl/FinestPandaBoxTGCtrl.py", line 47, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/sardana/lib/python3.11/site-packages/sardana_finest/ctrl/FinestPandaBoxTGCtrl.py", line 207, in AbortOne
    self.pandabox.send(Put("BITS2.A", "0"))
  File "/opt/conda/envs/sardana/lib/python3.11/site-packages/pandablocks/blocking.py", line 103, in send
    assert cr[id(command)] is ..., "Already got response for {command}"
           ~~^^^^^^^^^^^^^
KeyError: 139893205968464

In this case we got a response from a command that was sent from a different thread.

We can fix that by setting a lock in our code but it would be much nicer to fix it in the library.

I added a lock in the send method to make it thread safe.

I also fixed in this MR a typo: f was missing for the "Already got response for {command}" f-string.

Using the same BlockingClient to send command wasn't thread safe.

Responses from commands sent from different threads could be mixed.
Avoid this issue by using a lock.
Missing f for f-string
@beenje
Copy link
Contributor Author

beenje commented Jan 22, 2025

pre-commit complained on some files I didn't touch. I let ruff fix the formatting on those. (I deleted that commit).

I see it failed again in CI.
I noticed ruff repo is set to local, meaning depending on the version you have installed locally, you will get different results. Is there any reason for that?

I think it would be better to use the official repo https://github.com/astral-sh/ruff-pre-commit and pin the version so every developers use the same.

I could fix that in a separate PR or in this one.

@coretl coretl requested a review from evalott100 January 22, 2025 12:20
Copy link
Contributor

@evalott100 evalott100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link

codecov bot commented Jan 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.20%. Comparing base (5fbf3d9) to head (3c33fc9).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #97   +/-   ##
=======================================
  Coverage   97.19%   97.20%           
=======================================
  Files          12       12           
  Lines        1212     1215    +3     
=======================================
+ Hits         1178     1181    +3     
  Misses         34       34           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@evalott100
Copy link
Contributor

I noticed ruff repo is set to local, meaning depending on the version you have installed locally, you will get different results. Is there any reason for that?

Linting, and CI, are changed upstream in the copier template.

The short answer is, until we investigate using uv lockfiles it'll be using the local version which can be pinned in a given project's pyproject.toml. We do this because of the constraints in the vscode ruff extension.

The locally installed ruff that tox -e pre-commit uses should be the same version
that CI installs. Are you getting different linting errors when you run tox -e pre-commit in your local virtual environment?

@evalott100
Copy link
Contributor

evalott100 commented Jan 22, 2025

Going to merge and make a new PR with the newer ruff version's changes.

@evalott100 evalott100 merged commit 71f1e50 into PandABlocks:master Jan 22, 2025
22 of 24 checks passed
@beenje
Copy link
Contributor Author

beenje commented Jan 22, 2025

The locally installed ruff that tox -e pre-commit uses should be the same version that CI installs. Are you getting different linting errors when you run tox -e pre-commit in your local virtual environment?

Thanks for the explanation. I usually don't use tox.
I use pre-commit in many projects and have it installed globally, not per project. I usually run pre-commit run -a directly or do pre-commit install and let git commit run it automatically.

Same for ruff. We don't install it in every project. We let pre-commit install it and bump the version used from time to time.
But it's true that I also have a global version installed that can be used in vscode.

The changes I had were the same as what you pushed in #98
I must have mixed up the email I got... Checking again the GitHub Actions, it did pass successfully with my changes. Sorry.

Anyway, thanks for merging!

Do you plan on making a new release soon?

@evalott100
Copy link
Contributor

evalott100 commented Jan 22, 2025

Do you plan on making a new release soon?

I think it's time for a patch release, will do now! Though might take rerunning copier for a fix to our release CI.

I use pre-commit in many projects and have it installed globally, not per project. I usually run pre-commit run -a directly or do pre-commit install and let git commit run it automatically.

Makes sense, we'll be investigating package managers with lockfiles. For now we're settling for consistency in tox -e pre-commit on a fresh venv.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants