Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] feat: add PASS_CONTEXTVARS #582

Conversation

r1b
Copy link
Contributor

@r1b r1b commented Jan 19, 2021

Reference issue

#578

Tasks

  • Unit tests added that reproduce issue or prove feature is working
  • Fix or feature added
  • Documentation and examples updated (if relevant)
  • Unit tests passing and coverage at 100% after adding fix/feature
  • Apps updated and tested (if relevant)

To expand on #578:

We'd like to be able to capture pynetdicom logs and filter for "all logs produced by associations in a particular calling thread". For example, if we're running a threaded storage scp we'd like to capture log output and filter for "all logs produced by a particular request thread".

Python stamps LogRecords with a threadId but pynetdicom spawns multiple threads over the course of handling an association. Some logs will be produced in the calling thread, some in the DUL thread(s) and some in the Association thread(s).

Python does not have a way to define "thread groups" nor does it maintain a reference to the calling thread in spawned threads. We can hack around those things (e.g: grouping threads by name, subclassing Thread) but in the future, pynetdicom might not use threads for concurrency at all #527. We want a forward-looking solution that will work for concurrency models that might re-use threads. We'd also like to make minimal changes to pynetdicom, so avoiding things like e.g passing logger instances around or adding additional log handlers / filtering logic there.

We think contextvars can achieve the goal here. We can attach logger filters that refer to ContextVar instances defined outside of pynetdicom and effectively hook into the logger calls inside of pynetdicom. To make this work, we need to pass along the current context to each thread spawned by pynetdicom. This is the default behavior in asyncio, so it should play nicely with a future implementation of #527.

Let me know what you think!

pynetdicom/_config.py Outdated Show resolved Hide resolved
@r1b r1b changed the title feat: add PASS_CONTEXTVARS [MRG] feat: add PASS_CONTEXTVARS Jan 19, 2021
@codecov
Copy link

codecov bot commented Jan 19, 2021

Codecov Report

Merging #582 (534c17b) into master (3ca25f6) will not change coverage.
The diff coverage is 100.00%.

❗ Current head 534c17b differs from pull request most recent head 05ff0e2. Consider uploading reports for the commit 05ff0e2 to get more accurate results

@@            Coverage Diff            @@
##            master      #582   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           27        27           
  Lines         8426      8438   +12     
=========================================
+ Hits          8426      8438   +12     
Impacted Files Coverage Δ
pynetdicom/_config.py 100.00% <100.00%> (ø)
pynetdicom/ae.py 100.00% <100.00%> (ø)
pynetdicom/association.py 100.00% <100.00%> (ø)
pynetdicom/dimse.py 100.00% <100.00%> (ø)
pynetdicom/dul.py 100.00% <100.00%> (ø)
pynetdicom/utils.py 100.00% <100.00%> (ø)

@r1b r1b changed the title [MRG] feat: add PASS_CONTEXTVARS [WIP] feat: add PASS_CONTEXTVARS Jan 20, 2021
@r1b r1b force-pushed the feat/copy-contextvars-to-threads branch 3 times, most recently from ccd9ade to ac3a96b Compare January 20, 2021 18:56
@r1b r1b changed the title [WIP] feat: add PASS_CONTEXTVARS [MRG] feat: add PASS_CONTEXTVARS Jan 20, 2021
Copy link

@jherman-bfly jherman-bfly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some clarification would be appreciated for why make_target exists.

pynetdicom/utils.py Show resolved Hide resolved
pynetdicom/utils.py Outdated Show resolved Hide resolved
@r1b r1b force-pushed the feat/copy-contextvars-to-threads branch 2 times, most recently from d7578ad to ed1bc8e Compare January 29, 2021 18:53
@scaramallion
Copy link
Member

scaramallion commented Apr 17, 2021

Could you add something to the release notes as well? And add the new config variable to docs/reference/config.rst and utils.rst?

@r1b r1b force-pushed the feat/copy-contextvars-to-threads branch from ed1bc8e to 46ce276 Compare May 3, 2021 20:04
@r1b
Copy link
Contributor Author

r1b commented May 3, 2021

@scaramallion done

@r1b
Copy link
Contributor Author

r1b commented May 3, 2021

er oops forgot release notes, one sec

@r1b r1b force-pushed the feat/copy-contextvars-to-threads branch from 46ce276 to 3856212 Compare May 3, 2021 20:35
@r1b r1b force-pushed the feat/copy-contextvars-to-threads branch from 3856212 to 05ff0e2 Compare May 3, 2021 22:05
@scaramallion scaramallion merged commit 5b28410 into pydicom:master May 6, 2021
@r1b r1b mentioned this pull request May 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants