Convert boost hists to ROOT TGraphs #17

apaasch · 2024-03-08T13:34:00Z

Task structure to store boost histogram objects as TGraphAsymmErrors in rootfiles.
Main goal is to do this indepently from Root using uproot.
This is not yet possible but there is ongoing work from the uproot team addressing this issue.
Updates can be followed in this PR from uproot5.

As long as this feature is not yet implement the conversion is done in two steps:

Extract values from the boost histogram and store as arrays, which can be used as an input for a TGraph object. Store the dictionary as a pickel file.
Extract the arrays from the pickle file and use them as input for a TGraphAsymmError. This step is done independently from CF, since no Root module is implemented in the Sandboxes.

In the long term, the second step is included in the first one and remove the dependency from Root.

Additional

Move category IDs to base task

dsavoiu

Thanks for the PR, @apaasch. It looks OK to me so far, I only have a few comments related to code style and variable naming, but this should be easy to fix. One should also maybe clarify if the convert_to_root.py script is meant to run while columnflow is sourced or completely outside (it would also be good to add this information to the file).

dijet/tasks/root.py

dijet/tasks/base.py

dsavoiu · 2024-03-11T16:12:57Z

dijet/tasks/base.py

@@ -57,8 +60,8 @@ def get_datasets(self) -> tuple[list[str], bool]:
        # check that at least one config dataset matched
        if not dataset_insts_from_process:
            raise RuntimeError(
-                "no single dataset found in config matching "
-                f"process `{self.branch_data.process}`"
+                "no single dataset found in config matching ",


No comma needed here since the string continues on the next line.

Suggested change

"no single dataset found in config matching ",

"no single dataset found in config matching "

dsavoiu · 2024-03-11T16:13:07Z

dijet/tasks/base.py

@@ -68,8 +71,8 @@ def get_datasets(self) -> tuple[list[str], bool]:
        # check that at least one user-supplied dataset matched
        if not datasets_filtered:
            raise RuntimeError(
-                "no single user-supplied dataset matched "
-                f"process `{self.branch_data.process}`"
+                "no single user-supplied dataset matched ",


Suggested change

"no single user-supplied dataset matched ",

"no single user-supplied dataset matched "

dijet/tasks/jer.py

dijet/tasks/root.py

dsavoiu · 2024-03-11T16:26:25Z

dijet/scripts/convert_to_root.py

+    print(f"{arg}: {getattr(args, arg)}")
+
+# Construct the file path
+base_path = "/nfs/dust/cms/user/paaschal/WorkingArea/Analysis/JERC/DiJet/data/dijet_store/analysis_dijet/"


Can this script be run inside a columnflow environment (i.e. after sourcing setup.sh)? If so, it would be good to avoid hardcoding the path here and use the environment variables insteda (e.g. os.getenv("CF_STORE_LOCAL")).

Yes it can be run within the CF environment. You suggestion will be added to the next commit.

Redo flake8 changes Change naming scheme and structur in root task to be more efficient and readable. Co-authored-by: Daniel Savoiu <[email protected]>

apaasch

Thanks, all your suggestion have been added.

apaasch · 2024-03-12T09:17:44Z

dijet/scripts/convert_to_root.py

+    print(f"{arg}: {getattr(args, arg)}")
+
+# Construct the file path
+base_path = "/nfs/dust/cms/user/paaschal/WorkingArea/Analysis/JERC/DiJet/data/dijet_store/analysis_dijet/"


Yes it can be run within the CF environment. You suggestion will be added to the next commit.

apaasch · 2024-03-12T09:36:49Z

dijet/tasks/base.py

@@ -57,8 +60,8 @@ def get_datasets(self) -> tuple[list[str], bool]:
        # check that at least one config dataset matched
        if not dataset_insts_from_process:
            raise RuntimeError(
-                "no single dataset found in config matching "
-                f"process `{self.branch_data.process}`"
+                "no single dataset found in config matching ",


apaasch · 2024-03-12T09:36:52Z

dijet/tasks/base.py

@@ -68,8 +71,8 @@ def get_datasets(self) -> tuple[list[str], bool]:
        # check that at least one user-supplied dataset matched
        if not datasets_filtered:
            raise RuntimeError(
-                "no single user-supplied dataset matched "
-                f"process `{self.branch_data.process}`"
+                "no single user-supplied dataset matched ",


dsavoiu

Thanks for the changes! Looks good to me, but there is one small change I would make to the convert_to_root.py script (see below).

dijet/scripts/convert_to_root.py

dsavoiu

Thanks, this looks good to me now. Let's merge!

apaasch added 4 commits March 8, 2024 13:41

set nan values to 0

c8268bf

move category ids to base task

f42d6e3

task to convert JERs to root

4746f61

temporary script to convert JER to root

db95b91

apaasch assigned dsavoiu Mar 8, 2024

lint

d9bc103

dsavoiu requested changes Mar 11, 2024

View reviewed changes

apaasch and others added 3 commits March 12, 2024 10:11

Apply suggestions from code review

aae9383

Redo flake8 changes Change naming scheme and structur in root task to be more efficient and readable. Co-authored-by: Daniel Savoiu <[email protected]>

set path via cf variables

3cfeef9

lint

0950cf7

apaasch commented Mar 12, 2024

View reviewed changes

dsavoiu requested changes Mar 18, 2024

View reviewed changes

dijet/scripts/convert_to_root.py Outdated Show resolved Hide resolved

typo and rename uncertainty to calibrator

0c73ca0

dsavoiu approved these changes Mar 21, 2024

View reviewed changes

dsavoiu merged commit 2dcdeaa into uhh-cms:master Mar 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert boost hists to ROOT TGraphs #17

Convert boost hists to ROOT TGraphs #17

apaasch commented Mar 8, 2024

dsavoiu left a comment

dsavoiu Mar 11, 2024

apaasch Mar 12, 2024 •

edited

Loading

dsavoiu Mar 11, 2024

apaasch Mar 12, 2024 •

edited

Loading

dsavoiu Mar 11, 2024

apaasch Mar 12, 2024

apaasch left a comment

apaasch Mar 12, 2024

apaasch Mar 12, 2024 •

edited

Loading

apaasch Mar 12, 2024 •

edited

Loading

dsavoiu left a comment

dsavoiu left a comment

	"no single dataset found in config matching ",
	"no single dataset found in config matching "

	"no single user-supplied dataset matched ",
	"no single user-supplied dataset matched "

Convert boost hists to ROOT TGraphs #17

Convert boost hists to ROOT TGraphs #17

Conversation

apaasch commented Mar 8, 2024

dsavoiu left a comment

Choose a reason for hiding this comment

dsavoiu Mar 11, 2024

Choose a reason for hiding this comment

apaasch Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

dsavoiu Mar 11, 2024

Choose a reason for hiding this comment

apaasch Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

dsavoiu Mar 11, 2024

Choose a reason for hiding this comment

apaasch Mar 12, 2024

Choose a reason for hiding this comment

apaasch left a comment

Choose a reason for hiding this comment

apaasch Mar 12, 2024

Choose a reason for hiding this comment

apaasch Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

apaasch Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

dsavoiu left a comment

Choose a reason for hiding this comment

dsavoiu left a comment

Choose a reason for hiding this comment

apaasch Mar 12, 2024 •

edited

Loading

apaasch Mar 12, 2024 •

edited

Loading

apaasch Mar 12, 2024 •

edited

Loading

apaasch Mar 12, 2024 •

edited

Loading