Updated changes from main #14
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…262) * Read Simple Reco info to PhaseIITreeMaker and add Simple Tank Track Length variable * Updated Reweight Tool and added separate Reweight Flux Tool --------- Co-authored-by: James Minock <[email protected]>
…er to fix charge omission seen for MC Hits (#259) * Update ClusterFinderConfig Narrow cluster window - typical use is 5 hits in 50ns, not 5 hits in 100ns * Update EventSelectorConfig New timing offsets moved the PMTs "back" by an average of ~10ns. When looking for PMT/MRD coincidence, we this need to adjust this offset value. * Update ClusterClassifiers.cpp With new gains calibration we now have 121 active PMTs instead of 123. Adjust charge balance parameter to reflect the number of active PMTs that could capture light. In the future, will alter this to read in the channel keys so that it automatically grabs the correct number of active PMTs. * Update ClusterFinder.cpp Fixed issues seen with ClusterFinder and the omission of charge for MC Hits: - Change MC hit time away from 2ns discretization, as the new data hit times are float values. - As a result, need to change for loop to not just grab integer hit times (old), but also grab float hit times. - MC Hits with times < 0 were being ignored due to a variable set to 0. Tool now keeps all hit times, even if they are negative due to time smearing. - Reduce first-photon effect by modifying how hit timing is assigned for a "pulse"/digit. Followed what the tool DigitBuilder used: median photon hit time as the "pulse" time. Also, - In Data, mandate hits need to be > 0 (there is a weird spike at 0 in cluster times that are from this and are likely unphysical) TODO: Potentially expand the MC "pulse" integration window from 10ns to something larger. Average charge per hit is lower and total number of hits slightly higher in MC than data, indicating the pulse window may be too narrow and is cutting what should be one pulse into two. Need to check with calibration to see what the average pulse window length is for ~SPE pulses. * Update ClusterFinderConfig Same as data - cluster window 50ns, looking for at least 5 hits * Update ClusterFinderConfig Dropped line pointing to an external WCSim channel key ID (not really an issue cause they are the same) * Update DeadPMTIDs_p2v7.txt Updated mask to omit additional PMT to reflect new gains data. (PMT 337, aka 6 here, no longer active in data) * Update LoadWCSimConfig Added a small description on where to find collaboration WCSim samples. Also cut down on commented lines * Update LoadGenieEventConfig Updated tool config to read in new collaboration samples. Added description to each variable per doc 5230-v2 (LoadGenieEvent Tutorial) * Update PhaseIITreeMakerConfig In prep for whenever we have tank reco available * Update LoadGeometryConfig new gains * Update DeadPMTIDs_p2v7.txt Updated new inactive PMT from the gains calibration * Update LoadWCSimConfig Updated config to include paths to latest collaboration samples * Add files via upload PMT timing offsets derived from Laser calibration data. These offsets should be considered precision software corrections on top of the previous firmware corrections at the ADC level to correct for the large cabling delays. * Update LoadGeometry.h Updated header file with PMT timing offsets * Update LoadGeometryConfig Path to PMT timing offset .csv file * Update EventSelector.cpp 1. Updated default offset value (745) to 755 to account for new timing offsets of the PMTs. 2. In the new gains file, there are measurements for the 121 active PMTs. Two PMTs were deemed "inactive" and are no longer included. (Old file used to have 123 active PMTs). These PMTs were omitted as no data/hits were recorded on them during the source calibration. In principle there should only be hits in the active PMTs. However, even though there were no data/hits recorded in those tubes for the source calibration, for very energetic events occasionally one of the since "inactived" PMTs registers a hit for some reason and tries to access the map, which crashes this tool. Just added a condition that the channel must exist in the map in order to count that hit. * Update LoadGeometry.cpp Adjust LoadGeometry to include new PMT Timing offsets from .csv file, derived from the laser source calibration. * Update PhaseIIADCHitFinder.cpp - Grab new PMT timing offset values from .csv file to subtract from the hit times --> added this to all methods of hit finding - Instead of grabbing the x tick with the maximum ADC value as the hit time (always binned to 2ns), assign float hit times based when the pulse crosses 50% of its maximum above threshold. Currently only implemented for the Pulse Finding Approach "Threshold" and Pulse Window Type "Dynamic" (the default, used configuration of the code). TODO: - Add this hit timing approach to all configurations of the code. - (Maybe) per Bob's recommendation, it might be better to instead assign the hit time using the following: 1. get the time at 50% between peak and threshold 2. get the time at 20% " " " " 3. Draw a straight line between these two points to extrapolate the time to zero threshold This way it would lessen the effect of time shifts between large and small pulses. * Update PhaseIIADCHitFinder.h Added offset map to header * Add files via upload Old tool, new toolchain: output raw waveforms of PMT and aux channels from built, processed ANNIEEvents (that contain the raw waveforms). Useful to analyze PMT waveforms, as well as look at aux channel waveforms like the BoosterRWM and BoosterRF. * Add files via upload In case users want to eventbuild part files with raw waveforms (useful for source analysis), this toolchain complements the official event building toolchain that is used to create our processed data. Configurations for the tools to extract raw waveforms successfully from the PMTs are taken from Gian's AmBe analysis work. Currently, the event building for this toolchain has only been tested for Tank + CTC events. * Add files via upload Added README for event building toolchain, with reference to the ANNIE wiki page on how to properly event build. * Update README.md small syntax fix for EventBuilder README file * Update ClusterFinder.cpp Missed a bracket which caused the CI build error * Update LoadWCSim.cpp When using the "automatic" method for loading in GENIE files in the LoadGenieEvent, this was causing that tool to crash as it couldn't find the proper path based on the WCSim file name. Removed the to_string function * Update EventSelector.cpp Removed the charge threshold (> 200 pe) needed to consider tank/MRD coincidence. Given the README states "PMTMRDCoincCut: Flags events that do not have a certain time offset between PMT & MRD cluster" this should include all PMT clusters, and not just some that are over a certain charge threshold. This change in reality likely won't change much, as most muon tracks in the MRD deposit enough light to leave more than 200 pe worth of charge, but per our Slack convo it's probably best to just remove this condition as its not obvious.
…268) * Update DNNTrackLengthTrainConfig Changed the storing location of ScalingVarsStore.bs since the Data_Energy_Reco directory doesn't exist in the main ToolAnalysis repository. * Update EventSelectorConfig Applied the cuts that we use currently in the Energy Reconstruction. * Update HitCleanerConfig Lowered the verbosity * Update LoadWCSimConfig Upped the verbosity * Update LoadWCSimLAPPDConfig Upped the verbosity * Update TimeClusteringConfig Lowered the verbosity * Adding the PhaseIITreeMaker Tool Adding a config file for the PhaseIITreeMaker Tool so we can use it in the Energy Reconstruction ToolChain. * Rename PhaseIITreeMakerConfig.txt to PhaseIITreeMakerConfig Removed the .txt from the file name * Update ToolsConfig Added a line for the PhaseIITreeMaker Tool * Update DNNTrackLengthPredictConfig Changed the path of ScalingVarsStore.bs according to the change I made in the Train_Test/DNNTrackLengthTrainConfig * Update EventSelectorConfig Applied the current cuts used in the Energy Reconstruction * Update HitCleanerConfig Lowered the verbosity * Update LoadWCSimConfig Upped the verbosity * Update LoadWCSimLAPPDConfig Upped the verbosity * Update ToolsConfig Added a line for the PhaseIITreeMaker Tool * Adding PhaseIITreeMaker in /Predict Adding a config file for PhaseIITreeMaker just like I did for the Train_Test ToolChain * Rename PhaseIITreeMakerConfig.txt to PhaseIITreeMakerConfig Removed the .txt from the file name * Update PlotsTrackLengthAndEnergy.cpp Added a histogram that plots the Energy Relative Deviation between E_true and E_reco * Update PlotsTrackLengthAndEnergy.cpp Added a missing line * Updating PhaseIITreeMaker to include Digit info Added lines of code to the .h and .cpp files to include the digitX, digitY, digitZ and digitT variables in the root ouput * Update PhaseIITreeMaker.cpp re-trigger workflow --------- Co-authored-by: marc1uk <[email protected]>
…ool: (#267) - Renamed load_from_file -> load_from_csv - Introduced variable save_to_csv - Updated documentation at beginning of RingCounting.py to reflect changes - Updated README.md to reflect changes - Updated UserTools/RingCounting/RingCountingConfig to reflect changes Users using the RingCounting tool need to update their RingCountingConfig files following this change.
* Read Simple Reco info to PhaseIITreeMaker and add Simple Tank Track Length variable * Updated Reweight Tool and added separate Reweight Flux Tool * Updated reading in new ANNIE flux histograms --------- Co-authored-by: James Minock <[email protected]>
* Read Simple Reco info to PhaseIITreeMaker and add Simple Tank Track Length variable * Updated Reweight Tool and added separate Reweight Flux Tool * Added RingCounting variables to PhaseIITreeMaker --------- Co-authored-by: James Minock <[email protected]>
* Update Makefile PythonScript lives /ToolAnalysis. It can't be edited, so 'make clean' break at that point. * Update Makefile rework solution to make it less specific. this version removes all files that can be removed, skipping those that aren't writable. --------- Co-authored-by: marc1uk <[email protected]>
* Update BeamFetcherV2.cpp - fixed a bug where not all timestamps were being saved to the output ROOT file - clear beamdatamap after writing it to the root file. This increases the speed of the tool by ~2x - change the beam triggerword from 5 to 14 (undelayed beam trigger). This doesn't really affect the entries we get (< 1% difference in total counts and total recorded POT), but Yue's new event building (coming soon) builds on 14, so trying to stay consistent with that * Update BeamFetcherV2.cpp Missed two small changes * Update BeamFetcherV2.h
* Update ClusterFinderConfig With the new timing offsets for data, cluster window can be reduced from 50ns to 40ns, as MC shows all reflected + scattered light resulting is captured within 40ns of the initial hit. * Update ClusterFinderConfig New PMT timing offsets reduced spread in hit times that were a result of unsubtracted TOF differences. The cluster window can be reduced from 50ns to 40ns; this narrower window still captures all light from the interaction according to the MC.
* Update ChannelSPEGains2023.csv this channel does not have a gains measurement as its off/inactive. However, for some reason in very rare instances a hit is registered (not sure why this is the case). Set the gains to 0 in case a hit is registered, as to assign it 0 ADC value and not contribute any charge. * Update TankPMTTimingOffsets.csv Same as the gains, but assign a hit time offset of 0 (no measurement data on this channel from the laser calibration)
Add "pulse width" as a configuration variable in MC hits. It is traditionally 10ns in width, but this width is yet to be tuned to data.
If database entry for a particular device doesn't return any values, fill that device branch in the output root file with -9999 instead of leaving it empty. We use this tool in the BeamFetcherV2 toolchain to produce beam root files that are then used by the event building toolchain. The beam root files contain a branch for each device. Without this modification, as the BeamFetcherV2 toolchain loops over RAWData part files, grabs CTC timestamps and queries for DB values, if a device entry is empty the toolchain will fail to fill the entire device branch as the entries between branches will be unequal.
Co-authored-by: James Minock <[email protected]>
…ct x,y,z conventions (#284) Co-authored-by: James Minock <[email protected]>
…produced it. (#285) * A new tool that links vectors of MCHits to the particle that deposited the most energy to the cluster. * Changing variables over to pointers * Making this thing actually work. LoadWCSim outputs were not as expected... * Some final adjustments and finally a README
Co-authored-by: James Minock <[email protected]>
…d momentum transfer, bjorken x, elasticity y, and target nuclei (#280) Co-authored-by: James Minock <[email protected]>
…Gs and momentum, energy and momentum transfer, bjorken x, elasticity y, and target nuclei (#281) Co-authored-by: James Minock <[email protected]>
Co-authored-by: James Minock <[email protected]>
Expand file limit to allow tools like LoadANNIEEvent to loop over all part files of a run
* Update PhaseIIADCHitFinder.cpp Assigned correct 50% interpolation time instead of max ADC time * Update PhaseIIADCHitFinder.cpp clean up verbose of the hit finding tool
* Add files via upload New BeamQuality tool (from Andrew) to produce BeamStatuses from the BeamFetcherV2 that the EventBuilding can interface with * Update Factory.cpp * Update Unity.h * Update BeamQuality.cpp change beam trigger to 14 for the new event builder --------- Co-authored-by: marc1uk <[email protected]>
* Delete configfiles/EventBuilder/BeamDecoderConfig BeamFetcherV2 and BeamQuality tools replace BeamDecoder functionality * Add files via upload BeamFetcherV2 device list (currently set to read all of them) * Update MRDDataDecoderConfig drop verbose * Update README.md * Update ToolsConfig * Add files via upload
* Add check for TriggerWord being set * Set StopLoop to 1 when failed to get TriggerWord
* Update PMTDataDecoder.cpp Add functionality for reading the RWM + BRF waveforms * Update PMTDataDecoder.h * Add files via upload New Fit RWM+BRF tool that will find the rising edge of the auxiliary channel waveforms * Create README.md README for the new RWM fitting tool * Update FitRWMWaveform.cpp delete histograms to prevent potential memory leakage
* Update FindMrdTracks.cpp Don't hault the toolchain if there are no MRD tracks * Update TriggerDataDecoder.cpp Triggers are stored to the buffer - clear it to speed up the tool. Also adding verbosity conditions on std out * Update LoadRawDataConfig Wrong config variables - now it should work out of the box as intended * Update CreateMyList.sh Rather than passing the path of the RAWData file to the bash script, add it hard coded. Not great practice but this script is used for a very specific purpose to generate a list file containing part files from the RAWData, and the data location has not changed in years (and likely won't change for now). * Update FindMrdTracks.cpp revert changes
* Added MuonFitter Tool * Fixed minor bug for string delimination --------- Co-authored-by: James Minock <[email protected]> Co-authored-by: marc1uk <[email protected]>
* Add files via upload BeamFetcherV2 toolchain * Update README.md * Update LoadRawDataConfig
Replace LAPPD geometry file with the corrected version
Fix to allow event builder to construct MRD + CTC entries
* Add files via upload Adjusted tool to handle LAPPD integration * Update LoadRawData.cpp update comments * Add files via upload
… times (#304) * Add files via upload New tool to assign BNB spill structure to MC cluster times * Update Factory.cpp * Update Unity.h * Update AssignBunchTimingMC.cpp silly missed bug
* Update README.md Beef up README for PhaseIITreeMaker * Add files via upload Changes to PhaseIITreeMaker to read bunchTimes from the AssignBunchTimingMC tool
) * Add files via upload New PrintDQ tool to be used for a quick snapshot of beam run quality. * Update Factory.cpp * Update Unity.h * Add files via upload adjust verbose from v_error to v_debug if it can't find objects in the Stores. It's a printing tool so not accessing those Stores will just lead the counts to be 0 for that particular event. --------- Co-authored-by: marc1uk <[email protected]>
* RNNFit model for MuonFitter * Fixed read in of Fit script * moved model files out and added some documentation for model generation * removed extract_partnumber.py and added info to README for MuonFitter. Also changed location of MuonFitter README from MuonFitter/RNNFit/ to MuonFitter/ * Update README.md --------- Co-authored-by: James Minock <[email protected]> Co-authored-by: marc1uk <[email protected]>
* PrintDQ toolchain Toolchain to print run quality statistics * Delete configfiles/PrintDQ/CreateProcessedList.sh redundant script - there is another one that does the same thing * Update README.md update README
* minimal Toolchain for MuonFitter * Update FindMrdTracksConfig don't write FindMrdTracks results to debug ROOT file --------- Co-authored-by: James Minock <[email protected]> Co-authored-by: marc1uk <[email protected]>
Co-authored-by: James Minock <[email protected]>
Co-authored-by: James Minock <[email protected]>
* Added README's for MuonFitter * Updated torch warning and location of model files * Updated README file to clarify steps/issues regarding MuonFitter model generation * Fix to previous commit, deleted AutoDict_map_string_vector_double__.cxx --------- Co-authored-by: James Minock <[email protected]>
…_prepare.py (#320) Co-authored-by: James Minock <[email protected]>
Co-authored-by: James Minock <[email protected]> Co-authored-by: marc1uk <[email protected]>
* add RingCounting tool * RingCounting.py: added model loading and documentation * RingCounting.py: fix bug * RingCounting.py: rm unimplemented method call * RingCounting.py: fixed pmt masking, fixed file loading, added files_to_load path line commenting support * RingCounting.py: + docStrings + saving controlled by [[save_to]] * added RingCountingConfig, fixed loading error in RingCountingTool * RingCounting.py: + logging of successful file loading + pep8 formatting * Filled RingCountingTool README.md * Created RingCounting Tool (#1) Created RingCounting Tool --------- Co-authored-by: Daniel T. Schmid <[email protected]> * Update files_to_load.txt noop to trigger CI test * Fixed mrdtotalentries being smaller than tanktotalentries throwing a fatal error in LoadRawData.cpp when build type is set to "MRD", since in this case only MRD data is expected to be loaded. * Improved clarity of the load and save operation of the RingCounting tool: - Renamed load_from_file -> load_from_csv - Introduced variable save_to_csv - Updated documentation at beginning of RingCounting.py to reflect changes - Updated README.md to reflect changes - Updated UserTools/RingCounting/RingCountingConfig to reflect changes Users using the RingCounting tool need to update their RingCountingConfig files following this change. * Added ensemble support for RingCounting tool. - Moved logic for processing predictions to method process_predictions - Added new configuration variables model_is_ensemble, ensemble_model_count,ensemble_prediction_combination_mode - Added support for loading ensemble models and processing predictions - Added support for average and majority-voting ensemble - Added storing of majority-voting ensemble class prediction in new variables RingCountingVoting{SR,MR}Prediction in RecoEvent BoostStore - Updated documentation within RingCounting.py - Updated README.md - Updated configfiles/RingCounting/RingCountingConfig Users using the RingCounting tool need to update their RingCountingConfig file(s) following this change. --------- Co-authored-by: Daniel T. Schmid <[email protected]> Co-authored-by: marc1uk <[email protected]>
* Update ClusterFinderConfig Extend MC pulse width to align data and MC.
* Update AssignBunchTimingMC.cpp new neutrino propagation time to the tank, from James' new tank samples with a narrowed flux window. Add this tool (and its config) to James' CC analysis toolchain and the MC BeamCluster toolchain. Add config variables that the tool asks for. Tank clustering, MRD clustering, and trigger clustering are all default 1 but best to add it in. Also added the AssignBunchTiming
* Add FitRWM tool to Factory * Add FitRWM to Unity
This reverts commit 0d9aca0. Testing by James M. indicates that this PR introduced breaking changes. While we wait for assistance from Daniel in correcting them, revert to the previous working version. Co-authored-by: Marcus O'Flaherty <[email protected]>
* add .csv output for storing run metrics
Unsmeared hits are relative to MC global time. Adding in the event time was causing an artificial shift in the hit times if smeared hit times was turned off.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.