[A1n_d2n] [EXTERNAL] Parameter file checkout (and 'param-check.pl' checker script)
Brad Sawatzky
brads at jlab.org
Tue Dec 22 23:05:03 EST 2020
I pushed this up to the ANL GitLab repo and made a pull request to get
it merged in the main repo. I'll let Sylvester do the merge just to
maintain our standardized approval/review path. Bill has a pending
merge request in the queue too.
In the interim, interested parties can grab the script this way:
wget https://urldefense.proofpoint.com/v2/url?u=https-3A__eicweb.phy.anl.gov_Sawatzky_hallc-5Freplay_-2D_blob_param-2Dcheck_CALIBRATION_param-2Dcheck.pl&d=DwIBAg&c=CJqEzB1piLOyyvZjb8YUQw&r=BiQgYHr0DbfJNsNM2SdXVw&m=YehEaqXD9NCTa67pJRs-zVIfTFiN9KL1hMuD8X5OjEg&s=zc1Mzfp1aVGCA9l_QxYgtqWiO0u71snMiSWzJvuuje8&e=
chmod a+x param-check.pl # make it executable
./param-check --help # run it and display 'help'
More instructions/info are included at the bottom of this email.
The script was written s part of a sanity check of the parameter files
before we kick off a larger scale replay of the Pol-He3 runs. In
general I think the PARAM files (from an up-to-date pull on our main
hallc-replay repo) look OK. I do have some observations/questions
below though.
I'd appreciate if the students would skim the comment list below for
detectors they calibrated and confirm/verify that:
- The noted 'issue/question' is not a really a problem (and let me
know why).
- The noted file is an old/historical file that we are not using and
don't have to worry about. (In that case, we should consider
'git rm <file>' along with a good comment to remove such outdated
files. (They can always be retrieved from the git history later if
desired.)
Please run "param-check.pl -v 'yourfile.param'" on your detector's param
files and verify for yourself whether the vales (and variation, etc) in
your calibration file look correct.
You may also want to use the '--compare' feature of the param-checker.pl
script to look at old vs new calibration files and convince yourself
that the changes are reasonable and make sense to you.
=================================================================================
HMS observations
./param-check.pl -v PARAM/HMS/*/*.param |m
=================================================================================
PARAM/HMS/CAL/hcal_calib_phase1_2.param
PARAM/HMS/CAL/hcal_calib_phase3.param
- large variations in hcal_neg_gain_cor, hcal_pos_gain_cor
PARAM/HMS/CER/hcer_geom.param
- hcer_num_regions = 3 ; This value is from the 6 GeV era? -- do we care?
PARAM/HMS/DC/hdc_tzero_per_wire_*.param
- double check these, particularly the outliers
PARAM/HMS/HODO/hhodo_calib_1267.param
- c1-* params have large spread -- is this normal/reasonable?
PARAM/HMS/HODO/hhodo_calib.param
- hhodo_pos_sigma has a bunch of '100' vals -- is this effectively
undefined? Are they all OK?
=================================================================================
SHMS param file observations
./param-check.pl -v PARAM/SHMS/*/*.param |m
=================================================================================
PARAM/SHMS/CAL/pcal_calib_6619.param
PARAM/SHMS/CAL/pcal_calib_7109.param
- Many blocks have a zero gain... presumably due to insufficient
coverage for calibration?
- Is this a problem? If no, why not?
PARAM/SHMS/CAL/pcal_calib_9589.param
- Test file? (Might still be better stopgap than the above since a
'default' gain of 37 is used for uncalibrated blocks -- at least
they can contribute something?
PARAM/SHMS/CAL/pcal_calib.param
- Considerably better coverage (many runs merged)
- Some significant outliers, but most believable calib so far.
- Is this the 'final' our entire pol-he3 runset?
PARAM/SHMS/DC/pdc_tzero_per_wire_7109.param
- Large values are presumably defaults (ie calibration failed?)
- Are these OK?
PARAM/SHMS/GEN/ptracking.param
- broken commenting, fixed
PARAM/SHMS/HGCER/phgcer_geom.param
- Still has "FIXME! Waiting for numbers from detector" in it.
- Does it really need 'fixing' or are the values confirmed?
PARAM/SHMS/HODO/phodo_calib.param
- Don't understand what these params do, are all the zeros OK?
phodo_neg_invadc_adc, phodo_neg_invadc_offset
phodo_pos_invadc_adc, phodo_pos_invadc_offset
- These have lots of near zero elements, then a bunch of '100'
(default?) values
phodo_neg_sigma, phodo_pos_sigma
PARAM/SHMS/HODO/phodo_Vpcalib.param (OLD file)
PARAM/SHMS/HODO/phodo_TWcalib.param (OLD file)
- "pTDC_threshold=120 ;units of mV"
- This seems high, is this actually used anywhere in the code?
** NOTE: The SHMS Hodo calibration files are a known work in progress.
The above param files are not the ones we'll use for a pol-he3
replay.
=============================================================
This is the commit with info on the param-checker.pl script.
=============================================================
commit c7042c51f62b1dfcb1ea5101ba8141143a5f06db
Author: Brad Sawatzky <brads at jlab.org>
Date: Tue Dec 22 21:45:06 2020 -0500
Add a 'PARAM file validator/checker' script under CALIBRATION/
- This evovled from a bit of a hack to sanity check PARAM/ files
for reasonable formatting and values. Given the variety of
non-standardized parameter file formats there are some assumptions
and heuristics used allow this script to parse general files.
- It checks the following:
- general formatting
'param = val'
'param = <array>'
'param = <simple formula>'
- watches that data in a parameter array have the same type
(ie. number or string string and not a mix)
-
- It can provide some crude sanity check statistics on arrays of data
- It computes mean and std dev and will flag outliers
** IT IS UP TO THE USER TO INVESTIGATE AND DETERMINE IF THE
OUTLIERS ARE OK. IN MANY INSTANCES THEY ARE FINE/EXPECTED **
- It can be used to compare two parameter files. For example, an 'old
calibration' file (and a 'new calibration' file, or files that have
been calibrated against different sets of runs.
- For example:
~/param-check.pl -c hhodo_Vpcalib_3408.param hhodo_Vpcalib_4192.param
- It will print out similar statistics on the differences. Often the
changes should not be large and/or they may be a fairly constant shift
for all parameters.
- Examples:
- Check files for basic formatting/outliers
param-check.pl PARAM/HMS/*/*.param | less
- Check files for formatting/outliers and reprint values in
'standardized' format. Sometimes helpful.
param-check.pl -v PARAM/HMS/*/*.param | less
- Compare two files:
param-check.pl -v -c hhodo_Vpcalib_3408.param hhodo_Vpcalib_4192.param | less
- Here is the 'param-check.pl -h' help text (FWIW):
Usage: CALIBRATION/param-check.pl <files>
Parse and compare hcana parameter (PARAM/*.param) files.
You'll want to run this on a terminal that supports color.
By default, each file will sanity checked individually. Parsing
errors will be flagged, and (array only!) data that contain values
more than 'maxsigma' (10) std.dev away from the mean will be flagged.
Add '-v' to print out all parameters in a standardized format.
Two files (ie. before and after new calibration) can be compared with the
'-c' option. Differences are computed and evaluated between parameter
values.
** NOTE: Many flagged outliers are OK. It is up to /you/ to understand
what is OK (and why) and what is a problem/error!
FLAGS:
-c | --compare (Requires 2 files as arguments)
-m | --maxsigma (Warn if sigma exceeds value [10])
-v | --verbose
-h | --help
--
Brad Sawatzky, PhD <brads at jlab.org> -<>- Jefferson Lab / Hall C / C111
Ph: 757-269-5947 -<>- Fax: 757-269-5235 -<>- Pager: brads-page at jlab.org
The most exciting phrase to hear in science, the one that heralds new
discoveries, is not "Eureka!" but "That's funny..." -- Isaac Asimov
More information about the A1n_d2n
mailing list