[A1n_d2n] [EXTERNAL] Re: Parameter file checkout (and 'param-check.pl' checker script)

[Brad Sawatzky]

Just a heads-up that the 'param-check' branch below is gone now that
Sylvester merged the pull request.

Students should update their hallc_replay repo from the main 'upstream'
repo ASAP.  That will pull in a few other updates (depending on when you
last updated), and the script should appear up in the "CALIBRATION/"
directory too.

-- Brad

On Tue, 22 Dec 2020, Brad Sawatzky wrote:

> I pushed this up to the ANL GitLab repo and made a pull request to get
> it merged in the main repo.  I'll let Sylvester do the merge just to
> maintain our standardized approval/review path.  Bill has a pending
> merge request in the queue too.
> 
> In the interim, interested parties can grab the script this way:
>   wget https://urldefense.proofpoint.com/v2/url?u=https-3A__eicweb.phy.anl.gov_Sawatzky_hallc-5Freplay_-2D_blob_param-2Dcheck_CALIBRATION_param-2Dcheck.pl&d=DwIBAg&c=CJqEzB1piLOyyvZjb8YUQw&r=BiQgYHr0DbfJNsNM2SdXVw&m=FKNWal4xbdSwPSFtmzztkZKnu2sw5-wYdJB8ek2tq1w&s=c_5YFVz_--jy6tHiDpQAgbqa_9vmFgs20yTGOQ4-69Q&e=  
>   chmod a+x param-check.pl  # make it executable
>   ./param-check --help      # run it and display 'help'
> More instructions/info are included at the bottom of this email.
> 
> 
> The script was written s part of a sanity check of the parameter files
> before we kick off a larger scale replay of the Pol-He3 runs.  In
> general I think the PARAM files (from an up-to-date pull on our main
> hallc-replay repo) look OK.  I do have some observations/questions
> below though.
> 
> I'd appreciate if the students would skim the comment list below for
> detectors they calibrated and confirm/verify that:
>   - The noted 'issue/question' is not a really a problem (and let me
>     know why).
>   - The noted file is an old/historical file that we are not using and
>     don't have to worry about.  (In that case, we should consider 
>     'git rm <file>' along with a good comment to remove such outdated
>     files.  (They can always be retrieved from the git history later if
>     desired.)
> 
> Please run "param-check.pl -v 'yourfile.param'" on your detector's param
> files and verify for yourself whether the vales (and variation, etc) in
> your calibration file look correct.
> 
> You may also want to use the '--compare' feature of the param-checker.pl
> script to look at old vs new calibration files and convince yourself
> that the changes are reasonable and make sense to you.
> 
> 
> 
> =================================================================================
> HMS observations
>   ./param-check.pl -v PARAM/HMS/*/*.param |m
> =================================================================================
> 
> PARAM/HMS/CAL/hcal_calib_phase1_2.param
> PARAM/HMS/CAL/hcal_calib_phase3.param
>   - large variations in hcal_neg_gain_cor, hcal_pos_gain_cor
> 
> PARAM/HMS/CER/hcer_geom.param
>   - hcer_num_regions = 3 ; This value is from the 6 GeV era? -- do we care?
> 
> PARAM/HMS/DC/hdc_tzero_per_wire_*.param
>   - double check these, particularly the outliers
> 
> 
> PARAM/HMS/HODO/hhodo_calib_1267.param
>   - c1-* params have large spread -- is this normal/reasonable?
> 
> PARAM/HMS/HODO/hhodo_calib.param
>   - hhodo_pos_sigma has a bunch of '100' vals -- is this effectively
>     undefined?  Are they all OK?
> 
> 
> =================================================================================
> SHMS param file observations
>   ./param-check.pl -v PARAM/SHMS/*/*.param |m
> =================================================================================
> PARAM/SHMS/CAL/pcal_calib_6619.param
> PARAM/SHMS/CAL/pcal_calib_7109.param
>   - Many blocks have a zero gain...  presumably due to insufficient
>     coverage for calibration?
>   - Is this a problem?  If no, why not?
> 
> PARAM/SHMS/CAL/pcal_calib_9589.param
>   - Test file?  (Might still be better stopgap than the above since a
>     'default' gain of 37 is used for uncalibrated blocks -- at least
>     they can contribute something?
> 
> PARAM/SHMS/CAL/pcal_calib.param
>   - Considerably better coverage (many runs merged)
>   - Some significant outliers, but most believable calib so far.
>   - Is this the 'final' our entire pol-he3 runset?
> 
> PARAM/SHMS/DC/pdc_tzero_per_wire_7109.param
>   - Large values are presumably defaults (ie calibration failed?)
>   - Are these OK?
> 
> PARAM/SHMS/GEN/ptracking.param
>   - broken commenting, fixed
> 
> PARAM/SHMS/HGCER/phgcer_geom.param
>   - Still has "FIXME! Waiting for numbers from detector" in it.
>   - Does it really need 'fixing' or are the values confirmed?
> 
> 
> PARAM/SHMS/HODO/phodo_calib.param
>   - Don't understand what these params do, are all the zeros OK?
>     phodo_neg_invadc_adc, phodo_neg_invadc_offset
>     phodo_pos_invadc_adc, phodo_pos_invadc_offset
>   - These have lots of near zero elements, then a bunch of '100'
>     (default?) values
>     phodo_neg_sigma, phodo_pos_sigma
> 
> PARAM/SHMS/HODO/phodo_Vpcalib.param  (OLD file)
> PARAM/SHMS/HODO/phodo_TWcalib.param  (OLD file)
>   - "pTDC_threshold=120 ;units of mV"
>     - This seems high, is this actually used anywhere in the code?
> 
> ** NOTE: The SHMS Hodo calibration files are a known work in progress.
>      The above param files are not the ones we'll use for a pol-he3
>      replay.
> 
> 
> 
> 
> =============================================================
> This is the commit with info on the param-checker.pl script.
> =============================================================
> 
> commit c7042c51f62b1dfcb1ea5101ba8141143a5f06db
> Author: Brad Sawatzky <brads at jlab.org>
> Date:   Tue Dec 22 21:45:06 2020 -0500
> 
>     Add a 'PARAM file validator/checker' script under CALIBRATION/
> 
>     - This evovled from a bit of a hack to sanity check PARAM/ files
>       for reasonable formatting and values.  Given the variety of
>       non-standardized parameter file formats there are some assumptions
>       and heuristics used allow this script to parse general files.
> 
>     - It checks the following:
>       - general formatting
>         'param = val'
>         'param = <array>'
>         'param = <simple formula>'
>       - watches that data in a parameter array have the same type
>         (ie. number or string string and not a mix)
>       -
> 
>     - It can provide some crude sanity check statistics on arrays of data
>       - It computes mean and std dev and will flag outliers
>         ** IT IS UP TO THE USER TO INVESTIGATE AND DETERMINE IF THE
>            OUTLIERS ARE OK.  IN MANY INSTANCES THEY ARE FINE/EXPECTED **
> 
>     - It can be used to compare two parameter files.  For example, an 'old
>       calibration' file (and a 'new calibration' file, or files that have
>       been calibrated against different sets of runs.
>       - For example:
>         ~/param-check.pl -c hhodo_Vpcalib_3408.param hhodo_Vpcalib_4192.param
>       - It will print out similar statistics on the differences.  Often the
>         changes should not be large and/or they may be a fairly constant shift
>         for all parameters.
> 
>     - Examples:
>       - Check files for basic formatting/outliers
>           param-check.pl PARAM/HMS/*/*.param | less
>       - Check files for formatting/outliers and reprint values in
>         'standardized' format.  Sometimes helpful.
>           param-check.pl -v PARAM/HMS/*/*.param | less
>       - Compare two files:
>           param-check.pl -v -c hhodo_Vpcalib_3408.param hhodo_Vpcalib_4192.param  | less
> 
>     - Here is the 'param-check.pl -h' help text (FWIW):
> 
>         Usage: CALIBRATION/param-check.pl <files>
> 
>         Parse and compare hcana parameter (PARAM/*.param) files.
> 
>         You'll want to run this on a terminal that supports color.
> 
>         By default, each file will sanity checked individually.  Parsing
>         errors will be flagged, and (array only!) data that contain values
>         more than 'maxsigma' (10) std.dev away from the mean will be flagged.
> 
>         Add '-v' to print out all parameters in a standardized format.
> 
>         Two files (ie. before and after new calibration) can be compared with the
>         '-c' option.  Differences are computed and evaluated between parameter
>         values.
> 
>         ** NOTE: Many flagged outliers are OK.  It is up to /you/ to understand
>                  what is OK (and why) and what is a problem/error!
> 
>         FLAGS:
>         -c | --compare    (Requires 2 files as arguments)
>         -m | --maxsigma   (Warn if sigma exceeds value [10])
>         -v | --verbose
>         -h | --help
> 
-- 
Brad Sawatzky, PhD <brads at jlab.org>  -<>-  Jefferson Lab / Hall C / C111
Ph: 757-269-5947  -<>-  Fax: 757-269-5235  -<>- Pager: brads-page at jlab.org
The most exciting phrase to hear in science, the one that heralds new
  discoveries, is not "Eureka!" but "That's funny..."   -- Isaac Asimov