[Halld-offline] [EXTERNAL] auxilliary data file needed for running with HADR=4
Thomas Britton
tbritton at jlab.org
Fri May 15 14:25:54 EDT 2020
Let me align the csh and bash version. A divergence occurred back when ccdb was having issues. Those sqlite files should be up to date. But I have an inkling the cron job producing them was stopped. Possibly before the fix. Give me about 5 min Collin and I’ll write you privately when you should git pull
Thomas Britton
On May 15, 2020, at 2:23 PM, Richard Jones <richard.t.jones at uconn.edu> wrote:
Hello all,
Can someone help Colin point to the current database? It looks like he is getting something obsolete?
-Richard J.
On Fri, May 15, 2020 at 2:06 PM Colin Gleason <gleasonc at jlab.org<mailto:gleasonc at jlab.org>> wrote:
*Message sent from a system outside of UConn.*
I do not think so, I only see "calibtime: no time" and that it's pulling the ccdb from
ccdb sqlite path: jlab_batch_default sqlite:////scratch/slurm/14459962/31056_0/ccdb.sqlite, which i think is just the ccdb copied to the local run directory? Maybe Thomas can weigh in on how MCwrapper does this? I do not specify any calib date/time, just use whatever the default is.
I only see this issue arise with the HADR=4 option. With HADR=1 I do not see any issues.
On Fri, May 15, 2020 at 1:57 PM Richard Jones <richard.t.jones at uconn.edu<mailto:richard.t.jones at uconn.edu>> wrote:
Colin, have you back-dated your lookups with some kind of calibdate type specification.
-Richard
On Fri, May 15, 2020 at 1:56 PM Colin Gleason <gleasonc at jlab.org<mailto:gleasonc at jlab.org>> wrote:
*Message sent from a system outside of UConn.*
Richard,
Thanks for the information. My CCDB_CONNECTION is the standard GlueX one mysql://ccdb_user@hallddb.jlab.org/ccdb<https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttp-2D253A-2D252F-2D252Fccdb-2D5Fuser-2D2540hallddb.jlab.org-2D252Fccdb-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257Cbc281fedbdd048fbeafc08d7f8f93aa4-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637251621642432402-2D26sdata-2D3Dxy08TH3nI1N8efncTZMQcbeH-2D252BUw2LaN-2D252B3ZKtxgeTbMI-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253D-5Fp-5FZf1fP8D9GtGtaEgIPjViLbHJLGv-5FRV32KAKGtMbE-2526m-253DDWXCbzopWFmoGrJlEJxX271evN-2D3OXDw-5F6jVvif2GYQ-2526s-253DtH1GdfR2NS0B72JeD9zo8rL34992xn5XPIxNnR6ivho-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C0ff70c723695477bfa7608d7f8faaef8-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637251627879670571-26sdata-3DrmgUr-252BPJcQ8aN7Mp5ZFStRcXRW7TTc0hDFQ1Ha7iYnw-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ccFffx721N71hPpKcJGvJIqY9RM4gBTuzp9ir7rze5Q&m=HhQPGhhgXKdDhIQqUPWDLe1Gf2l4G9u0hmSG42KUZpA&s=y7Fw2_ohg1O42SVuqZXZ-Ja3hjgMiER5ym13sVrlPM8&e=>
CCDB_CONNECTION=mysql://ccdb_user@hallddb.jlab.org/ccdb<https://urldefense.proofpoint.com/v2/url?u=https-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttp-2D253A-2D252F-2D252Fccdb-2D5Fuser-2D2540hallddb.jlab.org-2D252Fccdb-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257Cbc281fedbdd048fbeafc08d7f8f93aa4-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637251621642432402-2D26sdata-2D3Dxy08TH3nI1N8efncTZMQcbeH-2D252BUw2LaN-2D252B3ZKtxgeTbMI-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253D-5Fp-5FZf1fP8D9GtGtaEgIPjViLbHJLGv-5FRV32KAKGtMbE-2526m-253DDWXCbzopWFmoGrJlEJxX271evN-2D3OXDw-5F6jVvif2GYQ-2526s-253DtH1GdfR2NS0B72JeD9zo8rL34992xn5XPIxNnR6ivho-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257C0ff70c723695477bfa7608d7f8faaef8-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637251627879680569-26sdata-3DgVyS-252F-252FRsQeujG7Ub9WM5M91j95M961eRX68lzn7fjOw-253D-26reserved-3D0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ccFffx721N71hPpKcJGvJIqY9RM4gBTuzp9ir7rze5Q&m=HhQPGhhgXKdDhIQqUPWDLe1Gf2l4G9u0hmSG42KUZpA&s=vEVHggCd1ciSI9IF9AQ-JOt8Oj7_gIGDQhz44BtM3mE&e=>
This is from the printenv statement of my environment and should be the most up to date version, right? Is there something going on under the hood that I'm missing?
On Fri, May 15, 2020 at 1:51 PM Richard Jones <richard.t.jones at uconn.edu<mailto:richard.t.jones at uconn.edu>> wrote:
Colin and Thomas,
This is a known problem. You need to use an updated version of the ccdb geometry. Medium 11 is vacuum, defined as one of the elements of the periodic table. No wonder fluka does not know its properties. I changed vacuum to be defined as a mixture of hydrogen and nitrogen and updated the ccdb a few weeks back. The fix was back-dated to all runs since 2016. Please use an updated version of ccdb.
-Richard
On Fri, May 15, 2020 at 1:40 PM Colin Gleason <gleasonc at jlab.org<mailto:gleasonc at jlab.org>> wrote:
*Message sent from a system outside of UConn.*
I dumped the environments and they are located in this google doc:
https://docs.google.com/document/d/1VAbf8b9dhmzP0Ul0tgwmx5-BsXQyna7zlW6welDZlxA/edit?usp=sharing<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Fdocs.google.com-2D252Fdocument-2D252Fd-2D252F1VAbf8b9dhmzP0Ul0tgwmx5-2D2DBsXQyna7zlW6welDZlxA-2D252Fedit-2D253Fusp-2D253Dsharing-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257Cdf38f324a47d4c1f2feb08d7f8f70eff-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637251612309155975-2D26sdata-2D3D-2D252FVnrjMhc4DjZxFkEqWFQQoTAOQuKDmhXVveHpOzlxGI-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253D-5Fp-5FZf1fP8D9GtGtaEgIPjViLbHJLGv-5FRV32KAKGtMbE-2526m-253DJt5DGYkn6UBnm7aaKn4QomtiJpeYlkhgLkhrYEJRDDY-2526s-253DZkkchbqMAM01RQcYE3H3Xuh5OPnJqh6q5WnW-2DOMntu8-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257Cbc281fedbdd048fbeafc08d7f8f93aa4-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637251621642442399-26sdata-3DG-252FFkW54AwUgWs8-252BFCG30tImsUP1r-252B9ZV2nLksUkwX2A-253D-26reserved-3D0%26d%3DDwMFaQ%26c%3DCJqEzB1piLOyyvZjb8YUQw%26r%3D_p_Zf1fP8D9GtGtaEgIPjViLbHJLGv_RV32KAKGtMbE%26m%3DDWXCbzopWFmoGrJlEJxX271evN-3OXDw_6jVvif2GYQ%26s%3Do-arH23Py45M0qgBv0SbMXsqRFkP54gVvZIApu3-0G8%26e%3D&data=02%7C01%7Crichard.t.jones%40uconn.edu%7C0ff70c723695477bfa7608d7f8faaef8%7C17f1a87e2a254eaab9df9d439034b080%7C0%7C0%7C637251627879680569&sdata=Ml1t8IsBldbhSCCje7OkgLYtdJHs5vuZKSIPvmwJxI4%3D&reserved=0>
On Fri, May 15, 2020 at 12:49 PM Thomas Britton <tbritton at jlab.org<mailto:tbritton at jlab.org>> wrote:
Is there a material not found? Medium 11?
Haven’t seen that before. Can you modify the MakeMC to dump your environment right before the block of logging info. Compare interactive to the one run on the farm. My guess is there is some small difference between the two....
Thomas Britton
On May 15, 2020, at 12:45 PM, Colin Gleason <gleasonc at jlab.org<mailto:gleasonc at jlab.org>> wrote:
No crashes- it runs fine without any errors. Logs are located at /volatile/halld/home/gleasonc/pippimeta/Simulation/2017-01_4.19_G3/HADR4_b/log. Looking carefully at onne of the log files, I find:
settofg: RF reference plane set to 65.0000000 cm
10 events simulated
20 events simulated
30 events simulated
40 events simulated
50 events simulated
60 events simulated
70 events simulated
80 events simulated
90 events simulated
100 events simulated
200 events simulated
MICAP GTMED: GEANT Medium 11 not found ==> STOP
RUNNING MCSMEAR
0
10000
It looks like this is where things go wrong, as only ~250 of the ~2k generated events are getting processed.
This issue, or STOP, does not appear when I run interactively. Running interactively would show:
**** NUMBER OF EVENTS PROCESSED = 10000
**** RANDOM NUMBER GENERATOR AFTER LAST COMPLETE EVENT 2120846575 460615182
**** TIME TO PROCESS ONE EVENT IS = 0.0529 SECONDS
MZEND. Usage statistics for 2 dynamic stores.
Map of store 0 /GCBANK/
------------------------
Division Number of times
Kind Max-size Garb-coll.
Mode Position used allowed Wiped user auto Pushd Redcd
1 QDIV1 0 1 5203 0 4999968 0 0 0 0 0
2 QDIV2 1 1 2267094 1019031 4999968 10000 0 0 0 0
19 system 1 8 2341269 68180 4999968 0 0 0 0 0
20 Constant 1 2 4999968 2635035 4000000 0 0 141 150 0
Map of store 1 /PAWC/
------------------------
Division Number of times
Kind Max-size Garb-coll.
Mode Position used allowed Wiped user auto Pushd Redcd
1 QDIV1 0 1 2 0 4999585 0 0 0 0 0
2 QDIV2 1 1 4998185 1100 4999585 0 0 0 0 0
19 system 1 8 4998585 111 4999585 0 0 0 0 0
20 HIGZ 0 4 4998585 0 5000000 0 0 0 0 0
RZEND. called for RZFILE
***** Normal exit from Hall D GEANT *****
RUNNING MCSMEAR
2
10000
On Fri, May 15, 2020 at 12:34 PM Thomas Britton <tbritton at jlab.org<mailto:tbritton at jlab.org>> wrote:
This is tough. From MCwrapper’s POV it does nothing special when that card is in any given state. Do you mean it crashes? We’ll need some stack traces.
Thomas Britton
Staff Scientist, Scientific Computing
Jefferson Lab
From: Colin Gleason<mailto:gleasonc at jlab.org>
Sent: Friday, May 15, 2020 12:29 PM
To: Richard Jones<mailto:richard.t.jones at uconn.edu>; Thomas Britton<mailto:tbritton at jlab.org>
Cc: HallD Software Group<mailto:HallD-Offline at jlab.org>
Subject: Re: [EXTERNAL] auxilliary data file needed for running with HADR=4
Hi all,
As a follow up to this and to keep a record for others, I can not get the HADR 4 option to work when submitting jobs to the ifarm via MCWrapper batch=2. I get a reasonable output when I run the job interactively, as seen above. I'm using the same environment, but I'm guessing somehow the correct file is not being found in batch mode. I do not see anything that stands out in the log files. Thomas, any suggestions or ideas on why MCWrapper does not recognize this file? My CERNLIB is located at /group/halld/Software/builds/Linux_CentOS7.7-x86_64-gcc4.8.5/cernlib/2005/lib.
-Colin
On Wed, May 13, 2020 at 9:48 AM Colin Gleason <gleasonc at jlab.org<mailto:gleasonc at jlab.org>> wrote:
Thanks for the information, Richard. Sure enough, I do not have $CERNLIB defined using the standard build. As a workaround to what Sean suggested, and since I am usig C shell, I added
setenv CERNLIB /group/halld/Software/builds/Linux_CentOS7.7-x86_64-gcc4.8.5/cernlib/2005/lib
to my setup_gluex.csh file that gets sourced when calling my environment. When I tested with 1k generated events, I got 89 reconstructed. Before that, as in without flukaaf.dat, I was getting 8. This compares to the ~7% reconstructed events using HADR=1. I will rerun my simulation using this setup.
-Ccolin
On Wed, May 13, 2020 at 8:53 AM Richard Jones <richard.t.jones at uconn.edu<mailto:richard.t.jones at uconn.edu>> wrote:
Colin and all,
An input data file flukaaf.dat is a needed input file for running hdgeant with HADR=4. This file usually resides in $CERNLIB. If CERNLIB is defined on your system, then things should work. If not, then you should be sure to place a copy of flukaaf.dat in the directory from which you run hdgeant. A copy of this is found in the halld-sim repo in the halld software area, or on cvmfs, or on github at the link provided below. Take your pick, they should all be the same.
* /cvmfs/oasis.opensciencegrid.org/gluex/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/cernlib/2005/lib/flukaaf.dat<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Furldefense.proofpoint.com-2D252Fv2-2D252Furl-2D253Fu-2D253Dhttp-2D2D3A-2D5F-2D5Foasis.opensciencegrid.org-2D5Fgluex-2D5Fgroup-2D5Fhalld-2D5FSoftware-2D5Fbuilds-2D5FLinux-2D2D5FCentOS7-2D2D2Dx86-2D2D5F64-2D2D2Dgcc4.8.5-2D2D2Dcntr-2D5Fcernlib-2D5F2005-2D5Flib-2D5Fflukaaf.dat-2D2526d-2D253DDwMFaQ-2D2526c-2D253DCJqEzB1piLOyyvZjb8YUQw-2D2526r-2D253D-2D5Fp-2D5FZf1fP8D9GtGtaEgIPjViLbHJLGv-2D5FRV32KAKGtMbE-2D2526m-2D253D2yyxB2SUMQDNM7BpmwdbEPaT6IB-2D2D5Cn9bHhxQqrrQKg-2D2526s-2D253D63gpT7ZWLQ0R-2D2DWa6hSh-2D2DzaVffpwWqaywCNXMmII8XB8-2D2526e-2D253D-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257Cdf38f324a47d4c1f2feb08d7f8f70eff-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637251612309155975-2D26sdata-2D3DymFaf91wg76i5zfjuF4IG3UFf8TFvRcJG-2D252F2Z3I5TUHo-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253D-5Fp-5FZf1fP8D9GtGtaEgIPjViLbHJLGv-5FRV32KAKGtMbE-2526m-253DJt5DGYkn6UBnm7aaKn4QomtiJpeYlkhgLkhrYEJRDDY-2526s-253DIw81ro7omlGojsvsoW4h2aXqzP4QN2bGn-5FaOGkp-2DKPo-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257Cbc281fedbdd048fbeafc08d7f8f93aa4-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637251621642442399-26sdata-3D4-252FE3INuVI-252BalKdO4qzDqAiu4Co4-252FUdB6ntEI73ed-252FEQ-253D-26reserved-3D0%26d%3DDwMFaQ%26c%3DCJqEzB1piLOyyvZjb8YUQw%26r%3D_p_Zf1fP8D9GtGtaEgIPjViLbHJLGv_RV32KAKGtMbE%26m%3DDWXCbzopWFmoGrJlEJxX271evN-3OXDw_6jVvif2GYQ%26s%3DElsjRfIiOjNVpllX38BFYPw7wZbR1XCWdKy1tegVtw0%26e%3D&data=02%7C01%7Crichard.t.jones%40uconn.edu%7C0ff70c723695477bfa7608d7f8faaef8%7C17f1a87e2a254eaab9df9d439034b080%7C0%7C0%7C637251627879690560&sdata=9L5onG%2F9rbxnTivXDdVFjqRUNskVxnJNN70lhZNc0Ls%3D&reserved=0>
* /u/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5/halld_sim/halld_sim-4.13.0/src/programs/Simulation/HDGeant/flukaaf.dat
* https://github.com/JeffersonLab/halld_sim/blob/master/src/programs/Simulation/HDGeant/flukaaf.dat<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam10.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam10.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Furldefense.proofpoint.com-2D252Fv2-2D252Furl-2D253Fu-2D253Dhttps-2D2D3A-2D5F-2D5Fgithub.com-2D5FJeffersonLab-2D5Fhalld-2D2D5Fsim-2D5Fblob-2D5Fmaster-2D5Fsrc-2D5Fprograms-2D5FSimulation-2D5FHDGeant-2D5Fflukaaf.dat-2D2526d-2D253DDwMFaQ-2D2526c-2D253DCJqEzB1piLOyyvZjb8YUQw-2D2526r-2D253D-2D5Fp-2D5FZf1fP8D9GtGtaEgIPjViLbHJLGv-2D5FRV32KAKGtMbE-2D2526m-2D253D2yyxB2SUMQDNM7BpmwdbEPaT6IB-2D2D5Cn9bHhxQqrrQKg-2D2526s-2D253DS73Xf0DOqe72vg3PyfcLRlVQxN1qwLRuyhuySiXPaDU-2D2526e-2D253D-2D26data-2D3D02-2D257C01-2D257Crichard.t.jones-2D2540uconn.edu-2D257Cdf38f324a47d4c1f2feb08d7f8f70eff-2D257C17f1a87e2a254eaab9df9d439034b080-2D257C0-2D257C0-2D257C637251612309165971-2D26sdata-2D3DMPZkxeQDOT7m8wO0TNFRWPiO4eKPv3y13uYs0hS06c8-2D253D-2D26reserved-2D3D0-2526d-253DDwMFaQ-2526c-253DCJqEzB1piLOyyvZjb8YUQw-2526r-253D-5Fp-5FZf1fP8D9GtGtaEgIPjViLbHJLGv-5FRV32KAKGtMbE-2526m-253DJt5DGYkn6UBnm7aaKn4QomtiJpeYlkhgLkhrYEJRDDY-2526s-253DTPp-2D8YcvNmgwSJNfPMaOhdklQDLDCVqmfTX6JFQHnRM-2526e-253D-26data-3D02-257C01-257Crichard.t.jones-2540uconn.edu-257Cbc281fedbdd048fbeafc08d7f8f93aa4-257C17f1a87e2a254eaab9df9d439034b080-257C0-257C0-257C637251621642452393-26sdata-3DzmvYgFwO5chd4pkS9fRK4J10zPKmRKQ154aP7BB4gDM-253D-26reserved-3D0%26d%3DDwMFaQ%26c%3DCJqEzB1piLOyyvZjb8YUQw%26r%3D_p_Zf1fP8D9GtGtaEgIPjViLbHJLGv_RV32KAKGtMbE%26m%3DDWXCbzopWFmoGrJlEJxX271evN-3OXDw_6jVvif2GYQ%26s%3DFrrI40ELGsqvq8RBb2Vevt6wdl3t5bd3L7c4hZq0Cvg%26e%3D&data=02%7C01%7Crichard.t.jones%40uconn.edu%7C0ff70c723695477bfa7608d7f8faaef8%7C17f1a87e2a254eaab9df9d439034b080%7C0%7C0%7C637251627879700552&sdata=PQedyCBMaJWqqO4rVAZM5d3GQDlu0qxVFA9WlFFaCHY%3D&reserved=0>
-Richard Jones
--
Colin Gleason
Postdoctoral Fellow
Indiana University
Department of Physics
--
Colin Gleason
Postdoctoral Fellow
Indiana University
Department of Physics
--
Colin Gleason
Postdoctoral Fellow
Indiana University
Department of Physics
--
Colin Gleason
Postdoctoral Fellow
Indiana University
Department of Physics
--
Colin Gleason
Postdoctoral Fellow
Indiana University
Department of Physics
--
Colin Gleason
Postdoctoral Fellow
Indiana University
Department of Physics
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20200515/6ae8601c/attachment-0002.html>
More information about the Halld-offline
mailing list