<div dir="auto">Jeremy has also profiled the recon before so if Maurik can't do it, I'm sure he can. </div><div class="gmail_extra"><br><div class="gmail_quote">On May 22, 2017 11:46 AM, "Rafayel Paremuzyan" <<a href="mailto:rafopar@jlab.org">rafopar@jlab.org</a>> wrote:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Alessandra and all,<br>
<br>
Yes clearly I see that pattern as well,<br>
going to the faster machine (pumpkin1), as Maurik suggested,<br>
recon speed increased, but 2016 recon was about x2 slower than the 2015 recon speed.<br>
I also tried to with -Xms=4000m -Xmx=4000m options, and without that options,<br>
but I didn't notice noticeable speed improvement.<br>
<br>
Another thing that is *concering* is I run<br>
same jar on batch farms, centos6. and centos7, and<br>
can't get same speed as we got during the pass0.<br>
Note, jar is the same, detector is the same, run number and file number is the same.<br>
Difference is the time when it started to tun (Oct 19 2016 vs May 21 2017)<br>
<br>
If someone is interested to look at job log files<br>
The log file that I run yesterday: /lustre/expphy/work/hallb/hps/<wbr>data/physrun2016/tpass1/logs/h<wbr>ps_008054.27_pass0XML.err<br>
The log file that run on Oct 19 2016 for pass0 /lustre/expphy/work/hallb/hps/<wbr>data/physrun2016/pass0/logs/hp<wbr>s_008054.27_R3.9.err<br>
<br>
I have never done profiling with java.<br>
I know Maurik is in workshop, he might not have time to do this,<br>
if someone else is set up doing this, it will probably be useful.<br>
<br>
Also, I noticed /group/hps is full<br>
<br>
Has anyone recently put some data there?<br>
<br>
Rafo<div class="elided-text"><br>
<br>
<br>
On 05/22/2017 04:04 AM, Alessandra Filippi wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="elided-text">
Hi Maurik, Rafo and all,<br>
I think that the different Java VM could impact on the absolute speed comparing different machines and jar versions compiled in different times... but I am running the same jar (hps-java 3.10, compiled afresh at slac on rhel6-64, jvm 1.7.0) on 2015 and 2016 data, and the factor ~2 of speed decrease for newer data is striking (whichever geometry version).<br>
About garbage collection, I don't use any flag in compilation, so it acts in default mode in both cases.<br>
<br>
Are 2016 data affected by more noisy hits that could extend the reconstruction time over testing all different strategies, to your knowledge?<br>
cheers<br>
Alessandra<br>
<br>
<br>
<br>
<br>
<br>
<br>
On Sun, 21 May 2017, Maurik Holtrop wrote:<br>
<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="elided-text">
Hello Rafo,<br>
One thing that probably is different between the last time we ran with the 3.8 jar and now<br>
is a different version of the Java VM. It could well be that the newer version of Java is<br>
not faster. Also, it is tricky to compare Endeavour with the Jlab farm computers. They are<br>
probably not equivalent in speed. At UNH, Pumpkin has the more modern processors, whereas<br>
Endeavour is now ~5 years old.<br>
<br>
Best,<br>
Maurik<br>
<br>
On May 21, 2017, at 6:54 AM, Rafayel Paremuzyan <<a href="mailto:rafopar@jlab.org" target="_blank">rafopar@jlab.org</a>> wrote:<br>
<br>
Hi Alessandra, Norman, all<br>
<br>
thank you for replay and your tests.<br>
<br>
I tested both 2015 and 2016 data using v4-4 detector on UNH computers.<br>
I have use 3.8 JAR (the jar for 2015 pass6). 3.9 JAR (the jar for 2016 pass0 recon),<br>
and the NEW jar v051717 (the newest jar tag is v051717)<br>
<br>
Ok, I also noticed that recon of 2015 data is faster that 2016 data.<br>
Also seems the new jar is 20% slower than the 3.9 jar for 2016 data, and about 60%<br>
slower for 2015 data.<br>
now recon speed is about 2.55 EV/S for 2015 data, This is too slow<br>
it cause more than 40h for a single file.<br>
<br>
Ths is summary of code speeds with different jar files<br>
V4-4 Detector, UNH (endeavour), 5K events are reconstructed<br>
<br>
Events per second<br>
Events per second<br>
Events per second<br>
<br>
3.8JAR (2015 recon jar)<br>
3.9JAR, 2016 pass0 recon jar<br>
v051717JAR, jar vor tpass1<br>
2015 Data 5772, file 20<br>
5.07<br>
5.19<br>
3.157<br>
2016 Data file 25<br>
<br>
3.11<br>
2.53<br>
<br>
*However* I looked into job Wall times for pass0 recon.<br>
The recon speed is more than 7.4 Events/sec, which is about x3 faster than with the<br>
new JAR.<br>
<br>
I again checked *same 3.9 jar*, which is slower again.<br>
I don't know why now the code speed is so low!<br>
<br>
<br>
Norman, I have tried the "-DdisableSvtAlignmentConstant<wbr>s", but it didn't work<br>
<br>
=================The command===============<br>
java -XX:+UseSerialGC -cp hps-distribution-3.11-v051717-<wbr>bin.jar<br>
org.hps.evio.EvioToLcio -x /org/hps/steering/recon/Physic<wbr>sRun2016FullRecon.lcsim -r<br>
-d HPS-PhysicsRun2016-v5-3-fieldm<wbr>ap_globalAlign -R 7796 -DoutputFile=out_7796_0<br>
-DdisableSvtAlignmentConstants hps_007796.evio.25 -n 10000<br>
<br>
============The error backtrache============<br>
2017-05-21 00:45:39 [CONFIG] org.hps.evio.EvioToLcio parse :: using steering<br>
resource /org/hps/steering/recon/Physic<wbr>sRun2016FullRecon.lcsim<br>
2017-05-21 00:45:39 [CONFIG] org.hps.evio.EvioToLcio parse :: set max events to<br>
10000<br>
2017-05-21 00:45:48 [INFO] org.hps.rundb.RunManager <init> :: ConnectionParameters {<br>
database: hps_run_db_v2, hostname: <a href="http://hpsdb.jlab.org" rel="noreferrer" target="_blank">hpsdb.jlab.org</a>, password: darkphoton, port: 3306,<br>
user: hpsuser }<br>
2017-05-21 00:45:48 [CONFIG] org.lcsim.job.JobControlManage<wbr>r addVariableDefinition<br>
:: outputFile = out_7796_0<br>
2017-05-21 00:45:48 [CONFIG] org.hps.evio.EvioToLcio parse :: set steering variable:<br>
outputFile=out_7796_0<br>
2017-05-21 00:45:48 [SEVERE] org.hps.evio.EvioToLcio parse :: bad variable format:<br>
disableSvtAlignmentConstants<br>
java.lang.IllegalArgumentExcep<wbr>tion: Bad variable format:<br>
disableSvtAlignmentConstants<br>
at org.hps.evio.EvioToLcio.parse(<wbr>EvioToLcio.java:393)<br>
at org.hps.evio.EvioToLcio.main(E<wbr>vioToLcio.java:97)<br>
<br>
Exception in thread "main" java.lang.IllegalArgumentExcep<wbr>tion: Bad variable format:<br>
disableSvtAlignmentConstants<br>
at org.hps.evio.EvioToLcio.parse(<wbr>EvioToLcio.java:393)<br>
at org.hps.evio.EvioToLcio.main(E<wbr>vioToLcio.java:97)<br>
<br>
Rafo<br>
<br>
<br>
<br>
<br>
<br>
<br>
On 05/20/2017 06:17 AM, Alessandra Filippi wrote:<br>
Hi Rafo, all,<br>
I also noticed that the reconstruction for 2016 data is about twice as<br>
slow as compared to 2015 (whichever geometry and reconstruction<br>
version).<br>
This happens when I run the aligned geometry as well as the "current"<br>
one (v5.0), and the geometry taken from the db as well (the result is<br>
the same as v5.0). I did not make any test with v4.4, though - actually<br>
as far as svt alignment is concerned it should be the same as v5.0. Can<br>
you please try and make the same short test with the newest jar with<br>
v4.4?<br>
This happens to me both with hps-java 5.10 and 5.11 (not the most<br>
updated one).<br>
<br>
I would be surprised if it could be something connected to the<br>
alignment, unless for some reason new positions and harder tracks<br>
trigger some long loops in the reconstruction. But this happens (to me)<br>
also with the<br>
standard geometry, so a check with the one used with pass0 (that should<br>
however be equivalent to v5.0) could at least help to rule out, or blame<br>
on, the alignment step.<br>
Thanks, cheers<br>
Alessandra<br>
<br>
<br>
ps. make also sure that the correct fieldmap is called in all the<br>
compact files - you never know!<br>
<br>
<br>
<br>
On Fri, 19 May 2017, Rafayel Paremuzyan wrote:<br>
<br>
Hi All,<br>
<br>
During the testing the recon for test pass1,<br>
I noticed the recon time is more than x2 longer wrt pass0<br>
recon time.<br>
<br>
To demonstrate it<br>
I submit 3 simple jobs with 10K events to reconstruct, with<br>
new pass1 xml<br>
file (this has the new jar v051717, and the new detector<br>
HPS-PhysicsRun2016-v5-3-fieldm<wbr>ap_globalAlign),<br>
and the old pass0 xml file (pass0 jar release 3.9, and the<br>
detector<br>
HPS-PhysicsRun2016-Nominal-v4-<wbr>4-fieldmap)<br>
<br>
Below is a printout from the jobs with a new JAR, v051717,<br>
the average time<br>
for 1000 events is more than 7 minutes<br>
===================== LOG from the v051717 JAR<br>
==============================<br>
2017-05-19 09:36:51 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10614074 with sequence 0<br>
2017-05-19 09:43:13 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10615074 with sequence 1000<br>
2017-05-19 09:49:18 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10616074 with sequence 2000<br>
2017-05-19 09:55:54 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10617074 with sequence 3000<br>
2017-05-19 10:02:55 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10618074 with sequence 4000<br>
2017-05-19 10:09:57 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10619074 with sequence 5000<br>
2017-05-19 10:16:13 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10620074 with sequence 6000<br>
2017-05-19 10:25:20 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10621074 with sequence 7000<br>
2017-05-19 10:32:56 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10622074 with sequence 8000<br>
2017-05-19 10:36:19 [WARNING]<br>
org.hps.recon.tracking.Tracker<wbr>ReconDriver<br>
process :: Discarding track with bad HelicalTrackHit<br>
(correction distance<br>
0.000000, chisq penalty 0.000000)<br>
2017-05-19 10:42:03 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10623074 with sequence 9000<br>
2017-05-19 10:47:44 [INFO] org.hps.evio.EvioToLcio run ::<br>
maxEvents 10000<br>
was reached<br>
2017-05-19 10:47:44 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
endOfData ::<br>
10000 events processed in job.<br>
2017-05-19 10:47:44 [INFO] org.hps.evio.EvioToLcio run ::<br>
Job finished<br>
successfully!<br>
<br>
<br>
And below is the Job log info from the pass0 jar. The<br>
average time for 1000<br>
events is less than 3 minutes<br>
===================== LOG from the 3.9 release JAR<br>
==============================<br>
2017-05-19 13:19:46 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10614074 with sequence 0<br>
2017-05-19 13:23:36 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10615074 with sequence 1000<br>
2017-05-19 13:27:03 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10616074 with sequence 2000<br>
2017-05-19 13:30:40 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10617074 with sequence 3000<br>
2017-05-19 13:34:20 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10618074 with sequence 4000<br>
2017-05-19 13:38:11 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10619074 with sequence 5000<br>
2017-05-19 13:41:43 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10620074 with sequence 6000<br>
2017-05-19 13:45:54 [WARNING]<br>
org.hps.recon.tracking.Tracker<wbr>ReconDriver<br>
process :: Discarding track with bad HelicalTrackHit<br>
(correction distance<br>
0.000000, chisq penalty 0.000000)<br>
2017-05-19 13:46:05 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10621074 with sequence 7000<br>
2017-05-19 13:50:08 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10622074 with sequence 8000<br>
2017-05-19 13:55:03 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
process :: Event<br>
10623074 with sequence 9000<br>
2017-05-19 13:58:27 [INFO] org.hps.evio.EvioToLcio run ::<br>
maxEvents 10000<br>
was reached<br>
2017-05-19 13:58:27 [INFO] org.lcsim.job.EventMarkerDrive<wbr>r<br>
endOfData ::<br>
10000 events processed in job.<br>
2017-05-19 13:58:27 [INFO] org.hps.evio.EvioToLcio run ::<br>
Job finished<br>
successfully!<br>
<br>
I also tried to do reconstruction by myself interactively,<br>
but I am getting<br>
error below,<br>
<br>
The command<br>
/apps/scicomp/java/jdk1.7/bin/<wbr>java -XX:+UseSerialGC -cp<br>
hps-distribution-3.9-bin.jar org.hps.evio.EvioToLcio -x<br>
/org/hps/steering/recon/Physic<wbr>sRun2016FullRecon.lcsim -r -d<br>
HPS-PhysicsRun2016-v5-3-fieldm<wbr>ap_globalAlign -R 7796<br>
-DoutputFile=out_7796_0<br>
hps_007796.evio.0 -n 10000<br>
<br>
The Error traceback<br>
017-05-19 14:58:44 [CONFIG] org.hps.evio.EvioToLcio parse ::<br>
using steering<br>
resource<br>
/org/hps/steering/recon/Physic<wbr>sRun2016FullRecon.lcsim<br>
2017-05-19 14:58:44 [CONFIG] org.hps.evio.EvioToLcio parse<br>
:: set max events<br>
to 10000<br>
2017-05-19 14:58:45 [CONFIG]<br>
org.lcsim.job.JobControlManage<wbr>r<br>
addVariableDefinition :: outputFile = out_7796_0<br>
2017-05-19 14:58:45 [CONFIG] org.hps.evio.EvioToLcio parse<br>
:: set steering<br>
variable: outputFile=out_7796_0<br>
2017-05-19 14:58:45 [CONFIG] org.lcsim.job.JobControlManage<wbr>r<br>
initializeLoop<br>
:: initializing LCSim loop<br>
2017-05-19 14:58:45 [CONFIG] org.lcsim.job.JobControlManage<wbr>r<br>
initializeLoop<br>
:: Event marker printing disabled.<br>
2017-05-19 14:58:45 [INFO]<br>
org.hps.conditions.database.Da<wbr>tabaseConditionsManager<br>
resetInstance ::<br>
DatabaseConditionsManager instance is reset<br>
Exception in thread "main" java.lang.UnsatisfiedLinkError<wbr>:<br>
/u/apps/scicomp/java/jdk1.7.0_<wbr>75/jre/lib/i386/xawt/libmawt.s<wbr>o:<br>
libXext.so.6:<br>
cannot open shared object file: No such file or directory<br>
at java.lang.ClassLoader$NativeLi<wbr>brary.load(Native<br>
Method)<br>
at<br>
java.lang.ClassLoader.loadLibr<wbr>ary1(ClassLoader.java:1965)<br>
at<br>
java.lang.ClassLoader.loadLibr<wbr>ary0(ClassLoader.java:1890)<br>
at<br>
java.lang.ClassLoader.loadLibr<wbr>ary(ClassLoader.java:1851)<br>
at java.lang.Runtime.load0(Runtim<wbr>e.java:795)<br>
at java.lang.System.load(System.j<wbr>ava:1062)<br>
at java.lang.ClassLoader$NativeLi<wbr>brary.load(Native<br>
Method)<br>
at<br>
java.lang.ClassLoader.loadLibr<wbr>ary1(ClassLoader.java:1965)<br>
at<br>
java.lang.ClassLoader.loadLibr<wbr>ary0(ClassLoader.java:1890)<br>
at<br>
java.lang.ClassLoader.loadLibr<wbr>ary(ClassLoader.java:1872)<br>
at java.lang.Runtime.loadLibrary0<wbr>(Runtime.java:849)<br>
at java.lang.System.loadLibrary(S<wbr>ystem.java:1088)<br>
at<br>
sun.security.action.LoadLibrar<wbr>yAction.run(LoadLibraryAction.<wbr>java:67)<br>
at<br>
sun.security.action.LoadLibrar<wbr>yAction.run(LoadLibraryAction.<wbr>java:47)<br>
at<br>
java.security.AccessController<wbr>.doPrivileged(Native Method)<br>
at<br>
java.awt.Toolkit.loadLibraries<wbr>(Toolkit.java:1653)<br>
at java.awt.Toolkit.<clinit>(Tool<wbr>kit.java:1682)<br>
at java.awt.Component.<clinit>(Co<wbr>mponent.java:595)<br>
at org.lcsim.util.aida.AIDA.<init<wbr>>(AIDA.java:68)<br>
at<br>
org.lcsim.util.aida.AIDA.defau<wbr>ltInstance(AIDA.java:53)<br>
at<br>
org.hps.evio.RfFitterDriver.<i<wbr>nit>(RfFitterDriver.java:31)<br>
at<br>
sun.reflect.NativeConstructorA<wbr>ccessorImpl.newInstance0(<wbr>Native<br>
Method)<br>
atsun.reflect.NativeConstructo<wbr>rAccessorImpl.newInstance(Nati<wbr>veConstructorAcce<br>
ssorImpl.java:57)<br>
atsun.reflect.DelegatingConstr<wbr>uctorAccessorImpl.newInstance(<wbr>DelegatingConstru<br>
ctorAccessorImpl.java:45)<br>
at<br>
java.lang.reflect.Constructor.<wbr>newInstance(Constructor.java:5<wbr>26)<br>
at java.lang.Class.newInstance(Cl<wbr>ass.java:379)<br>
at<br>
org.lcsim.job.JobControlManage<wbr>r.setupDrivers(JobControlManag<wbr>er.java:1199) <br>
at<br>
org.hps.job.JobManager.setupDr<wbr>ivers(JobManager.java:82)<br>
at<br>
org.lcsim.job.JobControlManage<wbr>r.setup(JobControlManager.<wbr>java:1052)<br>
at<br>
org.lcsim.job.JobControlManage<wbr>r.setup(JobControlManager.<wbr>java:1110)<br>
at<br>
org.hps.evio.EvioToLcio.parse(<wbr>EvioToLcio.java:407)<br>
at org.hps.evio.EvioToLcio.main(E<wbr>vioToLcio.java:97)<br>
<br>
<br>
<br>
I see this library libXext.so.6: in /usr/lib64, but not in<br>
/usr/lib,<br>
when I put /usr/lib64 in my LD_LIBRARY_PATH, then it<br>
complaines again (see<br>
below)<br>
<br>
Exception in thread "main" java.lang.UnsatisfiedLinkError<wbr>:<br>
/u/apps/scicomp/java/jdk1.7.0_<wbr>75/jre/lib/i386/xawt/libmawt.s<wbr>o:<br>
libXext.so.6:<br>
wrong ELF class: ELFCLASS64<br>
<br>
I would appreciate, if I get some help on running the<br>
reconstruction<br>
interactively, then I could look more closely into logs<br>
of the old, and new JAR files.<br>
<br>
Rafo<br>
<br>
<br>
______________________________<wbr>______________________________<wbr>________________ <br>
<br>
Use REPLY-ALL to reply to list<br>
<br>
To unsubscribe from the HPS-SOFTWARE list, click the<br>
following link:<br>
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__listser<wbr>v.slac.stanford.edu_cgi-2Dbin</a> <br>
_wa-3FSUBED1-3DHPS-2DSOFTWARE-<wbr>26A-3D1&d=DwIDaQ&c=lz9TcOasaIN<wbr>aaC3U7FbMev2lsutwpI4--09aP8Lu <br>
18s&r=0HDJrGO9TZQTE97J9Abt2A&m<wbr>=xnbGP5VHYWRAQRWRksVgMnYvBkXWI<wbr>4roLxztdJ0Tp9I&s=ppNYedSrn5DP <br>
aIZZJgRZu8tBDeSjroqbj_PoevFoFp<wbr>I&e=<br>
<br>
<br>
<br>
##############################<wbr>##############################<wbr>############<br>
Use REPLY-ALL to reply to list<br>
<br>
To unsubscribe from the HPS-SOFTWARE list, click the following link:<br>
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__listser<wbr>v.slac.stanford.edu_cgi-2Dbin</a> <br>
_wa-3FSUBED1-3DHPS-2DSOFTWARE-<wbr>26A-3D1&d=DwIDaQ&c=lz9TcOasaIN<wbr>aaC3U7FbMev2lsutwpI4--09aP8Lu <br>
18s&r=0HDJrGO9TZQTE97J9Abt2A&m<wbr>=xnbGP5VHYWRAQRWRksVgMnYvBkXWI<wbr>4roLxztdJ0Tp9I&s=ppNYedSrn5DP <br>
aIZZJgRZu8tBDeSjroqbj_PoevFoFp<wbr>I&e=<br>
<br>
<br>
<br>
______________________________<wbr>______________________________<wbr>______________________________ <br>
<br>
Use REPLY-ALL to reply to list<br>
<br>
To unsubscribe from the HPS-SOFTWARE list, click the following link:<br>
</div><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DHPS-2DSOFTWARE-26A-3D1&d=DwIDbA&c=lz9TcOasaINaaC3U7FbMev2lsutwpI4--09aP8Lu18s&r=0HDJrGO9TZQTE97J9Abt2A&m=nIEvNWJSNxvmECWKfMrVKMTZKNdym3zsAls7WG1fdfU&s=cI82nYRtYBzgLs1l5g3anHrSt0kPk4B98oUzllu5KfQ&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__listser<wbr>v.slac.stanford.edu_cgi-2Dbin_<wbr>wa-3FSUBED1-3DHPS-2DSOFTWARE-<wbr>26A-3D1&d=DwIDbA&c=lz9TcOasaIN<wbr>aaC3U7FbMev2lsutwpI4--<wbr>09aP8Lu18s&r=0HDJrGO9TZQTE97J9<wbr>Abt2A&m=nIEvNWJSNxvmECWKfMrVKM<wbr>TZKNdym3zsAls7WG1fdfU&s=cI82nY<wbr>RtYBzgLs1l5g3anHrSt0kPk4B98oUz<wbr>llu5KfQ&e=</a> <br>
<br>
<br>
<br>
</blockquote><div class="quoted-text">
<br>
##############################<wbr>##############################<wbr>############<br>
Use REPLY-ALL to reply to list<br>
<br>
To unsubscribe from the HPS-SOFTWARE list, click the following link:<br>
</div><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DHPS-2DSOFTWARE-26A-3D1&d=DwIDbA&c=lz9TcOasaINaaC3U7FbMev2lsutwpI4--09aP8Lu18s&r=0HDJrGO9TZQTE97J9Abt2A&m=nIEvNWJSNxvmECWKfMrVKMTZKNdym3zsAls7WG1fdfU&s=cI82nYRtYBzgLs1l5g3anHrSt0kPk4B98oUzllu5KfQ&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__listser<wbr>v.slac.stanford.edu_cgi-2Dbin_<wbr>wa-3FSUBED1-3DHPS-2DSOFTWARE-<wbr>26A-3D1&d=DwIDbA&c=lz9TcOasaIN<wbr>aaC3U7FbMev2lsutwpI4--<wbr>09aP8Lu18s&r=0HDJrGO9TZQTE97J9<wbr>Abt2A&m=nIEvNWJSNxvmECWKfMrVKM<wbr>TZKNdym3zsAls7WG1fdfU&s=cI82nY<wbr>RtYBzgLs1l5g3anHrSt0kPk4B98oUz<wbr>llu5KfQ&e=</a> <br>
</blockquote>
<br>
<br>
______________________________<wbr>_________________<br>
Hps-analysis mailing list<br>
<a href="mailto:Hps-analysis@jlab.org" target="_blank">Hps-analysis@jlab.org</a><br>
<a href="https://mailman.jlab.org/mailman/listinfo/hps-analysis" rel="noreferrer" target="_blank">https://mailman.jlab.org/mailm<wbr>an/listinfo/hps-analysis</a><br>
</blockquote></div><br></div>