<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Dear Colleagues,</p>
<p>The post-processing for the monitoring launch over the 2018-01
data is complete.<br>
</p>
The usual monitoring plots can be found on the web pages:<br>
<a class="moz-txt-link-freetext" href="https://halldweb.jlab.org/data_monitoring/Plot_Browser.html">https://halldweb.jlab.org/data_monitoring/Plot_Browser.html</a><a
class="moz-txt-link-freetext"
href="https://halldweb.jlab.org/cgi-bin/data_monitoring/monitoring/runBrowser.py"><br>
https://halldweb.jlab.org/cgi-bin/data_monitoring/monitoring/runBrowser.py</a><a
class="moz-txt-link-freetext"
href="https://halldweb.jlab.org/cgi-bin/data_monitoring/monitoring/versionBrowser.py"><br>
https://halldweb.jlab.org/cgi-bin/data_monitoring/monitoring/versionBrowser.py</a>
<br>
<p>The merged histogram files can be found on the work disk at: <br>
<i class="moz-txt-slash"><span class="moz-txt-tag">/work/halld/data_monitoring/RunPeriod-2018-01/mon_ver17/rootfiles/</span></i></p>
<p>The REST files and several skims can be accessed here: <br>
<i class="moz-txt-slash"><span class="moz-txt-tag">/</span>cache/halld/offline_monitoring/RunPeriod-2018-01/ver17/REST<span
class="moz-txt-tag">/</span></i></p>
Best regards, <br>
<p> Alex </p>
<br>
<div class="moz-cite-prefix">On 9/6/2018 1:51 PM, David Lawrence
wrote:<br>
</div>
<blockquote cite="mid:8ECC9EDA-93D2-413F-8582-91D37E9426A8@jlab.org"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Hi All,
<div class=""><br class="">
</div>
<div class=""> This is a bit of a delayed announcement, but I’m
declaring the Spring 2018 monitoring pass</div>
<div class="">ver17 basically done. Swif2 reports 5462 of the 5495
jobs completed successfully. I was able</div>
<div class="">to resubmit some jobs and have them succeed on
subsequent tries, but the last 33 seem to</div>
<div class="">have fatal errors that will need to be explored
more. </div>
<div class=""><br class="">
</div>
<div class="">Some points of interest:</div>
<div class=""><br class="">
</div>
<div class="">- This was the first large scale test of processing
raw data offsite at NERSC</div>
<div class=""><br class="">
</div>
<div class="">- This launch included the first 10 files of all
runs with @is_2018production and @status_approved</div>
<div class=""><br class="">
</div>
<div class="">- It took approximately 12 days to run through them
after submitting them all to swif2 on Aug. 21st</div>
<div class=""><br class="">
</div>
<div class="">- The rate of processing them was not stable. See
red curve on top plot here:</div>
<div class=""><a moz-do-not-send="true"
href="https://halldweb.jlab.org/data_monitoring/launch_analysis/2018_01/launches/offline_monitoring_RunPeriod2018_01_ver17/"
class="">https://halldweb.jlab.org/data_monitoring/launch_analysis/2018_01/launches/offline_monitoring_RunPeriod2018_01_ver17/</a>)</div>
<div class="">This was partly due to multiple stages and ripple
effects between them:</div>
<div class="">tape->cache, cache->globus->NERSC,
NERSC->slurm and then all of that in reverse.</div>
<div class=""><br class="">
</div>
<div class="">- The default mechanism for submitting jobs resulted
in only about 300 jobs running simultaneously.</div>
<div class="">Compare this to the 600-700 we have seen on the JLab
farm.</div>
<div class="">We will be exploring using a reservation for the
next NERSC launch to see if we can</div>
<div class="">improve that.</div>
<div class=""><br class="">
</div>
<div class="">- We noticed only about 2Gbps steady rate copying to
NERSC with rare bursts >=7Gbps.</div>
<div class="">SciComp and Networking folks diagnosed this and have
worked to improve it. The biggest</div>
<div class="">gain is expected to come when they reboot the
external router later this month so that it will</div>
<div class="">support jumbo frames.</div>
<div class=""><br class="">
<div class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class="">- Output
can be found in
/mss/halld/offline_monitoring/RunPeriod-2018-01/ver17 or</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class="">the
associated cache directory.</div>
<div style="text-align: start; text-indent: 0px; word-wrap:
break-word; -webkit-nbsp-mode: space; line-break:
after-white-space;" class=""><br class="">
</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class="">Regards,</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class="">-David</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class=""><font
class="" color="#6a6a6a"><br class="">
</font></div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class=""><font
class="" color="#6a6a6a">-------------------------------------------------------------</font></div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class=""><font
class="" color="#6a6a6a">David Lawrence Ph.D.</font></div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class=""><font
class="" color="#6a6a6a">Staff Scientist, Thomas Jefferson
National Accelerator Facility<br class="">
Newport News, VA</font></div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class=""><font
class="" color="#6a6a6a"><a moz-do-not-send="true"
href="mailto:davidl@jlab.org" class="">davidl@jlab.org</a><br
class="">
(757) 269-5567 W<br class="">
(757) 746-6697 C</font><br class="">
<br class="">
</div>
</div>
<br class="">
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
GlueX-Collaboration mailing list
<a class="moz-txt-link-abbreviated" href="mailto:GlueX-Collaboration@jlab.org">GlueX-Collaboration@jlab.org</a>
<a class="moz-txt-link-freetext" href="https://mailman.jlab.org/mailman/listinfo/gluex-collaboration">https://mailman.jlab.org/mailman/listinfo/gluex-collaboration</a>
</pre>
</blockquote>
<br>
</body>
</html>