<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">I agree with both Curtis and Richard. In fact GLUEX is in very good shape regarding data challenges. What Chip suggested, and I agree with him, is that the lab should set some expectation, the discussion was then what that expectation should be. While using the OSG is a good resource to use for this sort of thing the expectation, I think from both the lab and the collaboration, is that at least the first pass reconstruction and analysis will happen here. In that case it is important to run the real code on the actual cluster (or as close to it as we can get) where it will finally run. In that way we shake out any issues involving the interaction of GLUEX code not only with the cluster software and hardware but also with other users of the system. The scope of the tests and the schedule are intended to give us (ENP and IT) time to adjust what hardware we buy and the cluster management software to solve any issues found, for example density of computing cores in one box, memory, local disk, network bandwidth etc.<div><br></div><div>As such then the tests that we proposed are the minimum that we would expect you to do and, being reasonable folks, I expect that they are the sort of thing that you have already considered.</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Regards to all,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Graham</div><div><br><div><br><div><div>On Oct 31, 2013, at 10:37 am, Richard Jones <<a href="mailto:richard.t.jones@uconn.edu">richard.t.jones@uconn.edu</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">GlueX offline group,<div><br></div><div>In stating our goals for the next data challenge, we need to satisfy whatever requirements are provided to us by the Jlab 12 GeV project. However, let's not forget that simulations on the OSG are an integral part of our computing plan, and that to reach our scientific goals in the most optimum way possible, we need an integrated system for offline processing that includes both on-site and off-site computing. Let's make sure that our goals for the next data challenge are directed primarily by our own internal criteria as a collaboration, and are not merely dictated to us by the expectations of the lab. Those should serve as a kind of lower bound on what we expect to achieve.</div>
<div><br></div><div>-Richard J.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Oct 31, 2013 at 9:40 AM, Mark Ito <<a href="mailto:marki@jlab.org">marki@jlab.org</a>> wrote:<br>
<blockquote class="gmail_quote">
<div bgcolor="#FFFFFF" text="#000000">
Proposal from Graham and Chip, for your information and comment.<br>
<div><br>
<br>
-------- Original Message --------
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<th align="RIGHT" nowrap="" valign="BASELINE">Subject:
</th>
<td>Data challenge requirements</td>
</tr>
<tr>
<th align="RIGHT" nowrap="" valign="BASELINE">Date: </th>
<td>Wed, 30 Oct 2013 13:05:59 -0400</td>
</tr>
<tr>
<th align="RIGHT" nowrap="" valign="BASELINE">From: </th>
<td>Heyes Graham <a href="mailto:heyes@jlab.org"><heyes@jlab.org></a></td>
</tr>
<tr>
<th align="RIGHT" nowrap="" valign="BASELINE">To: </th>
<td>Ito Mark <a href="mailto:marki@jlab.org"><marki@jlab.org></a>, Lawrence David
<a href="mailto:davidl@jlab.org"><davidl@jlab.org></a>, Weygand Dennis
<a href="mailto:weygand@jlab.org"><weygand@jlab.org></a>, Veronique Ziegler
<a href="mailto:ziegler@jlab.org"><ziegler@jlab.org></a></td>
</tr>
<tr>
<th align="RIGHT" nowrap="" valign="BASELINE">CC: </th>
<td>Rolf Ent <a href="mailto:ent@jlab.org"><ent@jlab.org></a>, Rossi Patrizia
<a href="mailto:rossi@jlab.org"><rossi@jlab.org></a>, Watson Chip
<a href="mailto:watson@jlab.org"><watson@jlab.org></a></td>
</tr>
</tbody>
</table>
<br>
<br>
<div>Dear Offline folks,</div>
<div> Chip
and I had a discussion this morning about what is an acceptable
scope and schedule for data challenge tests of your analysis
frameworks. The definitive tests should be done at JLab using
the systems on which the final analysis will be done, i.e. the
ENP analysis cluster (Farm). The suggested scope and schedule is
based upon the dates that you have given for the first major
bumps in computing requirements, 5000 cores for GLUEX in Q2 of
2015 and 2400 for CLAS12 in Q2 of 2016:</div>
<div>
<ul>
<li> 10%
of final scale 24 months in advance.</li>
<li> 25%
of final scale 18 months in advance.</li>
<li> 50%
of final scale 12 months in advance.</li>
</ul>
<div>Each test should last for 2-4 days depending upon how many
problems are encountered. The larger scale tests will require
significant resources which Chip proposes that he will loan to
us from the LQCD clusters so as not to significantly disturb
other Farm users. He proposes moving the resources to us on a
Monday then moving them back to LQCD on a Friday giving us up
to four work days for testing.</div>
</div>
<div><br>
</div>
<div>I spoke with Mark this morning and he agreed that this is a
reasonable schedule and that GLUEX have already met the 24 month
test requirement. However the 25% scale test will be due in the
next few months.</div>
<div><br>
</div>
<div>Chip and I both think that this is a reasonable schedule and
not difficult to meet. We would like you to consider it as you
prepare presentations for the review.</div>
<div><br>
</div>
<div>Regards to all,</div>
<div> Graham</div>
<div><br>
</div>
<div><br>
</div>
<br>
</div>
<br>
</div>
<br>_______________________________________________<br>
Halld-offline mailing list<br>
<a href="mailto:Halld-offline@jlab.org">Halld-offline@jlab.org</a><br>
<a href="https://mailman.jlab.org/mailman/listinfo/halld-offline">https://mailman.jlab.org/mailman/listinfo/halld-offline</a><br></blockquote></div><br></div>
_______________________________________________<br>Halld-offline mailing list<br><a href="mailto:Halld-offline@jlab.org">Halld-offline@jlab.org</a><br>https://mailman.jlab.org/mailman/listinfo/halld-offline</blockquote></div><br></div></div></body></html>