<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
2nd try at sending this. Not sure what happened the first time...<br>
<div class="moz-forward-container"><br>
<br>
-------- Original Message --------
<table class="moz-email-headers-table" border="0" cellpadding="0"
cellspacing="0">
<tbody>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">Subject:
</th>
<td>Data Challenge Meeting Minutes, March 21, 2014</td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">Date: </th>
<td>Sat, 22 Mar 2014 01:30:17 -0400</td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">From: </th>
<td>Mark Ito <a class="moz-txt-link-rfc2396E" href="mailto:marki@jlab.org"><marki@jlab.org></a></td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">To: </th>
<td>GlueX Offline Software Email List
<a class="moz-txt-link-rfc2396E" href="mailto:halld-offline@jlab.org"><halld-offline@jlab.org></a></td>
</tr>
</tbody>
</table>
<br>
<br>
<pre>Folks,
Find the minutes below and at
<a class="moz-txt-link-freetext" href="https://halldweb1.jlab.org/wiki/index.php/GlueX_Data_Challenge_Meeting,_March_21,_2014#Minutes">https://halldweb1.jlab.org/wiki/index.php/GlueX_Data_Challenge_Meeting,_March_21,_2014#Minutes</a>
-- Mark
_________________________________________________________
GlueX Data Challenge Meeting, March 21, 2014
Minutes
Present:
* CMU: Paul Mattione
* IU: Kei Moriya
* JLab: Mark Ito, Simon Taylor
* MIT: Justin Stevens
* NU: Sean Dobbs
* UConn: Alex Barnes, Richard Jones
Announcements
Yesterday we froze the code and tagged version dc-2.7. This tag brings
in the change that Paul made to solve the reproducibility problem (see
below). Richard also discussed how the change we had requested, to
store the random number seed from bggen in its output for later use by
hdgeant, was ill-advised and thus was not included in this tag. He
wrote [36]email to the group yesterday on this. He noted that Eugene
Chudakov has replaced the original Pythia random number generator with
a more modern one. This means our simple practice of seeding the
generator with the file number will not get us into trouble with
tightly repeating loops.
We went over the [37]conditions page to make sure we are all in
agreement on details of the data challenge. We agreed that no changes
that affect functionality were needed, but Mark should clean up some of
the presentation on the page.
Review of Minutes from Last Time
We went through the [38]minutes of the March 14 meeting.
* Paul commented on [39]his fix to the reproducibility problem. At
root was a dependence on the order of objects stored in an STL map
which could change from one run on the data to the next. This
occurred in matching tracks in the CDC with those in the FDC.
* Mark reported that the return of nodes from the LQCD farm to the
physics batch farm is complete. SciComp is ready for us to start
submitting jobs.
* We noted that all action items listed at the end of the minutes had
in fact been acted on and resolved.
Activities at Production Sites
We reviewed activities at the various sites.
CMU
Paul has started a check-out and build of the latest tag. He reminded
us that the various conditions have different processing rates, in
terms of events per CPU-seconds. [40]His spreadsheet can help folks
plan their processing. The cluster will contribute 384 cores.
FSU
FSU had trouble connecting, understandable due to the evasive action we
took to establish communications for this meeting. [Added in press:
Aristeidis sent in a [41]report via email.]
JLab
* Last night, a few jobs were run through with the dc-2.7 tag. They
seem to have gone through.
* Plan is to run 24-hour jobs, with the number of events per job
depending on the EM background intensity.
* JLab will have roughly 1200 cores for DC2.
MIT
Justin led us through [42]his wiki page summarizing status at MIT. He
is using OpenStack in two contexts: the MIT Reuse Cluster and
FutureGrid (links on his wiki page). He expects to have 300 cores to
contribute.
OSG
Richard reported that the next step for OSG running is to set-up a
CernVM system to provide access to our software stack at all available
OSG sites. The system consists of a distributed set of read-only
replica servers.
MIT has access to an LHC tier-2 site in principle, we have not built up
any credit on that site and so those resources are probably not
available for this exercise.
The number of cores that we get from the OSG is uncertain due to the
nature of opportunistic running. Last time we peaked at 10,000 cores
and nothing that we know now says that this time should be any
different.
IU
Kei reported that there are significant computing resources in
Bloomington. These will likely be deployed for future large-scale
computing efforts.
Monitoring Discussion
We threw around some ideas on how to monitor quality of the results. We
will be creating ROOT histograms for each job. At a minimum we could do
spot checks on these histograms. Kei suggested processing the output of
hd_dump for each output file to get an event count and prove
analyzability. Mark proposed using hddm-xml and counting the number of
event tags found using simple tools grep and wc. Richard volunteered to
implement a program to do event counting at a lower level; it will be
more efficient. We all thought that this would be a nice tool to have.
[Added in press: this last change turned out to be a bit more complex
than anticipated. We will proceed without it; it may be added to the
mix later.]
File Distribution Discussion
We did not come to a general plan for how to ship output data around
for use in analysis. Paul pointed out that last time the OSG
contribution dwarfed everything else and there the mechanism is the
SRM. More thought will have to go into this.
Retrieved from
<a class="moz-txt-link-rfc2396E" href="https://halldweb1.jlab.org/wiki/index.php/GlueX_Data_Challenge_Meeting,_March_21,_2014">"https://halldweb1.jlab.org/wiki/index.php/GlueX_Data_Challenge_Meeting,_March_21,_2014"</a>
References
36.
<a class="moz-txt-link-freetext" href="https://mailman.jlab.org/pipermail/halld-offline/2014-March/001612.html">https://mailman.jlab.org/pipermail/halld-offline/2014-March/001612.html</a>
37.
<a class="moz-txt-link-freetext" href="https://halldweb1.jlab.org/data_challenge/02/conditions/data_challenge_2.html">https://halldweb1.jlab.org/data_challenge/02/conditions/data_challenge_2.html</a>
38.
<a class="moz-txt-link-freetext" href="https://halldweb1.jlab.org/wiki/index.php/GlueX_Data_Challenge_Meeting,_March_14,_2014#Minutes">https://halldweb1.jlab.org/wiki/index.php/GlueX_Data_Challenge_Meeting,_March_14,_2014#Minutes</a>
39.
<a class="moz-txt-link-freetext" href="https://mailman.jlab.org/pipermail/halld-offline/2014-March/001609.html">https://mailman.jlab.org/pipermail/halld-offline/2014-March/001609.html</a>
40.
<a class="moz-txt-link-freetext" href="https://mailman.jlab.org/pipermail/halld-offline/2014-March/001577.html">https://mailman.jlab.org/pipermail/halld-offline/2014-March/001577.html</a>
41.
<a class="moz-txt-link-freetext" href="https://mailman.jlab.org/pipermail/halld-offline/2014-March/001618.html">https://mailman.jlab.org/pipermail/halld-offline/2014-March/001618.html</a>
42.
<a class="moz-txt-link-freetext" href="https://halldweb1.jlab.org/wiki/index.php/MIT/FutureGrid_Data_Challenge_2_Production">https://halldweb1.jlab.org/wiki/index.php/MIT/FutureGrid_Data_Challenge_2_Production</a>
--
Mark M. Ito, Jefferson Lab, <a class="moz-txt-link-abbreviated" href="mailto:marki@jlab.org">marki@jlab.org</a>, (757)269-5295
</pre>
<br>
</div>
<br>
</body>
</html>