[Halld-offline] Offline Software Meeting Minutes, September 14, 2016

[Mark Ito]

Friends,

Please find the minutes at 
https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_September_14,_2016#Minutes 
and reproduced below.

   -- Mark
_________________________


    GlueX Offline Meeting, September 14, 2016, Minutes

Present:

  * *CMU*: Naomi Jarvis, Curtis Meyer, Mike Staib
  * *JLab*: Alexander Austregesilo, Brad Cannon, Mark Ito (chair), David
    Lawrence, Paul Mattione, Nathan Sparks, Simon Taylor, Beni Zihlmann
  * *NU*: Sean Dobbs


      Announcements

 1. Fix to JEventSource_EVIOpp crashes on PS skims
    <https://mailman.jlab.org/pipermail/halld-offline/2016-September/002449.html>.
    We looked over Richard Jones's bug fix to get around anomalies in
    the pair spectrometer skims. Sean has reviewed the problem and the
    change. With recent changes to the skimming software, it is possible
    that the anomalies have disappeared.
 2. sim-recon version 2.5.0
    <https://github.com/JeffersonLab/sim-recon/releases/tag/2.5.0>. See
    the release notes in the link for details. At JLab this release has
    been build against ROOT 6.06.08. A build against ROOT 5 is coming soon.


      Review of minutes from the last meeting

We look over the minutes from August 31 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_August_31,_2016#Minutes>. 
We mainly discussed the white paper 
<https://halldweb.jlab.org/talks/2016/Scientific%20Computing%20Allocations%20Process.pdf> 
describing a proposal for review of large projects on the JLab farm. 
Mark again encouraged us to look it over.


      Launches

Alex reported progress on the reconstruction launch and its 
corresponding analysis launch.


        Reconstruction Launch

The launch was scheduled to begin Friday, two weeks ago, but due to 
last-minute software changes and associated problems, it was delayed 
until Saturday. After Batch 1 had retrieved all of its files Alex 
started Batch 2, to keep the tape library busy.

Processing of these two batches took 9 days total (ending this past 
Monday). The memory footprint was a bit larger than in version 1. The 
requested amount had to be increased. Each job was run with 14 threads.

We looked at the Launch webpage 
<https://halldweb.jlab.org/data_monitoring/recon/summary_swif_output_recon_2016-02_ver02_batch01.html> 
which shows various statistics on the jobs. The success rate was 99.97%. 
The number of pending jobs never went much above 500. The number of 
running jobs sometimes dipped to zero.


        Analysis Launch

The jobs for Batch 1 and 2 are running now. There are 50 analysis 
plugins, in addition to the standard ones, from 12 or 13 contributors. 
Of the 9 k, jobs, 7 k are finished.

We looked at this launch page 
<https://halldweb.jlab.org/data_monitoring/analysis/summary_swif_output_analysis_2016-02_ver02_batch01.html> 
as well. Because the input REST files were already resident on the cache 
disk, the number of active jobs has been stable at a few hundred.

When these jobs are finished, Alex will go back to reconstruction on 
Batches 3 and 4.


      Software Review

Curtis created a wiki page 
<https://halldweb.jlab.org/wiki/index.php/Software_Review_4> to collect 
information for review preparation. He also started writing a document 
<http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=3108> 
that we will use to organize content for presentation at the review. He 
is soliciting input from the Collaboration on topics to be presented 
and, even better, paragraphs, tables, and figures to flesh out the 
sections.

Mark emphasized that the Launches should be featured prominently. Each 
launch has its own character and the statistics from each the launch 
tell a different story in each case. It is a chance to not just show 
projections, but to present performance of the software IRL.

We may try to put the document into a private repository in GitHub. Mark 
will look into this.


      Actions Items

 1. Look into private repositories on GitHub. (Mark)
 2. Contribute paragraphs and figures to the Software Review document. (All)
 3. Read white paper on review of large farm projects. (All)

-- 
marki at jlab.org, (757)269-5295

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20160914/dce63ab7/attachment.html>