[Halld-offline] Offline Software Meeting Minutes, August 17, 2016

[Mark Ito]

Folks,

Please find the minutes below and at 
https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_August_17,_2016#Minutes 
.

   -- Mark

____________________________


    GlueX Offline Meeting, August 17, 2016, Minutes

Present:

  * *CMU*: Naomi Jarvis, Curtis Meyer
  * *FIU*: Mahmoud Kamel
  * *JLab*: Alexander Austregesilo, Amber Boehnlein, Brad Cannon, Mark
    Ito (chair), David Lawrence, Paul Mattione, Nathan Sparks, Simon Taylor
  * *NU*: Sean Dobbs

You can view a recording of this meeting <https://bluejeans.com/s/acGZ/> 
on the BlueJeans site.


      Announcements

 1. *New release: sim-recon 2.3.0*. This release
    <https://mailman.jlab.org/pipermail/halld-offline/2016-August/002431.html>
    came out a week ago. With recent changes to tracking and tagger
    reconstruction, it looks like it is time for another one.
 2. *Analysis Actions in ROOT DSelector*. Paul led us through his email
    <https://mailman.jlab.org/pipermail/halld-offline/2016-August/002433.html>
    describing the new features, added so that you do not have to do
    everything yourself, for example calculating the beam asymmetry
    angle for pseudoscalers.
      * Nathan asked about custom analysis actions. Paul remarked that
        they can be added but for proof to work, you have to package
        your code.


      Review of minutes from the last meeting

We went over the minutes from August 3 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_August_3,_2016#Minutes>. 


  * No work has been done on managing the growing number of plugins in
    sim-recon.
  * The new farm nodes have arrived at the Lab.
  * The last release of sim-recon was built against both ROOT5 and
    ROOT6. Sean and Alex Barnes have been updating macros to work with
    the ROOT6 version.
      o Alex A. discovered a strange problem when building the code for
        the monitoring launch. There is a conflict between the
        definition of CPLUS_INCLUDE_PATH, as done by the RCDB
        environment set-up, and rootcling from ROOT6, and only under the
        bash shell (tcsh is OK). The work-around is to simply unset
        CPLUS_INCLUDE_PATH.


      Analysis Launch

Alex went over his recent email 
<https://mailman.jlab.org/pipermail/halld-offline/2016-August/002432.html> 
describing the completion of the analysis launch. In total the size of 
the output is 10 TB. He will make a table of how much space is used by 
each of the channels.

Six files did not get processed completely; for most of these the REST 
files were 0 bytes. The files are pinned on the cache disk for 30 days 
now. Batch 2 had a few more plugins and authors than Batch 1, with 56 
trees produced.


      Monitoring Launch

Alex started this again this morning. The code chosen uses ROOT6, the 
new EVIO parser from David, and yesterday's fixes to the tracking code 
from Simon. There was a problem with the end-of-file handling with the 
new EVIO parser that David fixed earlier in the week.

Because the input files were already cached from previous attempts at 
start-up, this time the jobs have started executing quickly. The launch 
is 10% done after three hours.


      sim1.1

All 8 k jobs have completed successfully, save for 20 where the output 
REST files were missing or of zero length. Mark has resubmitted these 
jobs. There was a problem last week with jobs corresponding to runs that 
did not have a definition for the collimator size in the RCDB. Since the 
job parameters depend on this definition, the jobs failed. Mark found 
the problem; Sean fixed the values in the RCDB.

Information on conditions used and location of output files can be found 
on the sim1.1 conditions page 
<https://halldweb.jlab.org/gluex_simulations/sim1.1/>.

Alex A. volunteered to do an analysis launch on the sim1.1 REST files 
once all of them are done.

Sean has been working on putting dead FDC channels into the CCDB for use 
in a future iteration of the simulation. He is getting the list from 
Alex A. The next iteration will likely be just before the DNP meeting in 
Vancouver.


      HDDM multi-threaded I/O benchmarks

Richard reported on recent work with multi-threading both input and 
output routines in HDDM. For jobs that do not do a lot of event 
processing, the compression/decompression available in the I/O stages 
can be limiting. See his slides 
<https://halldweb.jlab.org/wiki/images/0/00/HDDM_multithreaded_extensions.pdf> 
for the details. He looked at compression schemes (bzip2, gzip, none), 
input and output, streaming access and random access, as well as input 
of compressed output produced by a multi-threaded task in different 
combinations for a total of 24 runs of the benchmark software. As an 
example, with four threads (on a quad-core machine) he gets a factor of 
three in input speed for gzipped data when compared to a single-threaded 
run. In many cases he saturated the raw disk bandwidth with the tests.

Exploiting this feature in JANA turns out not to be completely trivial 
and volunteers for the effort have not come forward. For CPU intensive 
tasks, like reconstruction, the gain is not significant. In other 
contexts it can be important. For now we know that in principle we can 
crank up the bandwidth is we need to in the future.

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=GlueX_Offline_Meeting,_August_17,_2016&oldid=76909"

  * This page was last modified on 17 August 2016, at 17:45.

-- 
marki at jlab.org, (757)269-5295

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20160817/33e7335c/attachment.html>