[Halld-offline] Software Meeting Minutes, August 20, 2019

[Mark Ito]

Folks,

Please find the minutes here 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Software_Meeting,_August_20,_2019#Minutes> 
and below.

   -- Mark

___________________________________


    GlueX Software Meeting, August 20, 2019, Minutes

Present:

  * *CMU: * Naomi Jarvis
  * *JLab: * David Abbott, Stuart Fegan, Mark Ito (chair), David
    Lawrence, Simon Taylor, Carl Timmer, Beni Zihlmann


      Announcements

 1. /mss/halld/halld-scratch will be zero'ed
    <https://mailman.jlab.org/pipermail/halld-offline/2019-August/003734.html>.
    The exact schedule has not been set.
 2. Cache files where the disk version was different from that on tape
    <https://mailman.jlab.org/pipermail/halld-offline/2019-August/003735.html>.
    These files were lost.


      Online Skims

David described not only the new system for performing skims of special 
triggers online, but also the new architecture for writing data to disk 
in the counting room as it comes out of CODA. See his slides 
<https://docs.google.com/presentation/d/1NuU8b5qo6M_WnNfpc4F1w-vhaFkP7atXN_5DnzzVIxs/edit?usp=sharing> 
for details.

  * The basic idea is to do skims of special triggers (BCAL-LED
    triggers, random triggers, sync events, etc.) in the counting room
    while we take data. Those skim files are then immediately available
    for calibrations. This could speed up calibrations needed in advance
    of the first reconstruction pass on the data.
  * To speed up the process, blocks of triggers that do not contain the
    special triggers can be skipped. Time is saved by avoiding
    dis-entanglement of the events. This means that there will be a
    reduction of the number of PS triggers in the skimmed output; all
    blocks have PS triggers, but not all PS triggers are in blocks with
    special triggers.
  * Information from the block headers will be put into a relational
    database, including information on the number of each type of
    trigger, the first and last events in each output file, etc. These
    quantities can be migrated to the RCDB later.
  * The architecture of the new Hall D Data recording scheme will use
    fast copies using remote direct memory access (RDMA) to transfer
    data from IB interface to IB interface without involving the CPU.
    RAID servers will send and receive data from ramdisks and the data
    written from memory to arrays of traditional partitions on multiple
    RAID servers. From there the jmigrate system will look for data to
    be shipped to the Computer Center for storage on tape.
  * There are still some issues to deal with, including more thorough
    testing, and definition of a back-pressure mechanism (especially if
    the skim process cannot keep up).


      Review of minutes from the last Software Meeting

We went over the minutes from August 6 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Software_Meeting,_August_6,_2019#Minutes>. 
David reported that the reconstruction launch at NERSC got going again 
last week, but ran into problems over the weekend due to a change in how 
tape is handled at JLab.


      Review of recent discussion on the GlueX Software Help List

We briefly discuss the issue with SQLite versions of the CCDB 
<https://groups.google.com/forum/#!topic/gluex-software/Krdl0FxwMGQ>. 
There is still no clear-cut, works-everywhere solution.

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=GlueX_Software_Meeting,_August_20,_2019&oldid=93602"

  * This page was last modified on 20 August 2019, at 17:41.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20190820/444f2d53/attachment.html>