[Halld-offline] Software Meeting Minutes, February 2, 2021

[Mark Ito]

Folks,

Please find the minutes here 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Software_Meeting,_February_2,_2021#Minutes> 
and below.

   -- Mark

      _____________________________________


    GlueX Software Meeting, February 2, 2021, Minutes

Present: Alex Austregesilo, Thomas Britton, Sean Dobbs, Mark Ito 
(chair), Igal Jaegle, Richard Jones, David Lawrence, Simon Taylor, 
Nilanga Wickramaarachchi

There is a recording of this meeting 
<https://bluejeans.com/s/8jPxgwuScQc/>. Log into the BlueJeans site 
<https://jlab.bluejeans.com> first to gain access (use your JLab 
credentials).


      Announcements

 1. New version set: 4.35.0
    <https://mailman.jlab.org/pipermail/halld-offline/2021-January/008443.html>.
    This one has the latest release of AmpTools.
 2. Tape Recovery News
    <https://mailman.jlab.org/pipermail/halld-offline/2021-January/008447.html>.
    The latest from Bryan Hess from last week.
 3. Old packages backed up and deleted from group disk
    <https://mailman.jlab.org/pipermail/halld-offline/2021-February/008449.html>.
    Peter Pauli reported that halld_recon 4.5.0 needed to put back for
    simulations on the OSG. Mark did that this morning.
 4. HSF Training: Continuous Integration in GitHub, February 16-19
    <https://mailman.jlab.org/pipermail/jlab-scicomp-briefs/2021q1/000833.html>.
    Please see the Indico page for details.


      Review of Minutes from the Last Software Meeting

We went over the minutes from January 19 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Software_Meeting,_January_19,_2021#Minutes> 
without significant comment.


      Minutes from the Last HDGeant4 Meeting

We went over the minutes from January 26 
<https://halldweb.jlab.org/wiki/index.php/HDGeant4_Meeting,_January_26,_2021#Minutes>. 
Mark mentioned that he is close to incorporating Richard's Diracxx 
package <https://github.com/JeffersonLab/Diracxx> into our set of 
standard software.


      Collaboration Meeting Planning

We decided that it was premature to make a definite plan for 
software-related talks at the upcoming collaboration meeting 
<https://halldweb.jlab.org/wiki/index.php/GlueX-Collaboration-Feb-2021>. 
We will revisit the topic once the agenda is announced.


      Reconstruction on the OSG

Richard gave us an update on progress toward doing large-scale 
reconstruction on the Open Science Grid (OSG). He described issues with 
running the jobs, which have largely been solved. In the process of 
comparing the results on the OSG with those from the JLab farm, he did 
not find agreement and traced it to non-reproducibility of the resulting 
reconstructed events from repeated runs on identical data.

Further investigation reveals that most of the problems are in the 
Analysis Library. He continues to pursue the painstaking exploration of 
the problem.

Please see Richard's slides 
<https://docs.google.com/presentation/d/1GzmYiLmANsP-xxgHHGObSAam3pKSX6wFNNhEL25vcXg/edit#slide=id.p> 
for the details.


      Standardized Tests

Sean brought up the need to step up our testing game. We discussed ideas 
to both augment our current procedures and develop new ones.

  * We do a build test and simple execution tests on new pull requests.
    The output of the execution tests are not examined.
  * We do a nightly build test of all GlueX software. There is no
    nightly execution test done.
  * Every three days we do a reconstruction test with the latest nightly
    build. There is no automated verification of the results, just an
    email with a link to selected histograms.
  * Twice a week we do a simulation test, from event generation to
    reconstruction and histogram production. There is no automatic
    verification of results, although a small number of are plotted as
    function of calendar time
    <https://halldweb.jlab.org/disk_management/b1pi_mean.pngquantities>.
    The test also suffers from low statistics.
  * There is no test of output reproducibility.

Sean proposed we add this to future software meeting agendas. Richard 
suggested starting a wiki page to collect ideas.


      Action Item Review

 1. Ask the Computer Center about staging the start of jobs in time.
    (Mark I.)
 2. Make sure that the automatic tests of HDGeant4 pull requests have
    been fully implemented. (Mark I., Sean)
 3. Finish conversion of halld_recon to use JANA2. (Nathan)
 4. Release CCDB 2.0 (Dmitry, Mark I.)
 5. Start a wiki page to collect ideas on software testing. (Mark)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20210203/328a32e5/attachment.html>