[Halld-offline] Software Meeting Minutes, March 3, 2020

[Mark Ito]

Folks,

Please find the minutes here 
<https://halldweb.jlab.org/wiki/index.php/Gluex_Software_Meeting,_March_3,_2020#Minutes> 
and below.

   -- Mark


    Gluex Software Meeting, March 3, 2020, Minutes

Present:

  * *FSU: * Sean Dobbs
  * *JLab: * Alex Austregesilo, Hovanes Egiyan, Mark Ito (chair), Igal
    Jaegle, David Lawrence, Keigo Mizutani, Andrew Schick, Beni Zihlmann
  * *ODU: * Nilanga Wickramaarachchi

There is a recording of his meeting <https://bluejeans.com/s/C9Djj> on 
the BlueJeans site. Use your JLab credentials to access it.


      Announcements

 1. /volatile/halld moving to a new Lustre system
    <https://mailman.jlab.org/pipermail/halld-offline/2020-February/003928.html>
    Files on the old partition will be deleted tomorrow.
 2. Deprecating DIST in favor of HALLD_VERSIONS for version.xml files
    <https://mailman.jlab.org/pipermail/halld-offline/2020-February/003934.html>.
    Currently existing version set files will remain in DIST until May 1.


      Review of Minutes from the Last Software Meeting

We went over the minutes from February 18 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Software_Meeting,_February_18,_2020#Minutes>. 


  * The plan to delete some old CDC tables from the CCDB went slightly
    awry
    <https://mailman.jlab.org/pipermail/halld-offline/2020-February/003913.html>.
    We had to restore from backup and scuttle the plan.
  * The issue on having the collaboration in general maintain data
    integrity <https://github.com/JeffersonLab/rcdb/issues/86> on the
    RCDB has been submitted.
  * Dakota Christian's issue with MC
    <https://groups.google.com/forum/#!topic/gluex-software/HDDMbBfBqVA>
    turned out to be sensitivity to the order of decays in genr8.


      recon.xml files matching between MC and data analysis launches

We discuss an issue raised in this question to the software help group 
<https://groups.google.com/forum/#!topic/gluex-software/KV195ZFLFAg> 
from Nacer Hamdi.

The fundamental complication raised is that to do reconstruction we rely 
on a particular version of halld_recon. Subsequent Monte Carlo 
simulation therefore need to use the same version of halld_recon to be 
consistent. On the other hand, analysis launches on REST data produced 
by the reconstruction launches produce ROOT trees using the analysis 
library (among other packages) also provided by halld_recon. So root 
tree creation for simulation, to be consistent, needs to use this second 
version of halld_recon. This was first discussed back in May of last 
year 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Software_Meeting,_May_28,_2019#Discussion_of_splitting_analysis_libraries_from_halld_recon>. 
The solution we settled on was to create a new XML files 
(version_correlations.xml) that would identify pairs of halld_recon 
versions (actually version set files) so that the appropriate versions 
could be deployed on Monte Carlo.

  * Nacer's question pointed out that not all useful version set pairs
    have been recorded and built.
  * Mark pointed out that the implementation currently in MCwrapper-bot
    to guide the user to a choice of consistent versions is complicated.
    It would be difficult to describe at a level that can be reproduced
    easily by non-experts.
  * Mark further pointed out the the correlation XML files was a
    stop-gap solution. It was easy to implement and works, but was not
    necessarily the "best" solution.
  * Mark's suggestion is that we go back and revisit the idea of
    breaking out the analysis libraries from halld_recon. Recall that
    from that discussion last year, this is not as easy as it might seem
    on the surface; there are in fact many libraries involved in the
    overlap between reconstruction and analysis, not just the analysis
    library.
  * Alex was skeptical about whether further deconstruction of
    halld_recon would result in a simpler system since there would still
    be libraries used in both reconstruction and analysis and so the
    two-version problem would still obtain.
  * Alex proposes that the choice of MCwrapper be based on the version
    set of the analysis launch that is being simulated. Then the choice
    of both halld_recon versions should be unambiguous.
  * Sean pointed out that is all of the information about what versions
    were used for which launches is stored in a database, an interface
    should be able to pick out correct pairs of versions sets.
  * Alex has spoken to Thomas Britton about implementing the
    "pick-an-analysis-launch" approach for MCwrapper-bot. It was alleged
    that Thomas has agreed to work on it (someday).


      Review of recent issues and pull requests

We went over several of the issue pages. See the list above in the agenda.

In the course of the review, Sean raised the issue of access control for 
the CCDB. We have seen mistakes go into the default variation of the 
CCDB on the master replica on the server on hallddb.jlab.org. Most of 
the time these have been caught and corrected promptly, but there is an 
example where a non-optimal set of constants was used for production 
reconstruction. Approaches:

 1. Sean advocates creating a white list so that only certain users can
    write to the default variation on the master.
      * [added in press] This was not in the original implementation
        because it would prevent users from creating private variations
        on the database server, private variation that could be used for
        testing and learning. To maintain that capability an custom
        access control system would have to be coded and managed.
 2. David floated the idea of having a the master on a separate server
    altogether. This server could be configured to require a password
    when connecting.

Mark agreed to talk to Dmitry Romanov again to review what hooks are 
already available and get his ideas on an approach.

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=Gluex_Software_Meeting,_March_3,_2020&oldid=97134"

  * This page was last modified on 9 March 2020, at 18:06.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20200309/39fa4ed0/attachment.html>