[Halld-offline] Offline Software Meeting Minutes, August 5, 2015

[Mark Ito]

Folks,

Please find the minutes below and at http://tinyurl.com/qeteeyf .

   -- Mark
   ________________________


      GlueX Offline Meeting, August 5, 2015, Minutes

Present:

  * *CMU*: Curtis Meyer
  * *FIU*: Mahmoud Kamel
  * *FSU*: Aristeidis Tsaris
  * *JLab*: Alex Barnes, Mark Dalton, Mark Ito (chair), David Lawrence,
    Paul Mattione, Kei Moriya, Eric Pooser, Nathan Sparks, Justin
    Stevens, Simon Taylor, Beni Zihlmann
  * *NU*: Sean Dobbs


      Announcements

 1. *The meaning of ifarm has changed*
    <https://mailman.jlab.org/pipermail/jlab-scicomp-briefs/2015q3/000102.html>.
    Use "ifarm62" for CentOS 6.2, "ifarm65" for CentOS 6.5. "ifarm" is
    the same as "ifarm65" (it used to point to CentOS 6.2 nodes).
 2. *Policy on CCDB Variations for Reconstructing Simulated Data
    <https://halldweb.jlab.org/wiki/index.php/Policy_on_CCDB_Variations_for_Reconstructing_Simulated_Data>*.
    This new wiki page summarizes our policy on how to use CCDB
    variations with Monte Carlo data.
 3. *New releases hdds 3.3, sim-recon 1.4.0*
    <https://mailman.jlab.org/pipermail/halld-offline/2015-July/002108.html>.
    Note that the release notes are now being posted on GitHub.
 4. *ccdb_changes email list*
    <https://mailman.jlab.org/pipermail/halld-offline/2015-August/002111.html>.
    The new list sends out a daily digest of changes to the CCDB.
 5. *New Git Guide*
    <https://mailman.jlab.org/pipermail/halld-offline/2015-August/002112.html>
    Paul transcribed his notes from his personal climb of the Git
    learning curve.
 6. *Github for Mac*
    <https://mailman.jlab.org/pipermail/halld-offline/2015-August/002113.html>.
    David pointed us to a GitHub-distributed application for the Mac
    desktop.
 7. *Git versions on the JLab CUE*
    <https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_FAQ#I_am_getting_a_.22403_Forbidden.22_error_when_trying_to_push_to_a_GitHub_repository_from_JLab._What_is_wrong.3F>.
    Simon and Elton found some problems with specific versions of Git
    installed at JLab. Suggested combination: /apps/bin/git on ifarm65
    or jlabl*.
 8. *Emulation mode changes*
    <https://mailman.jlab.org/pipermail/halld-offline/2015-August/002115.html>.
    David reviewed his recent email describing his proposed changes to
    control of Flash-ADC emulation, aka, creating summary data from raw
    waveforms in the offline environment.


      Review of minutes from July 22

We went over the minutes 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_July_22,_2015#Minutes>. 
There was discussion on a few items:

  * We still need to work out details of doing an HDDM/EVIO merge so
    that we can mix in real background events with simulated physics events.
  * David mentioned that there are some changes needed in JANA to
    support use of ROOT 6.
  * Mark found out that the lack of notification to repository watchers
    when a direct push to GitHub repositories is a feature, not a bug.
    See this page
    <https://help.github.com/articles/receiving-email-notifications-for-pushes-to-a-repository/>
    on GitHub for details. Turns out only two email addresses can be
    added to the distribution list; one of them should be a group email
    address. We decided not to turn on the feature to avoid excessive
    traffic on halld-offline at jlab.org.


      Offline Monitoring

Kei gave a detailed report 
<https://halldweb.jlab.org/wiki/images/8/83/2015-07-29-offline_monitoring.pdf> 
at the previous calibration meeting 
<https://halldweb.jlab.org/wiki/index.php/July_29,_2015_Calibration>. He 
continues to work with Chris Larrieu on SWIF features and is relying 
more and more on it for the launches.


      Spring 2015 Commissioning Simulations

Sean reports that the creation of the HDDM raw data for the Spring 2015 
Commissioning Simulations 
<https://halldweb.jlab.org/wiki/index.php/Spring_2015_Commissioning_Simulations> 
is practically complete. The data is available in the 
/volatile/halld/detcom_02/smeared directory. It still remains to do the 
reconstruction pass on the data.


      Geant4 Update

  * Richard has completed the incorporation of the three types of event
    sources:

particle gun, coherent bremsstrahlung (now implemented as a C++ class), 
and HDDM. Note that genr8 and bggen both create HDDM-formatted events.

  * Secondary vertices can be stored and examined later.
  * The complete PDG particle list has been implemented.
  * Richard is suggesting using curated physics lists from the LHC
    experiments rather than struggling to assemble our own. He is
    following David's example and is using a hybrid of LHCb and FRITIOF.
  * Next is to do a detailed comparison of particle propagation between
    GEANT 3 and Geant4.
  * He is using the boost library for Python hooks and some special
    functions. This means that we will have to install this package on
    our machines to get a successful build.
  * The next major task to work on creating the detector hits from the
    sensitive detectors.
  * The code can be cloned from GitHub and built. The URL is
    https://github.com/rjones30/HDGeant4.git.


      Overhaul of the ROOT TTree Format

Paul led us through his email 
<https://mailman.jlab.org/pipermail/halld-offline/2015-July/002103.html> 
outlining the need for changes to the tree format. He said,

    It needs to be done, I'm going to it,..., if you have any comments
    by Friday let me know.

Mark asked if the primary motivation was to take advantage of 
multi-threaded processing with PROOF. Paul explained the situation is 
more complicated than that one issue. User code might be simpler in 
certain circumstances in the new scheme and multi-threaded processing 
would be an option not available now.


      Conversion from Subversion to Git: discussion

We had an unstructured discussion about our experience so far with Git.

  * People seem to generally like it.
  * David is still finding some details and concepts that need to be
    understood to use Git effectively.
  * Simon will try the "git diff --cached" command to look at difference
    between staged files and repository.
  * David noted that there is a bit of blame sharing with pull requests
    being issued by one person and merged in by another, a new twist.
  * David is liking the local nature of Git and the ease in making
    sandboxes to play with changes. He also noted the ease in changing
    from branch to branch. I that context, he test drove the "git stash"
    command and it performed as expected.
  * Mark solicited comments on whether the workflow we are using is too
    protective against changes or too accommodating of them. There was
    no real comment, which he interpreted as "about right".
  * Justin mentioned that at the S&T review, he was asked how we protect
    against badly broken code getting checked into the repository. He
    told the committee that we have changed to Git and that gives us a
    measure of control.
  * David noted that the email volume from watching the repositories
    seems higher than with Subversion.
  * David thought that although we are merging pull requests without a
    lot of review right now, going forward, as more junior members of
    the collaboration make pull requests, those requests in particular
    might get more careful scrutiny, as is appropriate.
  * Sean proposed that we have builds triggered on pull requests. He
    mentioned that there are tools out there to do things like that.


      Proposed move to Git: build_scripts, gluex_install, jproj

Mark I. proposed moving version control of some of the packages he has 
been maintaining from Subversion to Git. There were no objections, but 
the topic generated discussion on other related topics.

  * Mark I. is proposing to create three new repositories on GitHub.
    Justin thought we might want to have a "scripts" repository and have
    these projects in subdirectories of that repository, like they are
    currently in Subversion. Mark thought that the repositories should
    be more like single-purpose projects one repository per project.
    David noted that this scheme will lead to a lot of repositories.
  * Justin observed that since all of the repositories are under the
    Jefferson Lab organization at GitHub on a single level, it is very
    difficult to browse the list and figure out which ones one needs.
    The name "build_scripts" gives you no clue that the repository is
    relevant for GlueX work.
      o Curtis thought we might want to prepend "HD" or something
        similar to the names of all of our repositories.
      o Mark I. noted that if one goes to the "GlueX Team" page at
        GitHub, one can click on a link to get a list of all
        repositories that the team is allowed to write to.
      o Mark thought that the situation is like the DocDB in that you
        cannot simply browse the titles to find the document you need.
        Rather you need a reference into the title space to get the
        document you want.
  * David pointed out that to get everything needed to build a working
    system, there are several packages that one needs to get and that
    the list cannot be intuited or guessed at. One needs an HDPM or a
    gluex_install.
      o David further noted that gluex_install does not work on Mac. We
        discussed ways that Mark I. could do some testing on a Mac platform.
      o Paul noted that there is a concept called "submodules" in Git
        that might help.

We did not come to conclusions on these issues, but clearly we will have 
to return to them.


      Action Items

 1. Work out details of doing an HDDM/EVIO merge. (Sean, David)
 2. Complete work on changes to be compatible with ROOT 6. (Beni, David)
 3. Overhaul of the ROOT TTree Format. (Paul)
 4. Come up with a policy on organizing Git repositories and repository
    names. (Mark and interested parties)

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=GlueX_Offline_Meeting,_August_5,_2015&oldid=69214"

  * This page was last modified on 6 August 2015, at 17:16.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20150806/5436ca15/attachment.html>