[Halld-offline] Offline Software Meeting Minutes, September 30, 2015

Fri Oct 2 14:34:16 EDT 2015

Folks,

Find the minutes below and at 
https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_September_30,_2015#Minutes 
.

   -- Mark
   _______________________

  GlueX Offline Meeting, September 30, 2015, Minutes

Present:

  * *CMU*: Curtis Meyer, Mike Staib
  * *FIU*: Mahmoud Kamel
  * *JLab*: Amber Boehnlein, Mark Ito (chair), David Lawrence, Paul
    Mattione, Kei Moriya, Eric Pooser, Nathan Sparks, Simon Taylor, Beni
    Zihlmann
  * *NU*: Sean Dobbs
  * *UConn*: James McIntyre

There is arecording of this meeting <https://bluejeans.com/s/8Df9/>on 
the BlueJeans site.

      Announcements

 1. *Team Maintainers and Admins*.Team Maintainers
    <https://halldweb.jlab.org/wiki/index.php/GitHub_and_GlueX>are
    members of the"GlueX Team"
    <https://github.com/orgs/JeffersonLab/teams/gluex>that are able to
    add new members to the team on GitHub. Admins are members of
    the"GlueX Admin"
    <https://github.com/orgs/JeffersonLab/teams/gluex-admin>team and
    have privilege to create and delete repositories owned by the GlueX
    Team.
      * Currently GlueX Team members are not allowed to create
        repositories owned by team. There is a setting to allow that
        privilege. Mark asked whether we should turn that on. David
        suggested that we keep things as they are; team members have
        private accounts and can create and delete repositories at will
        using those accounts. Mature private repositories can be forked
        into the GlueX Team when they are ready by one of the admins. We
        agreed with David; keep things as they are.
 2. *Collaboration Meeting
    <https://halldweb.jlab.org/wiki/index.php/GlueX-Collaboration-Oct-2015>October
    8-10, 2015 at Jefferson Lab*. We reviewed the agenda as posted. Paul
    may need more time than listed, but we decided not to do
    micro-adjustments to the agenda.

      Review of minutes from September 16

We went overthe minutes 
<https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_September_16,_2015#Minutes>.

  * *New Work Disk*. The initial rsync of our work disk to the new
    Lustre server completed last week. It only took about 4 days. The
    switch-over will require a down period for the final "top-off" rsync
    of several hours. We will postpone that until after the
    collaboration meeting.
  * Sean has confirmed that*mailman.jlab.org is now accessible*from
    off-site.

      Offline Monitoring

Kei gave the report. Seehis slides 
<https://halldweb1.jlab.org/wiki/images/e/e5/2015-09-30-offline_monitoring.pdf>for 
details. Topics covered included:

  * Current launch (number of jobs, nature of failures)
  * Statistics on the launch (memory use, CPU time vs. wall time, etc.)
  * Farm environment (other jobs running in competition with the launch)
  * Memory Usage on Farm (number of jobs memory limited, not limited by
    number of job slots)
      o Hall D can be memory efficient due to JANA multi-threading
      o Other, non-Hall-D jobs may not be as memory efficient
      o We may be tape limited in any case. (Paul)
      o If an entire node is reserved for Hall D, then our memory
        efficiency will translate into greater core usage efficiency.
        (David)
  * Moving Forward.
      o Due to new plugins, now require 8GB of RAM to run jobs - some
        pruning of unnecessary histograms would be welcome
      o Request by Mike S. that only certain runs be run with
        -PBCAL:USE_TDC=1, unfortunately bug in script made all runs
        processed with this option
          + a negative effect on uncalibrated runs
      o Started on BCAL cosmics 2015-06 ver03 on September 29 (Tue)
      o Next launch is this week, October 2(Fri): no demand for this
      o Any need for running over 2014-10 Data?: no requests received

        SWIF Summary Output

Kei showed us anew webpage 
<https://halldweb.jlab.org/data_monitoring/launch_analysis/tmp/summary_swif_output_offline_monitoring_RunPeriod2015_03_ver15_hd_rawdata.html>that 
can be automatically generated to display statistics about each launch. 
Some of the plots are among those he has been showing in the launch 
reports, some are new. There is also a listing of all problem jobs.

      Spring 2015 Commissioning Simulations

Mark reported that the re-do of theSpring 2015 Commissioning Simulations 
<https://halldweb.jlab.org/wiki/index.php/Spring_2015_Commissioning_Simulations>with 
new BCAL code is about 80% done. Output files can be found at 
/volatile/halld/detcom_02_1.

      2016 Commissioning Simulations

Sean has started putting together awiki page 
<https://halldweb.jlab.org/wiki/index.php/Sim1_Conditions>with 
information about standard conditions to be simulated next.

  * Names for projects: something like sim1, sim2,...
  * The data sets will not be necessarily tied to a particular real data
    run, rather they will be generic and can be used for studies under
    those generic conditions, for example signal studies for particular
    reactions or rate studies.
  * We may need a larger data set than those we have been generating for
    commissioning simulations recently, perhaps something on the scale
    of past data challenges. Right now the only large data set we have
    is from Data Challenge 2, and that is getting kind of old now.

      Auto-Build on Pull Request

Sean described the recent effort to set up a build of sim-recon 
triggered by a pull request on GitHub as we discussed at the last meeting.

  * He, Nathan, and Mark met with Marty Wise and he has installed some
    packages on halldweb to support the system.
  * The system uses the "gluex" user on GitHub.
  * Mark put together the scripts to do the build. Although initiated on
    halldweb, the build is executed on the ifarm.
  * The CGI script on halldweb not only initiates the build but reports
    the results back to GitHub as a comment on the originating pull request.
  * There seems to be an issue having the entire process succeed from
    GitHub's point of view, possibly because the build takes a while and
    the system may be timing out. The build itself seems to succeed.
  * The system is not active yet, but will be turned on once this last
    issue is solved.

      Noise Studies

Sean showed some initial results from a study comparing raw hit 
distributions between simulated E&M background data and pair 
spectrometer triggers. Seehis slides 
<https://halldweb.jlab.org/wiki/images/b/b9/Sdobbs_OfflineMtg_20150930.pdf>for 
all of the plots. There are a number of large discrepancies, but at this 
stage there were a lot of questions about whether the comparisons were 
fair. It does appear that the pair spectrometer triggers have a large 
prompt component. If that conclusion holds up, then those data would not 
be useful for these comparisons. If a random trigger could be set up, 
that would solve the problem.

Mark remarked that it would be very nice if we could understand the 
comparison. In particular, it would give us confidence in extrapolations 
to higher beam rates using simulated data.

      b1pi results review

We decided to postpone discussing theautomatic b1pi test 
<https://halldweb.jlab.org/wiki/index.php/Automatic_Tests_of_GlueX_Software>.

      Review of recent pull requests

We also deferred this item. The idea was to review as a group thelist of 
pull requests 
<https://github.com/JeffersonLab/sim-recon/pulls?q=is%3Aopen+is%3Apr>initiated 
since the last meeting.

        Other Related Git Issues

Another set of topics that were postponed:

  * policy on comments when merging
  * alternate workflows for submitting pull requests
  * using git rebase, should we?

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=GlueX_Offline_Meeting,_September_30,_2015&oldid=70543"

  * This page was last modified on 2 October 2015, at 14:31.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20151002/94b56df3/attachment.html>