[Halld-offline] Offline Software Meeting Minutes, July 22, 2015

[Mark Ito]

Folks,

Find the minutes below and at

https://halldweb.jlab.org/wiki/index.php/GlueX_Offline_Meeting,_July_22,_2015#Minutes

   -- Mark
___________________________________________________

GlueX Offline Software Meeting, Minutes

Present:

  * *BlueJeans*: Kei Moriya
  * *CMU*: Curtis Meyer
  * *FSU*: Aristeidis Tsaris
  * *JLab*: Alex Barnes, Amber Boehnlein, Mark Ito (chair), David
    Lawrence, Paul Mattione, Eric Pooser, Matt Shepherd, Nathan Sparks,
    Simon Taylor, Beni Zihlmann
  * *NU*: Sean Dobbs


Find a recording of this meeting at https://bluejeans.com/s/8qJ6/ .

Announcements

 1. *Turning the Fine-Mesh Field Map into a Resource
    <https://mailman.jlab.org/pipermail/halld-offline/2015-July/002081.html>*.
    Sean led us through his email. Before this change the fine-mesh map
    was computed on the fly if not present and then stored in the
    current working directory. This change allows the user to put the
    files in a single configurable location. If there is no fine-mesh
    field for a given field map file, the system falls back to the old
    procedure.
 2. *Moving the Plug-Ins*. We went over the emails from Paul
    <https://mailman.jlab.org/pipermail/halld-offline/2015-July/002084.html>
    and Sean
    <https://mailman.jlab.org/pipermail/halld-offline/2015-July/002091.html>.
    David had to do some work to modify the SCons scheme to build the
    plug-ins in their new location.
 3. *Instructions for Setting Up the GlueX Environment
    <https://halldweb.jlab.org/wiki/index.php/Setting_Up_the_GlueX_Environment>*.
    Mark reminded us about the wiki page he announced earlier. Paul has
    recently added a list of environment variables that the user might
    want to set that are not set by the general use scripts.
 4. *Nightly Builds Working from the Git Repositories*. The scripts have
    been modified to get the latest versions of HDDS and sim-recon using
    Git. They use a local clone of the GitHub repositories.
 5. *SQL Injection Vulnerability, Offline Monitoring Pages
    <https://mailman.jlab.org/pipermail/halld/2015-July/000252.html>*. A
    team from DOE probing for cyber security vulnerabilities performed a
    successful SQL injection
    <https://en.wikipedia.org/wiki/SQL_injection> attack against a
    couple of the Offline Monitoring CGI Python scripts. Justin had the
    scripts fixed on the same day and they are back up and running now.


Offline Monitoring

Kei gave a brief report:

  * The last launch was two Fridays ago.
  * The next launch is scheduled for this Friday.
  * He plans to run a hybrid system of SWIF and jproj for this launch,
    taking advantage of SWIF's faster job submission mechanism. He is
    working with Chris Larrieu of SciComp on this.
  * For the launch after that he hopes to have converted to an all-SWIF
    system.
  * He plans to give an update on the conversion process and the final
    product at the next Offline Meeting. His goal is to produce
    something that others can use for their farm projects.


Sean reported that Marty Wise has fixed a problem on our database server 
(hallddb.jlab.org) that was causing the offline monitoring database 
access to hang. In the process Marty identified some queries that could 
use some optimization. Sean also mentioned that he and Justin are 
looking into whether the database tables need to be re-factored.

Spring 2015 Commissioning Simulations

Sean gave the report. Mark has done a first pass through running all of 
the jobs listed on the wiki page 
<https://halldweb.jlab.org/wiki/index.php/Spring_2015_Commissioning_Simulations>. 
He is filling in jobs that got stuck in bggen and timed-out without 
producing output.
Sean planning to run the same monitoring jobs on these files as is used 
in the offline monitoring jobs.

Geant4 Update

David gave the update. He has extracted some of the key classes from 
Richard's HDGeant4 repository and incorporated them in the CPP 
simulation he has been working on, in particular those that handle the 
geometry. Right now execution is very fast, but he is not sure that all 
of the material is seen by the generated particles. He suspects there is 
some subtlety having to do with Geant4's "parallel worlds" feature.
Beni asked if was possible to run HDGeant4 and pipe the output to 
mcsmear. David told us that he thought that problem has already been 
solved for HDGEANT, but he has not tried it himself.

Mixing Simulated Events

Sean described his work on mixing electromagnetic background (EMB) 
events with hadronic events from bggen. The hope is that time can be 
saved during simulation if the EMB does not have to be generated for 
every event.
He got David's hddm_merge_events program running. The program does not 
attempt to combine hits that are close in time when doing the mixing. He 
did comparisons between events that were generated with EMB in the 
standard way (where combination is done) and events where pre-generated 
EMB events were mixed with clean bggen events. In most cases the 
agreement is pretty good, within a few percent or better. The approach 
looks promising but there are still some unanswered questions. See his 
slides 
<https://halldweb.jlab.org/wiki/images/7/75/Sdobbs_OfflineMtg_EMBkgd_July2015.pdf> 
for details.
We discussed using real EMB events rather than simulated ones. This 
would likely give much more realistic simulated events.

  * Currently, we could use pair spectrometer triggers that we already have.
  * A dedicated random trigger would probably be better. There is a
    question of whether or not it needs to be synchronized with the
    machine RF.
  * Right now there is a practical difficulty: real data is in EVIO
    format and the simulated events are in HDDM format. The mixing
    program works when both sources are HDDM. Sean and David will
    discuss a way around this.


Conversion from Subversion to Git

Mark summarized the conversion we did last week. In particular he went 
over new items on the wiki page 
<https://halldweb.jlab.org/wiki/index.php/Conversion_from_Subversion_to_Git> 
documenting the process. Among the items he mentioned were:

  * There is local copy of the Git repositories for hdds and sim-recon
    on our group disk at JLab. These are updated at noon and midnight
    and are now used to support the nightly builds.
  * He posted some notes on the details of the conversion process to
    serve as a guide for future conversion efforts.


Sean asked about how we approach conversion of other directories from 
Subversion to Git in the future. At one point Mark proposed that we 
discuss any such conversion at this meeting and have the group decide on 
a case-by-case basis. We decided to adopt this as our policy.

ROOT 6

Beni has succeeded in building our software (under Ubuntu) using ROOT 6. 
He has already checked in the necessary changes to sim-recon. The system 
is still compatible with ROOT 5; the code senses which version is being 
used. He also had to make changes to JANA, but did not check those 
changes into the JANA repository.
GCC version 4.8 is required. The JLab batch farm is at 4.4 so we cannot 
use ROOT 6 for large-scale production at the Lab.
We thought we should go ahead and complete the work needed to have a 
ROOT-6-compatible system. We will continue to try and maintain 
compatibility with ROOT 5 so we don't lock ourselves out of the JLab Farm.

Action Item Review

 1. Work out details of doing an HDDM/EVIO merge.
 2. Complete work on changes to be compatible with ROOT 6.

Retrieved from 
"https://halldweb.jlab.org/wiki/index.php?title=GlueX_Offline_Meeting,_July_22,_2015 
<https://halldweb.jlab.org/wiki/index.php?title=GlueX_Offline_Meeting,_July_22,_2015&oldid=68832>"

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20150723/8a2062b5/attachment.html>