[Halld-offline] No offline meeting, Fri. Mar 23

[Richard Jones]

Dave, Mark and all,

On the OSG front, I am working on testing a workflow that allows direct
access to raw data over xrootd. So far things are looking very promising. I
have been able to keep 4 threads of hd_ana busy within a single process
simply opening the file within a container on an osg worker host over
xrootd from where it sits on the uconn xrootd server. This avoids the need
to first stage the data, and then run over it once it has been moved to a
local disk. The output still goes to local disk and gets returned to the
submit host via the normal condor file transfer mechanism, although it
should be possible to write files over direct access xrootd as well. All of
this runs within Mark's standard GlueX container without modification. I
did need to install the xrootd client libraries on oasis. Right now they
live outside the /group/halld tree, but I spoke with Mark and he is
interested in adding xrootd libraries to our standard /group/halld build
tree.  I will report on performance with streaming data at the next meeting.

-Richard J.

On Fri, Mar 23, 2018 at 7:37 AM, David Lawrence <davidl at jlab.org> wrote:

> Since we’re skipping the meeting this week I’ll give a quick report on a
> meeting with SciComp
> earlier this week on moving towards production processing of raw data at
> NERSC.
>
> 1. Chip has received a 50 MCPU-hour allocation along with 50TB of disk
> space at NERSC
>
> 2. Chris Larrieu is going to start developing system that allows us to
> submit NERSC jobs from JLAB.
>
> 3. I have asked NERSC about implementing CVMFS so we can try and parallel
> our OSG efforts.
>   (needs follow up)
>
> 4. SciComp would like to have a system ready to try some type of data
> challenge this summer. We should
> decide if there is a reconstruction launch that would be useful for this
> (e.g. Spring 2018 data or
> replay of older data set with newer code/calibrations). If we let them
> know when we will be ready, they
> can use that as a target date for an initial system.
>
>
>
>
> -------------------------------------------------------------
> David Lawrence Ph.D.
> Staff Scientist, Thomas Jefferson National Accelerator Facility
> Newport News, VA
> davidl at jlab.org
> (757) 269-5567 W
> (757) 746-6697 C
>
> On Mar 22, 2018, at 10:41 PM, Mark Ito <marki at jlab.org> wrote:
>
> Folks,
>
> I am traveling and have not made arrangements for a replacement chairman,
> so I propose we skip this week's offline and container meetings. If you do
> not hear from me, assume the meeting is off.
>
>   -- Mark
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
>
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20180323/bb604c11/attachment-0002.html>