[Halld-offline] Fwd: Fwd: Re: Data Challenge Meeting Minutes, January 31, 2014

[Richard Jones]

Sean,

Yes, Globus Online is very much like dropbox.  So for individual users and
a desktop compute platform, pushing a few large files around, it is a handy
solution.  I recently attended a Globus Online workshop, where I asked if
it would be suitable for a file access and storage for grid jobs, the
person replied that this might be a future area where effort might be
directed, but at present it was not recommended, because

   1. Globus Online was designed for the interactive environment, not batch
   2. The authentication is completely different and orthogonal to the grid
   authentication infrastructure, so none of the privs that are shipped around
   with grid jobs can be used to authenticate GO transactions.

 I would never consider using dropbox for grid jobs, even though with the
right amount of effort and determination it might be made to work.  I think
it is not a good fit.

I think xrootd is an obvious choice for root files.  We run a xrootd
service here at UConn, in parallel with the dcache SRM.  By the time they
are reduced to root files, the volume of the data is very compact, like
DST's, and a distributed xrootd service makes a lot of sense in that
context, especially when combined with PROOF.  We use PROOF fairly heavily
in our group, because of how it allows hundreds of root processing threads
to run simultaneously within a single application operating on a single
tree.

-Richard Jones


On Tue, Feb 4, 2014 at 12:03 PM, Sean Dobbs <s-dobbs at northwestern.edu>wrote:

> Dear all,
>
> Has there ever been any discussion of running an XrootD service at JLab?
>  It is an increasingly popular service for accessing data over the grid
> these days.
>
> BTW, I've used Globus Online on the desktop at a workshop before, and it
> was really easy - about as hard as setting up Dropbox or a similar service.
>
>
> Cheers,
> Sean
>
>
> On Tue, Feb 4, 2014 at 8:23 AM, Mark Ito <marki at jlab.org> wrote:
>
>>  >From Sandy Philpott:
>>
>>
>> -------- Original Message --------  Subject: Re: [Halld-offline] Data
>> Challenge Meeting Minutes, January 31, 2014  Date: Mon, 3 Feb 2014
>> 16:58:51 -0500 (EST)  From: Sandy Philpott <philpott at jlab.org><philpott at jlab.org>  To:
>> Mark Ito <marki at jlab.org> <marki at jlab.org>
>>
>> Hi Mark,
>>
>> As discussed...
>>
>> IT/SciComp is ready for the Hall D data challenge. When we get your test
>> description details -- rates to/from tape, disk requirements, timeframe, etc,
>> we'll flip the farm nodes in HPC back to the farm, along with the 12s nodes
>> needed to get 2250 cores total in the cluster, so that Hall D will be able to
>> run 1250 cores with other jobs running.  The nodes to be moved from HPC will have
>> to drain jobs, which can take up to 48 hours.  We want to make sure you are
>> ready to run when we do it so that we don't waste CPUs sitting idle.
>>
>> There's no SRM ability at JLab, and recall from the review that reviewers
>> suggested GlueX consider something besides SRM.  At one point in 2010 Curtis
>> had  firewall openings via our Cyber group for client tools installed
>> in user space on the ifarm nodes; check with Greg and company on that status
>> if still needed, as network configs for ifarms have undergone changed since then.
>>
>> IT supports Globus Online for offsite data transfers using the site's 10 gigabit
>> pipe; info available at https://scicomp.jlab.org/docs/?q=node/11
>>
>> Let us know when you're ready to run and what you need, then let the
>> node flipping and data challenge begin. I understand from today's discussion
>> you expect the timeframe to be sometime near the end of February.  Note that
>> the farm is currently almost idle at this point, so you shouldn't have any
>> trouble getting test jobs and prototypes started in the meantime.
>>
>> Regards,
>> Sandy
>>
>> ----- Original Message -----
>> From: "Mark Ito" <marki at jlab.org> <marki at jlab.org>
>> To: "GlueX Offline Software Email List" <halld-offline at jlab.org> <halld-offline at jlab.org>
>> Sent: Monday, February 3, 2014 1:05:43 PM
>> Subject: [Halld-offline] Data Challenge Meeting Minutes, January 31, 2014
>>
>> ...
>>
>> 9. Is the JLab CC ready for us?
>> + Mark will talk to Sandy about our plans.
>> 10. What ability will we have for SRM at Jefferson Lab?
>> + Mark will talk to Sandy about the status of the system.
>>
>> ...
>>
>>
>>
>>
>
>
> --
> Dr. Sean Dobbs
> Department of Physics & Astronomy
> Northwestern University
> phone: 847-467-2826
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.jlab.org/pipermail/halld-offline/attachments/20140204/6aadf6ee/attachment-0001.html