[Halld-offline] advice on high speed file copies offsite

[Shepherd, Matthew]

Chip and Mike,

Thanks for your replies.  I'll try the globus online
method to do transfers.  I was getting only about
100-200 Mbs using scp and trying to find the bottleneck.  

Matt

---------------------------------------------------------------------
Matthew Shepherd, Associate Professor
Department of Physics, Indiana University, Swain West 265
727 East Third Street, Bloomington, IN 47405

Office Phone:  +1 812 856 5808

> On Jan 27, 2015, at 2:10 PM, Chip Watson <watson at jlab.org> wrote:
> 
> Matt,
> 
> The gateway node is intentionaly not tuned for best TCP streaming so that we don't consume more than 50% of the lab's bandwidth.  Further, JLab is on a MAN shared with ODU and W&M, with a single uplink to D.C.  People generally get 2-5 Gbps using globus online to do the transfers (gridftp under the hood).
> 
> You also need to keep in mind that the JLab file systems are all pretty busy serving the batch system.  It is not at all unusual for a single file stream to only achieve 1 Gbps.  If you want to use scp, you will need to use multiple instances of it to match what globus and gridftp can achieve.
> 
> Chip
> 
> On 1/27/15 2:01 PM, Shepherd, Matthew wrote:
>> Hi all,
>> 
>> Does anyone have suggestions for doing high speed
>> file copies offsite?
>> 
>> I consulted this page:
>> 
>> 
>> https://scicomp.jlab.org/docs/node/11
>> 
>> 
>> Which suggests to use the node:  globus.jlab.org
>> 
>> This node has real limited functionality.  I can't get rsync to
>> work and there is no tar to use with ssh.
>> 
>> I tried just:
>> 
>> scp -r
>> 
>> This works, but I'd like to know if I can go faster.
>> I'm getting speeds that are orders of magnitude below
>> the 10 Gbps pipe that is at JLab and also at my
>> end.  I'm sure there is other competing traffic, but
>> I think part of the problem might be the scp is poorly
>> optimized for coping many small files.
>> 
>> Any experience with this in the offline group?
>> 
>> Matt
>> 
>> ---------------------------------------------------------------------
>> Matthew Shepherd, Associate Professor
>> Department of Physics, Indiana University, Swain West 265
>> 727 East Third Street, Bloomington, IN 47405
>> 
>> Office Phone:  +1 812 856 5808
>> 
>> 
>> _______________________________________________
>> Halld-offline mailing list
>> 
>> Halld-offline at jlab.org
>> https://mailman.jlab.org/mailman/listinfo/halld-offline
> 
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline