<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<font size="+1">Matt,<br>
<br>
The gateway node is intentionaly not tuned for best TCP streaming
so that we don't consume more than 50% of the lab's bandwidth.
Further, JLab is on a MAN shared with ODU and W&M, with a
single uplink to D.C. People generally get 2-5 Gbps using globus
online to do the transfers (gridftp under the hood).<br>
<br>
You also need to keep in mind that the JLab file systems are all
pretty busy serving the batch system. It is not at all unusual
for a single file stream to only achieve 1 Gbps. If you want to
use scp, you will need to use multiple instances of it to match
what globus and gridftp can achieve.<br>
<br>
Chip<br>
<br>
</font>
<div class="moz-cite-prefix">On 1/27/15 2:01 PM, Shepherd, Matthew
wrote:<br>
</div>
<blockquote
cite="mid:9AFDBE8B-98C5-47F2-B4C7-D966D7A52A4E@indiana.edu"
type="cite">
<pre wrap="">Hi all,
Does anyone have suggestions for doing high speed
file copies offsite?
I consulted this page:
<a class="moz-txt-link-freetext" href="https://scicomp.jlab.org/docs/node/11">https://scicomp.jlab.org/docs/node/11</a>
Which suggests to use the node: globus.jlab.org
This node has real limited functionality. I can't get rsync to
work and there is no tar to use with ssh.
I tried just:
scp -r
This works, but I'd like to know if I can go faster.
I'm getting speeds that are orders of magnitude below
the 10 Gbps pipe that is at JLab and also at my
end. I'm sure there is other competing traffic, but
I think part of the problem might be the scp is poorly
optimized for coping many small files.
Any experience with this in the offline group?
Matt
---------------------------------------------------------------------
Matthew Shepherd, Associate Professor
Department of Physics, Indiana University, Swain West 265
727 East Third Street, Bloomington, IN 47405
Office Phone: +1 812 856 5808
_______________________________________________
Halld-offline mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Halld-offline@jlab.org">Halld-offline@jlab.org</a>
<a class="moz-txt-link-freetext" href="https://mailman.jlab.org/mailman/listinfo/halld-offline">https://mailman.jlab.org/mailman/listinfo/halld-offline</a>
</pre>
</blockquote>
<br>
</body>
</html>