[Halld-offline] expected bandwidth offsite

[Richard Jones]

Matt and all,

I am running a 15TB transfer right now (4 parallel streams / job, 2
simultaneous jobs at a time) and am only seeing 15 MB/s averaged
(fluctuating, rising to 100MB/s and then sinking below 10MB/s for extended
periods before rebounding). Our network folks are looking into it, but we
have 10Gb/s all the way except for the last 3 feet. I suspect the problems
are on our end, since iperf gives me steady 900Mb/s between the same uconn
server and a perfsonar server at the lab. What did you use for the number
of parallel streams / job, and number of simultaneous jobs to get the
performance you saw?

Just so you know, I would be pretty happy to get 50MB/s sustained. By the
end of the summer, my goal is to get 100MB/s sustained, but one step at a
time.

-Richard Jones

On Mon, May 16, 2016 at 2:23 PM, Shepherd, Matthew <mashephe at indiana.edu>
wrote:

>
> Hi Sandy,
>
> (cc'ed to offline list in case anyone also cares about this topic or has
> input.)
>
> I copied about 10 TB of GlueX data (a calibration skim) from JLab to our
> machine here at Indiana.  I used the web-based globus-connect system.
>
> The source at JLab was:
>
> expphy/cache/halld/home/gxproj3/calib/GlueX-CalibRun-2016-04-04/
>
> The transfer took about 2.5 days (5/12 - 5/14) and when complete I had
> averaged 49 MB/s.
>
> 49 MB/s (roughly 400 Mb/s) is not too shabby, but I'd like to know what
> you think we might be able to get going offsite.
>
> Here, we've got 10 gigabit everywhere:  fiber coming into a switch and
> then into 10 gigabit interface cards.  Some informal local testing has
> produced speeds in the neighborhood of about 3-4 Gb/s, roughly ten times
> what I got from JLab.  However, that doesn't necessarily rule out a
> bottleneck somewhere outside the lab.
>
> The transfer rate is acceptable, but I thought I'd ask in case there is
> some obvious things we can do to get a few factors speed up.
>
> Matt
>
> ---------------------------------------------------------------------
> Matthew Shepherd, Associate Professor
> Department of Physics, Indiana University, Swain West 265
> 727 East Third Street, Bloomington, IN 47405
>
> Office Phone:  +1 812 856 5808
>
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20160516/d7ac7619/attachment-0002.html>