<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Century;
panose-1:2 4 6 4 5 5 5 2 3 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Dear Helpdesk, <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We are about to start our second mass data production from the SBS GMN dataset. This will require roughly 100 kcore-hours to reduce about 2 PB of data down to about 10 TB or less. Each of these ~100k single-thread reconstruction jobs will
produce a single ROOT output file ranging in size from about 25 MB at the low end to over 200 MB at the high end. Our intention is to write each of these files to both volatile and to tape under a similar directory structure, and to automate the writing to
tape from directly within the farm job itself using the “-output” option of swif2 add-job.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am writing to give you a heads-up about this plan and ask you to weigh in about the tape aspect because I know these file sizes are on the small side from a tape efficiency standpoint, in terms of overhead of staging in/out, etc. However,
we can’t NOT write them to tape since they cost (in total) about 100 kcore-hours to reproduce.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Although the number of these files will be relatively large (about 100k), I normally expect them to be available on /volatile and/or pinned in /cache for further analysis. Moreover, the total amount of data will be relatively small (<10
TB), such that the entire reduced dataset can easily be moved offsite and/or to people’s local machines/workstations/computing setups. As such, I don’t expect a huge amount of “churn” from repeated caching of these files from tape by analysis jobs on the farm
itself over the remaining lifecycle of the GMN analysis. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The alternative would be to write the files to /volatile and then attempt to combine them into larger files and/or compressed tarballs that would be more efficient from a tape standpoint. But that is less convenient than writing directly
to tape and would require more manual bookkeeping/scripting on our end. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Originally I wanted to chain 10 CODA files of 20 GB each into a single replay job of about 10 core-hours, which would have led to bigger root files that would be more efficient for tape storage, but my experience with that approach was
that it reduced our throughput on the farm and increased the failure rate of our jobs due to the logistical overhead of staging 200 GB of input data and requiring it to be available for ~10 hours, as opposed to 20 GB of input data for ~1 hour.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Anyway, if you don’t mind us writing the ~100k ROOT files of 25-200 MB each to tape, then we will write them directly from the farm jobs, which is the most convenient option.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best,<o:p></o:p></p>
<p class="MsoNormal">Andrew<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Century",serif;color:black;mso-ligatures:none">Andrew Puckett<br>
Associate Professor, Physics Department<br>
University of Connecticut<br>
196 Auditorium Road, Unit 3046<br>
Storrs, CT 06269-3046<br>
Office phone: (860) 486-7137<br>
</span><span style="font-size:12.0pt;font-family:"Century",serif;mso-ligatures:none"><a href="https://puckett.physics.uconn.edu" title="https://puckett.physics.uconn.edu"><span style="color:#0563C1">https://puckett.physics.uconn.edu</span></a><span style="color:black"><br>
</span><a href="mailto:andrew.puckett@uconn.edu"><span style="color:#0563C1">puckett@jlab.org</span></a></span><o:p></o:p></p>
</div>
</div>
</body>
</html>