<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Hi Vardan,</div>
<div><br class="">
</div>
<div>What is the "reconstruction application"? The number I quoted was for standard simulation jobs. The larger jobs used for real data of course depend on the size of the job, but never more than 1 GB per thread.</div>
<div><br class="">
</div>
<div>-Nathan</div>
<div><br class="">
</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Feb 7, 2023, at 11:11 AM, Vardan Gyurjyan <<a href="mailto:gurjyan@jlab.org" class="">gurjyan@jlab.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
What is the reconstruction application's resident (not the virtual) memory usage for the exclusive usage of a node?<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
--------------------------------------------------<br class="">
Vardan H. Gyurjyan, Ph.D.<br class="">
Staff Scientist<br class="">
Thomas Jefferson Accelerator Facility<br class="">
Newport News, VA, 23606<br class="">
E-mail:<span class="Apple-converted-space"> </span><a href="mailto:gurjyan@jlab.org" style="color: purple; text-decoration: underline;" class="">gurjyan@jlab.org</a><br class="">
757-269-5879 (JLAB)<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0in 0in;" class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class=""><span style="font-size: 12pt;" class="">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;" class="">Clas12_software <<a href="mailto:clas12_software-bounces@jlab.org" class="">clas12_software-bounces@jlab.org</a>>
on behalf of Nathan Baltzell via Clas12_software <<a href="mailto:clas12_software@jlab.org" class="">clas12_software@jlab.org</a>><br class="">
<b class="">Reply-To:<span class="Apple-converted-space"> </span></b>Nathan Baltzell <<a href="mailto:baltzell@jlab.org" class="">baltzell@jlab.org</a>><br class="">
<b class="">Date:<span class="Apple-converted-space"> </span></b>Tuesday, February 7, 2023 at 11:05 AM<br class="">
<b class="">To:<span class="Apple-converted-space"> </span></b>clas12 software <<a href="mailto:clas12_software@jlab.org" class="">clas12_software@jlab.org</a>><br class="">
<b class="">Subject:<span class="Apple-converted-space"> </span></b>[Clas12_software] jlab batch job memory requests<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
FYI Everyone,<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
We've brought up memory efficiency of batch job requests at CLAS12 collaboration and software meetings in previous years. Lots of jobs requesting a lot more memory than they actually use can make the farm unnecessarily idle and significantly reduce throughput
for everyone.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
*** And now Scicomp has a larger initiative to improve farm efficiency, which includes contacting people running memory-inefficient jobs and potentially throttling their jobs if no action is taken. ***<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
You can check metrics of your batch jobs at:<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<a href="https://scicomp.jlab.org/" style="color: purple; text-decoration: underline;" class="">https://scicomp.jlab.org</a> <o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
There's a search feature at 'Slurm Jobs' (left sidebar) -> 'Jobs Query' (top), and 'Recent Jobs' (top), and also 'Memory Efficiency' (top). <o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Before launching a large number of new types of jobs, you can measure how much memory your jobs use. For example, by submitting a couple jobs and using that website, or by running your job interactively and checking in htop or ps or other system utilities.
And then set your SLURM/SWIF job memory request accordingly.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Note, standard CLAS12 simulation jobs (gemc plus recon-util) require less than 1.7 GB of memory.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
-Nathan</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</body>
</html>