<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hello Norman,<div class=""><br class=""></div><div class="">Thanks for the note.</div><div class="">We always set memory request as 4G, and it works well.</div><div class=""><br class=""></div><div class="">The problem is not due to memory. Instead, there is a maximum heap limit for JVM of the batch machine. According to tests, the limit is about 2G.</div><div class="">If we can not increase the limit, we need to be careful about the heap usage when making codes.</div><div class=""><br class=""></div><div class="">Recently, we tried to add histograms into DQM for all different triggers. The heap usage is over the limit (2G). </div><div class="">So we have to temporarily shut down drivers for some triggers, and add them back after fixing the issue.</div><div class=""><br class=""></div><div class="">Best regards,</div><div class="">Tongtong</div><div class=""><br class=""></div><div class=""><div><blockquote type="cite" class=""><div class="">On Aug 5, 2019, at 4:16 PM, Graf, Norman A. <<a href="mailto:ngraf@slac.stanford.edu" class="">ngraf@slac.stanford.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div id="divtagdefaultwrapper" dir="ltr" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-size: 12pt; font-family: Calibri, Helvetica, sans-serif;" class=""><div style="margin-top: 0px; margin-bottom: 0px;" class="">One thing to note is that job requests for large amounts of memory may restrict the available queues/machines the jobs will run on. Most likely not an issue at the moment, but something to watch out for.</div><div style="margin-top: 0px; margin-bottom: 0px;" class=""><br class=""></div><div style="margin-top: 0px; margin-bottom: 0px;" class="">Norman</div><br class=""><br class=""><div style="" class=""><hr tabindex="-1" style="display: inline-block; width: 633.078125px;" class=""><div id="divRplyFwdMsg" dir="ltr" class=""><font face="Calibri, sans-serif" style="font-size: 11pt;" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>Hps-analysis <<a href="mailto:hps-analysis-bounces@jlab.org" class="">hps-analysis-bounces@jlab.org</a>> on behalf of Tongtong Cao <<a href="mailto:tongtongcao1@gmail.com" class="">tongtongcao1@gmail.com</a>><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Monday, August 5, 2019 11:41 AM<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Nathan Baltzell <<a href="mailto:baltzell@jlab.org" class="">baltzell@jlab.org</a>><br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:hps-analysis@jlab.org" class="">hps-analysis@jlab.org</a><span class="Apple-converted-space"> </span><<a href="mailto:hps-analysis@jlab.org" class="">hps-analysis@jlab.org</a>>; hps-software <<a href="mailto:hps-software@slac.stanford.edu" class="">hps-software@slac.stanford.edu</a>><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [Hps-analysis] Requests for data cooking</font><div class=""> </div></div><div class="BodyFragment"><font size="2" class=""><span style="font-size: 11pt;" class=""><div class="PlainText">Hello Nathan,<br class=""><br class="">Thanks for the reminder.<br class="">The memory request has been be specified as 4G. No problem for memory.<br class=""><br class="">-Tongtong<br class=""><br class=""><br class="">> On Aug 5, 2019, at 1:49 PM, Nathan Baltzell <<a href="mailto:baltzell@jlab.org" class="">baltzell@jlab.org</a>> wrote:<br class="">><span class="Apple-converted-space"> </span><br class="">> Tongtong,<br class="">><span class="Apple-converted-space"> </span><br class="">> The memory request is (or at least should be) specified in your job submission setup.<br class="">><span class="Apple-converted-space"> </span><br class="">> -Nathan<br class="">><span class="Apple-converted-space"> </span><br class="">>> On Aug 5, 2019, at 12:20, Tongtong Cao <<a href="mailto:tongtongcao1@gmail.com" class="">tongtongcao1@gmail.com</a>> wrote:<br class="">>><span class="Apple-converted-space"> </span><br class="">>> Dear all,<br class="">>><span class="Apple-converted-space"> </span><br class="">>> Jobs for runs 10169 and 10170 triggered by hps_v10.cnf were submitted for cooking just now.<br class="">>><span class="Apple-converted-space"> </span><br class="">>> DQM histogram for runs 10136 and 10149, including separate plots for most of triggers, have been produced.<span class="Apple-converted-space"> </span><br class="">>> You can browse them through<span class="Apple-converted-space"> </span><a class="OWAAutoLink" id="LPlnk404599" href="https://hpsweb.jlab.org/dqm/dqm.php" previewremoved="true">https://hpsweb.jlab.org/dqm/dqm.php</a><br class="">>><span class="Apple-converted-space"> </span><br class="">>> We will add plots for extra triggers if heap usage issue is fixed.<br class="">>> There is a limit (seems ~2G) for the maximum heap size in the JLab batch machine. But the heap usage will be over the limit if too many DQM plots are included.<br class="">>><span class="Apple-converted-space"> </span><br class="">>> Best regards,<br class="">>> Tongtong<br class="">>><span class="Apple-converted-space"> </span><br class="">>>> On Aug 2, 2019, at 12:53 PM, Tongtong Cao <<a href="mailto:tongtongcao1@gmail.com" class="">tongtongcao1@gmail.com</a>> wrote:<br class="">>>><span class="Apple-converted-space"> </span><br class="">>>> Dear all,<br class="">>>><span class="Apple-converted-space"> </span><br class="">>>> Jobs for runs 10136 and 10149 were submitted for cooking just now.<br class="">>>><span class="Apple-converted-space"> </span><br class="">>>> For DQM plots of cooked runs, you can browse through<span class="Apple-converted-space"> </span><a class="OWAAutoLink" id="LPlnk850266" href="https://hpsweb.jlab.org/dqm/dqm.php" previewremoved="true">https://hpsweb.jlab.org/dqm/dqm.php</a><br class="">>>><span class="Apple-converted-space"> </span><br class="">>>> Best regards,<br class="">>>> Tongtong<br class="">>>><span class="Apple-converted-space"> </span><br class="">>>>> On Aug 1, 2019, at 11:04 AM, Tongtong Cao <<a href="mailto:tongtongcao1@gmail.com" class="">tongtongcao1@gmail.com</a>> wrote:<br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> Dear all,<br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> Since yesterday afternoon, we have been taking data with the trigger configuration of hps_v9_2.cnf<br class="">>>>> Jobs for runs 10128, 10129 and 10130 were submitted for cooking just now.<span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>> Best regards,<br class="">>>>> Tongtong<br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>>>> On Jul 31, 2019, at 1:24 PM, Tongtong Cao <<a href="mailto:tongtongcao1@gmail.com" class="">tongtongcao1@gmail.com</a>> wrote:<br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Dear all,<br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Since yesterday afternoon, we have been taken runs triggered by various trigger configurations.<br class="">>>>>> Jobs for runs 10096, 10097, 10101, 10103, 10104, 10115, 10117 and 10118 were submitted for cooking just now.<br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Run’s setup could be found at<span class="Apple-converted-space"> </span><a class="OWAAutoLink" id="LPlnk765592" href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_spreadsheets_d_1Ru4weeIOqtcpebiXKxFGCbPXGWXZF-2DW2JAolHcpvIFU_edit-23gid-3D43855609&d=DwIFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=JaSEOiNc_6InrJmbYDvKU2tZqhhONpIkbyl_AUnkDSY&m=mgXu2Pooq94AeSMSPL4GcyXy44W-1STlfK_8v1NoCAw&s=mlQFbQT6g6VYshnHCe4XsxhIqaXSo4E5NnbzuhT5kV0&e=" previewremoved="true">https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_spreadsheets_d_1Ru4weeIOqtcpebiXKxFGCbPXGWXZF-2DW2JAolHcpvIFU_edit-23gid-3D43855609&d=DwIFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=JaSEOiNc_6InrJmbYDvKU2tZqhhONpIkbyl_AUnkDSY&m=mgXu2Pooq94AeSMSPL4GcyXy44W-1STlfK_8v1NoCAw&s=mlQFbQT6g6VYshnHCe4XsxhIqaXSo4E5NnbzuhT5kV0&e=</a><span class="Apple-converted-space"> </span><br class="">>>>>> Trigger configuration files could be found at /usr/clas12/release/1.4.0/parms/trigger/HPS/Run2019/ in the clondaq3 machine.<br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>> Best regards,<br class="">>>>>> Tongtong<br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> On Jul 30, 2019, at 11:03 AM, Tongtong Cao <<a href="mailto:tongtongcao1@gmail.com" class="">tongtongcao1@gmail.com</a>> wrote:<br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> Dear all,<br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> More production runs with trigger hps_v9.cnf have been taken since yesterday.<br class="">>>>>>> Jobs for runs 10084, 10085, 10089, 10090, 10091, and 10092 have been submitted for cooking.<br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> As mentioned yesterday, DQM has been added into cooking processing.<br class="">>>>>>> Cooking outputs are saved in /volatile/hallb/hps/data/run2019/ConcurrentCook<br class="">>>>>>> Each run includes three folders to save outputs:<br class="">>>>>>> 1) .slcio files from reconstruction is saved in /volatile/hallb/hps/data/run2019/ConcurrentCook/${run}/recon<br class="">>>>>>> 2) .root files from DQM is saved in /volatile/hallb/hps/data/run2019/ConcurrentCook/${run}/dqm<br class="">>>>>>> 3) .root files from<span class="Apple-converted-space"> </span><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__make-5Froot.cc_&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=JaSEOiNc_6InrJmbYDvKU2tZqhhONpIkbyl_AUnkDSY&m=wlIGbnm_u-Rik4619akfx29iLxHVQbWtUhpdILBlxiE&s=0scit3XHBO3dH913Sy87tadZvocTJ4HuVgSAIrt-DS4&e=" class="">Make_Root.cc</a><span class="Apple-converted-space"> </span>are saved in /volatile/hallb/hps/data/run2019/ConcurrentCook/${run}/rootTree<br class="">>>>>>> Additionally, log files are saved in /volatile/hallb/hps/data/run2019/ConcurrentCook/logs<br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>> Best regards,<br class="">>>>>>> Tongtong<br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>>> On Jul 29, 2019, at 7:41 PM, Tongtong Cao <<a href="mailto:tongtongcao1@gmail.com" class="">tongtongcao1@gmail.com</a>> wrote:<br class="">>>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>>>> /volatile/hallb/hps/data/run2019/ConcurrentCook/${run}/dqm<br class="">>>>>>><span class="Apple-converted-space"> </span><br class="">>>>>><span class="Apple-converted-space"> </span><br class="">>>>><span class="Apple-converted-space"> </span><br class="">>>><span class="Apple-converted-space"> </span><br class="">>><span class="Apple-converted-space"> </span><br class="">>> _______________________________________________<br class="">>> Hps-analysis mailing list<br class="">>><span class="Apple-converted-space"> </span><a href="mailto:Hps-analysis@jlab.org" class="">Hps-analysis@jlab.org</a><br class="">>><span class="Apple-converted-space"> </span><a class="OWAAutoLink" id="LPlnk513860" href="https://mailman.jlab.org/mailman/listinfo/hps-analysis" previewremoved="true">https://mailman.jlab.org/mailman/listinfo/hps-analysis</a><br class="">><span class="Apple-converted-space"> </span><br class=""><br class=""><br class="">_______________________________________________<br class="">Hps-analysis mailing list<br class=""><a href="mailto:Hps-analysis@jlab.org" class="">Hps-analysis@jlab.org</a><br class=""><a class="OWAAutoLink" id="LPlnk55258" href="https://mailman.jlab.org/mailman/listinfo/hps-analysis" previewremoved="true">https://mailman.jlab.org/mailman/listinfo/hps-analysis</a><br class=""></div></span></font></div></div></div></div></blockquote></div><br class=""></div></body></html>