<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="text-align: left; text-indent: 0px; line-height: 28px; margin: 30px 0px 16px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
We greatly appreciate your patience for this outage. From here on, please treat systems as production and report issues. Exclusions to this and overall status listed below.</div>
<div style="text-align: left; text-indent: 0px; line-height: 28px; margin: 30px 0px 16px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="font-weight: 700;">Status of farm nodes:</span> The majority of the farm nodes are back online.</div>
<ul style="text-align: left; margin: 16px 0px; padding-left: 0px;" data-processed="true" data-complete="true">
<li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); margin: 0px 0px 16px; list-style-position: initial; list-style-type: disc;">
<span role="presentation" style="font-weight: 700;">Farm19 nodes:</span> Firmware updates are currently in progress for approximately 50% of the nodes in farm19. These nodes will be brought online as their updates are completed.</li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); margin: 0px 0px 16px; list-style-position: initial; list-style-type: disc;">
<span role="presentation" style="font-weight: 700;">ifarm2402:</span> This node is currently offline but is scheduled to be restored today.</li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); margin: 0px 0px 16px; list-style-position: initial; list-style-type: disc;">
<div role="presentation"><span style="font-weight: 700;">ifarm2401:</span> Users are asked to "play nice" on this node. Please use the shared resources responsibly, especially while ifarm2402 node is offline.</div>
</li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); margin: 0px 0px 16px; list-style-position: initial; list-style-type: disc;">
<div role="presentation"><span style="font-weight: 700;">GPU nodes:</span> A small number of GPU nodes require maintenance. These will be restored as work on them is finished.</div>
</li><li style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); margin-right: 0px; margin-left: 0px;">
<div role="presentation"><span style="font-weight: 700;">JupyterHub:</span> Should be working as expected, except for the mentioned mentioned offline GPU nodes.</div>
</li></ul>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Wesley Moore <wmoore@jlab.org><br>
<b>Sent:</b> Wednesday, September 17, 2025 5:21 PM<br>
<b>To:</b> Wesley Moore via Jlab-scicomp-briefs <jlab-scicomp-briefs@jlab.org><br>
<b>Subject:</b> Re: Scheduled Outage: Farm Nodes & /work Areas – Sept 23–24</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div class="x_elementToProof" style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Just a reminder. The farm outage and /work disk shelf replacement to begin <u>Tuesday, September 23</u><sup><u>rd</u></sup><u> at 8am</u>. See attached announcement for further details.</div>
<div id="x_appendonsend"></div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg">
<div style="direction:ltr; font-family:Calibri,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<b>From:</b> Wesley Moore<br>
<b>Sent:</b> Thursday, September 11, 2025 9:49 AM<br>
<b>To:</b> Wesley Moore via Jlab-scicomp-briefs <jlab-scicomp-briefs@jlab.org><br>
<b>Subject:</b> Scheduled Outage: Farm Nodes & /work Areas – Sept 23–24</div>
<div style="direction:ltr"> </div>
</div>
<div style="direction:ltr; margin-top:1em; margin-bottom:1em; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Dear SciComp Users,</div>
<div style="direction:ltr; margin-top:1em; margin-bottom:1em; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
A <b>scheduled outage</b> is planned for <b>Tuesday, September 23 through noon on Wednesday, September 24</b>, affecting
<b>all farm nodes</b>, including <b>ifarm</b> and <b>JupyterHub</b>.</div>
<div style="direction:ltr; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>Reason for Outage:</b></div>
<div style="direction:ltr; margin-top:1em; margin-bottom:1em; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
A <b>failing disk shelf</b> affecting the <code>/work</code> areas requires urgent replacement. During this window, we will also perform:</div>
<ul data-editing-info="{"applyListStyleFromLevel":true}" style="direction:ltr; margin-top:0px; margin-bottom:0px; list-style-type:disc">
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; line-height:1.2; margin-top:0px; margin-bottom:0px">
Reboots across farm nodes</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; line-height:1.2; margin-top:0px; margin-bottom:0px">
Application of <b>Maintenance Day (MD)</b> system patches</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; line-height:1.2; margin-top:0px; margin-bottom:0px">
Firmware updates to farm19, farm23, sciml</p>
</li></ul>
<div style="direction:ltr; line-height:1.2; margin-top:0px; margin-bottom:0px; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="direction:ltr; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>Affected Systems:</b></div>
<ul data-end="654" data-start="595" style="direction:ltr">
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px">All farm nodes</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px"><code>/work</code> areas</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px">ifarm</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px">JupyterHub</p>
</li></ul>
<div style="direction:ltr; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>Downtime Window:</b></div>
<ul data-end="760" data-start="677" style="direction:ltr">
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px"><b>Start:</b> Monday, September 23, 8am</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px"><b>End:</b> By
<b>noon, Wednesday, September 24</b></p>
</li></ul>
<div style="direction:ltr; margin-top:1em; margin-bottom:1em; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
While we aim to restore services as promptly as possible, this extended outage window allows for:</div>
<ul data-end="997" data-start="860" style="direction:ltr">
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px">Safety of on-site personnel</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px">Proper hardware replacement and testing</p>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<p role="presentation" style="direction:ltr; margin-top:0px; margin-bottom:0px">Confidence in system stability before resuming production</p>
</li></ul>
<div style="direction:ltr; margin-top:1em; margin-bottom:1em; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
A follow-up <b>"all-clear" notice</b> will be sent when systems are fully operational.</div>
<div style="direction:ltr; margin-top:1em; margin-bottom:1em; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
We appreciate your understanding and patience during this essential maintenance.</div>
<div style="direction:ltr; margin-top:1em; margin-bottom:1em; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Best regards,<br>
<b>Scientific Operations Team</b></div>
<div style="direction:ltr; font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
</div>
</body>
</html>