[Jlab-scicomp-briefs] Scheduled Outage: Farm Nodes & /work Areas – Sept 23–24

[Wesley Moore]

We greatly appreciate your patience for this outage. From here on, please treat systems as production and report issues. Exclusions to this and overall status listed below.
Status of farm nodes: The majority of the farm nodes are back online.

  *   Farm19 nodes: Firmware updates are currently in progress for approximately 50% of the nodes in farm19. These nodes will be brought online as their updates are completed.
  *   ifarm2402: This node is currently offline but is scheduled to be restored today.
  *
ifarm2401: Users are asked to "play nice" on this node. Please use the shared resources responsibly, especially while ifarm2402 node is offline.
  *
GPU nodes: A small number of GPU nodes require maintenance. These will be restored as work on them is finished.
  *
JupyterHub: Should be working as expected, except for the mentioned mentioned offline GPU nodes.

________________________________
From: Wesley Moore <wmoore at jlab.org>
Sent: Wednesday, September 17, 2025 5:21 PM
To: Wesley Moore via Jlab-scicomp-briefs <jlab-scicomp-briefs at jlab.org>
Subject: Re: Scheduled Outage: Farm Nodes & /work Areas – Sept 23–24

Just a reminder. The farm outage and /work disk shelf replacement to begin Tuesday, September 23rd at 8am. See attached announcement for further details.

________________________________
From: Wesley Moore
Sent: Thursday, September 11, 2025 9:49 AM
To: Wesley Moore via Jlab-scicomp-briefs <jlab-scicomp-briefs at jlab.org>
Subject: Scheduled Outage: Farm Nodes & /work Areas – Sept 23–24

Dear SciComp Users,
A scheduled outage is planned for Tuesday, September 23 through noon on Wednesday, September 24, affecting all farm nodes, including ifarm and JupyterHub.
Reason for Outage:
A failing disk shelf affecting the /work areas requires urgent replacement. During this window, we will also perform:

  *   Reboots across farm nodes

  *   Application of Maintenance Day (MD) system patches

  *   Firmware updates to farm19, farm23, sciml

Affected Systems:

  *   All farm nodes

  *   /work areas

  *   ifarm

  *   JupyterHub

Downtime Window:

  *   Start: Monday, September 23, 8am

  *   End: By noon, Wednesday, September 24

While we aim to restore services as promptly as possible, this extended outage window allows for:

  *   Safety of on-site personnel

  *   Proper hardware replacement and testing

  *   Confidence in system stability before resuming production

A follow-up "all-clear" notice will be sent when systems are fully operational.
We appreciate your understanding and patience during this essential maintenance.
Best regards,
Scientific Operations Team

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/jlab-scicomp-briefs/attachments/20250924/59285666/attachment.htm>