From tknelson at slac.stanford.edu Tue Dec 2 17:13:29 2025 From: tknelson at slac.stanford.edu (Timothy Nelson) Date: Tue, 02 Dec 2025 22:13:29 -0000 Subject: [Hps] [EXTERNAL] [Indico] [Event reminder] Weekly Meeting - reconstruction focus (03/12/2025, 09:00 America/Los_Angeles) Message-ID: <176471360942.2697267.6803712493239603186@indico.slac.stanford.edu> Please note that the event "Weekly Meeting - reconstruction focus" will start on 3 Dec 2025, 09:00 (America/Los_Angeles). You can access the full event here: https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10203&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=Q_DOvL0XxEgJDJaBoBufB6h0vVvSxumUmIzxJUB5_JPvHRzdhYhAGT0i1wgd6jnv&s=tZ7pyCIEIQ8Y6W9z7ExrWcfwEXCRz0xOFfQDzkCvBlY&e= Description ----------- Zoom link:?https://urldefense.proofpoint.com/v2/url?u=https-3A__stanford.zoom.us_j_93402131889-3Fpwd-3DaPDTt6wDZJcMcQQ3AGnAEagIhJEDVF.1&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=Q_DOvL0XxEgJDJaBoBufB6h0vVvSxumUmIzxJUB5_JPvHRzdhYhAGT0i1wgd6jnv&s=0eOlPh_tDJwRwMzuNH88rodjsz-1Ir63-S6Ya1Q4-No&e= Agenda for Weekly Meeting - reconstruction focus ------------------------------------------------ - [09:00 - 09:05] News / EC Report - [09:05 - 09:10] PPC Report - [09:15 - 10:20] Group Summaries: Group Updates - [09:15 - 09:20] Analysis Update - [09:20 - 09:50] Reconstruction Update - [09:50 - 09:55] Monte Carlo Update - [09:55 - 10:00] Detector Update - [10:20 - 10:25] upcoming meetings / AOB -- Indico :: Email Notifier https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10203&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=Q_DOvL0XxEgJDJaBoBufB6h0vVvSxumUmIzxJUB5_JPvHRzdhYhAGT0i1wgd6jnv&s=tZ7pyCIEIQ8Y6W9z7ExrWcfwEXCRz0xOFfQDzkCvBlY&e= -------------- next part -------------- A non-text attachment was scrubbed... Name: event.ics Type: text/calendar Size: 536 bytes Desc: not available URL: From jlab-scicomp-briefs at jlab.org Thu Dec 4 10:33:39 2025 From: jlab-scicomp-briefs at jlab.org (Wesley Moore via Jlab-scicomp-briefs) Date: Thu, 4 Dec 2025 15:33:39 +0000 Subject: [Hps] [Jlab-scicomp-briefs] Summary of recent filesystem/lustre issues Message-ID: We wanted to provide a brief summary of the recent stability issues some users have experienced. The primary cause has been failing 20TB disks. When these disks begin to die, they can hang in a way that causes ZFS to suspend I/O, which in turn leads to system-wide filesystem stalls and unresponsiveness. In addition, we also identified and replaced a bad memory module (DIMM) in one of the SCOSS systems, which likely caused ? or contributed to ? several of the crashes we observed. These hardware failures together were the major sources of our recent problems. We will continue monitoring closely. Best regards, Wesley, on behalf of the Scientific Computing Operations Team -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -- This is an announcement-only list for Jefferson Lab Scientific Computing Updates . Subscription and List Archive: https://mailman.jlab.org/mailman/listinfo/jlab-scicomp-briefs For help: https://jlab.servicenowservices.com/scicomp From jlab-scicomp-briefs at jlab.org Mon Dec 8 12:46:14 2025 From: jlab-scicomp-briefs at jlab.org (Wesley Moore via Jlab-scicomp-briefs) Date: Mon, 8 Dec 2025 17:46:14 +0000 Subject: [Hps] [Jlab-scicomp-briefs] Tape Library Outage Message-ID: Dear Users, The tape library is currently down for repairs. IBM is actively working to resolve the issue. At this time, we do not have an estimated timeline for restoration of service. We will provide updates as soon as more information becomes available. Best regards, Wesley, on behalf of the Scientific Computing Operations Team -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -- This is an announcement-only list for Jefferson Lab Scientific Computing Updates . Subscription and List Archive: https://mailman.jlab.org/mailman/listinfo/jlab-scicomp-briefs For help: https://jlab.servicenowservices.com/scicomp From jlab-scicomp-briefs at jlab.org Mon Dec 8 15:41:29 2025 From: jlab-scicomp-briefs at jlab.org (Wesley Moore via Jlab-scicomp-briefs) Date: Mon, 8 Dec 2025 20:41:29 +0000 Subject: [Hps] [Jlab-scicomp-briefs] Tape Library Outage In-Reply-To: References: Message-ID: Tape library operations resumed. The tape library has resumed processing requests normally. IBM will be performing additional work on the system tomorrow; however, this maintenance is not expected to cause any service disruptions. We will continue monitoring the system closely and provide further updates as needed. Thank you for your patience. Best regards, Wesley, on behalf of the Scientific Computing Operations Team ________________________________ From: Jlab-scicomp-briefs on behalf of Wesley Moore via Jlab-scicomp-briefs Sent: Monday, December 8, 2025 12:46 PM To: Wesley Moore via Jlab-scicomp-briefs Subject: [Jlab-scicomp-briefs] Tape Library Outage Dear Users, The tape library is currently down for repairs. IBM is actively working to resolve the issue. At this time, we do not have an estimated timeline for restoration of service. We will provide updates as soon as more information becomes available. Best regards, Wesley, on behalf of the Scientific Computing Operations Team -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -- This is an announcement-only list for Jefferson Lab Scientific Computing Updates . Subscription and List Archive: https://mailman.jlab.org/mailman/listinfo/jlab-scicomp-briefs For help: https://jlab.servicenowservices.com/scicomp From tknelson at slac.stanford.edu Tue Dec 9 10:38:53 2025 From: tknelson at slac.stanford.edu (Timothy Nelson) Date: Tue, 09 Dec 2025 15:38:53 -0000 Subject: [Hps] [EXTERNAL] [Indico] [Event reminder] Weekly Meeting - Monte Carlo Focus (09/12/2025, 09:00 America/Los_Angeles) Message-ID: <176529473355.2697267.10464644185424315044@indico.slac.stanford.edu> Please note that the event "Weekly Meeting - Monte Carlo Focus" will start on 9 Dec 2025, 09:00 (America/Los_Angeles). You can access the full event here: https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10209&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=v8BJL-5CazXo6NwHemZ_EnkJLdEAyyanWZKcN5_zhddN5APnVFn0VJ1P2iwzxwaU&s=g-_SLokLWnqSfjscknfYWKBDODGieLea6nd9r0J6uzM&e= Description ----------- Zoom link:?https://urldefense.proofpoint.com/v2/url?u=https-3A__stanford.zoom.us_j_93402131889-3Fpwd-3DaPDTt6wDZJcMcQQ3AGnAEagIhJEDVF.1&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=v8BJL-5CazXo6NwHemZ_EnkJLdEAyyanWZKcN5_zhddN5APnVFn0VJ1P2iwzxwaU&s=W1qNYrTj1ZO8TRwoUts5GLBtm1ARF5Ztph1MBpe3GZc&e= Agenda for Weekly Meeting - Monte Carlo Focus --------------------------------------------- - [09:00 - 09:05] News / EC Report - [09:05 - 09:10] PPC Report - [09:10 - 09:55] Group Summaries: Group Updates - [09:10 - 09:15] Analysis Update - [09:15 - 09:20] Reconstruction Update - [09:20 - 09:50] Monte Carlo Update - [09:50 - 09:55] Detector Update - [09:55 - 10:00] upcoming meetings / AOB -- Indico :: Email Notifier https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10209&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=v8BJL-5CazXo6NwHemZ_EnkJLdEAyyanWZKcN5_zhddN5APnVFn0VJ1P2iwzxwaU&s=g-_SLokLWnqSfjscknfYWKBDODGieLea6nd9r0J6uzM&e= -------------- next part -------------- A non-text attachment was scrubbed... Name: event.ics Type: text/calendar Size: 533 bytes Desc: not available URL: From tknelson at slac.stanford.edu Tue Dec 9 11:01:31 2025 From: tknelson at slac.stanford.edu (Nelson, Timothy Knight) Date: Tue, 9 Dec 2025 16:01:31 +0000 Subject: [Hps] [EXTERNAL] [Indico] [Event reminder] Weekly Meeting - Monte Carlo Focus (09/12/2025, 09:00 America/Los_Angeles) In-Reply-To: <176529473355.2697267.10464644185424315044@indico.slac.stanford.edu> References: <176529473355.2697267.10464644185424315044@indico.slac.stanford.edu> Message-ID: Sorry? wrong date: will move to tomorrow! Tim > On Dec 9, 2025, at 7:38 AM, SLAC Indico wrote: > > Please note that the event "Weekly Meeting - Monte Carlo Focus" will start on 9 Dec 2025, 09:00 (America/Los_Angeles). > > You can access the full event here: > https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10209&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=v8BJL-5CazXo6NwHemZ_EnkJLdEAyyanWZKcN5_zhddN5APnVFn0VJ1P2iwzxwaU&s=g-_SLokLWnqSfjscknfYWKBDODGieLea6nd9r0J6uzM&e= > > Description > ----------- > > Zoom link: https://urldefense.proofpoint.com/v2/url?u=https-3A__stanford.zoom.us_j_93402131889-3Fpwd-3DaPDTt6wDZJcMcQQ3AGnAEagIhJEDVF.1&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=v8BJL-5CazXo6NwHemZ_EnkJLdEAyyanWZKcN5_zhddN5APnVFn0VJ1P2iwzxwaU&s=W1qNYrTj1ZO8TRwoUts5GLBtm1ARF5Ztph1MBpe3GZc&e= > > > Agenda for Weekly Meeting - Monte Carlo Focus > --------------------------------------------- > > - [09:00 - 09:05] News / EC Report > - [09:05 - 09:10] PPC Report > - [09:10 - 09:55] Group Summaries: Group Updates > - [09:10 - 09:15] Analysis Update > - [09:15 - 09:20] Reconstruction Update > - [09:20 - 09:50] Monte Carlo Update > - [09:50 - 09:55] Detector Update > - [09:55 - 10:00] upcoming meetings / AOB > > -- > Indico :: Email Notifier > https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10209&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=v8BJL-5CazXo6NwHemZ_EnkJLdEAyyanWZKcN5_zhddN5APnVFn0VJ1P2iwzxwaU&s=g-_SLokLWnqSfjscknfYWKBDODGieLea6nd9r0J6uzM&e= _______________________________________________ > Hps mailing list > Hps at jlab.org > https://mailman.jlab.org/mailman/listinfo/hps From tknelson at slac.stanford.edu Tue Dec 9 11:02:57 2025 From: tknelson at slac.stanford.edu (Timothy Nelson) Date: Tue, 09 Dec 2025 16:02:57 -0000 Subject: [Hps] [EXTERNAL] [Indico] [Event reminder] Weekly Meeting - Monte Carlo Focus (10/12/2025, 09:00 America/Los_Angeles) Message-ID: <176529617777.2697267.17250412633327921080@indico.slac.stanford.edu> Please note that the event "Weekly Meeting - Monte Carlo Focus" will start on 10 Dec 2025, 09:00 (America/Los_Angeles). You can access the full event here: https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10209&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=ePhQLjd5jmw5U3yNqzdTWDBn1F0TbUzj-BuMS4k7-M6v-vC2Ygh8aPspshawP3Vq&s=lVJxqAcp4KzMl0CDfiDkFAZeLj01ukKNitUsu8netAc&e= Description ----------- Zoom link:?https://urldefense.proofpoint.com/v2/url?u=https-3A__stanford.zoom.us_j_93402131889-3Fpwd-3DaPDTt6wDZJcMcQQ3AGnAEagIhJEDVF.1&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=ePhQLjd5jmw5U3yNqzdTWDBn1F0TbUzj-BuMS4k7-M6v-vC2Ygh8aPspshawP3Vq&s=KcOJoZXqTlqtIYjeo4rmn1nXrDj510FpH4lYIDLJyk0&e= Agenda for Weekly Meeting - Monte Carlo Focus --------------------------------------------- - [09:00 - 09:05] News / EC Report - [09:05 - 09:10] PPC Report - [09:10 - 09:55] Group Summaries: Group Updates - [09:10 - 09:15] Analysis Update - [09:15 - 09:20] Reconstruction Update - [09:20 - 09:50] Monte Carlo Update - [09:50 - 09:55] Detector Update - [09:55 - 10:00] upcoming meetings / AOB -- Indico :: Email Notifier https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10209&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=ePhQLjd5jmw5U3yNqzdTWDBn1F0TbUzj-BuMS4k7-M6v-vC2Ygh8aPspshawP3Vq&s=lVJxqAcp4KzMl0CDfiDkFAZeLj01ukKNitUsu8netAc&e= -------------- next part -------------- A non-text attachment was scrubbed... Name: event.ics Type: text/calendar Size: 533 bytes Desc: not available URL: From jlab-scicomp-briefs at jlab.org Fri Dec 12 09:53:07 2025 From: jlab-scicomp-briefs at jlab.org (Wesley Moore via Jlab-scicomp-briefs) Date: Fri, 12 Dec 2025 14:53:07 +0000 Subject: [Hps] [Jlab-scicomp-briefs] December 17 Compute Cluster Power Outage Message-ID: Dear Users, On December 17th, there will be a power outage affecting all compute clusters to allow a contractor to install new PDUs. The outage window will begin at 6:00 AM. Due to the nature of the work, we can only provide a general timeframe, and we expect the interruption to last most of the day. Full recovery time is still to be determined. As always, we will prioritize restoring core production services first so jobs can resume as quickly as possible. Affected services include, but are not limited to: * General compute cluster availability (Farm & QCD) * JupyterHub * Tape library Systems expected to be unaffected by the outage: * Interactives (ifarm &. qcdi) * Lustre filesystems * Work disk filesystems * code.jlab.org and associated runners To reduce the impact on users, we will keep Maintenance Day (December 16th) very light to help ensure no user-facing disruptions to running jobs ahead of the outage. We will provide updates as more information becomes available. Best regards, Wesley, on behalf of the Scientific Computing Operations Team -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -- This is an announcement-only list for Jefferson Lab Scientific Computing Updates . Subscription and List Archive: https://mailman.jlab.org/mailman/listinfo/jlab-scicomp-briefs For help: https://jlab.servicenowservices.com/scicomp From jlab-scicomp-briefs at jlab.org Mon Dec 15 14:00:00 2025 From: jlab-scicomp-briefs at jlab.org (Wesley Moore via Jlab-scicomp-briefs) Date: Mon, 15 Dec 2025 19:00:00 +0000 Subject: [Hps] [Jlab-scicomp-briefs] December 17 Compute Cluster Power Outage In-Reply-To: References: Message-ID: Dear Users, This is a gentle reminder of the planned compute cluster outage on December 17, beginning at 6:00 AM, to support contractor work installing new PDUs. Please see previous email for details. Please plan accordingly and ensure any critical work is completed ahead of the outage. As always, we will prioritize restoring core production services as quickly as possible once power is restored. Thank you for your understanding. Best regards, Wesley, on behalf of the Scientific Computing Operations Team ________________________________ From: Jlab-scicomp-briefs on behalf of Wesley Moore via Jlab-scicomp-briefs Sent: Friday, December 12, 2025 9:53 AM To: Wesley Moore via Jlab-scicomp-briefs ; lqcd-users at jlab.org ; epsci at jlab.org Subject: [Jlab-scicomp-briefs] December 17 Compute Cluster Power Outage Dear Users, On December 17th, there will be a power outage affecting all compute clusters to allow a contractor to install new PDUs. The outage window will begin at 6:00 AM. Due to the nature of the work, we can only provide a general timeframe, and we expect the interruption to last most of the day. Full recovery time is still to be determined. As always, we will prioritize restoring core production services first so jobs can resume as quickly as possible. Affected services include, but are not limited to: * General compute cluster availability (Farm & QCD) * JupyterHub * Tape library Systems expected to be unaffected by the outage: * Interactives (ifarm &. qcdi) * Lustre filesystems * Work disk filesystems * code.jlab.org and associated runners To reduce the impact on users, we will keep Maintenance Day (December 16th) very light to help ensure no user-facing disruptions to running jobs ahead of the outage. We will provide updates as more information becomes available. Best regards, Wesley, on behalf of the Scientific Computing Operations Team -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -- This is an announcement-only list for Jefferson Lab Scientific Computing Updates . Subscription and List Archive: https://mailman.jlab.org/mailman/listinfo/jlab-scicomp-briefs For help: https://jlab.servicenowservices.com/scicomp From tknelson at slac.stanford.edu Wed Dec 17 00:17:13 2025 From: tknelson at slac.stanford.edu (Timothy Nelson) Date: Wed, 17 Dec 2025 05:17:13 -0000 Subject: [Hps] [EXTERNAL] [Indico] [Event reminder] Weekly Meeting - data processing focus! (17/12/2025, 09:00 America/Los_Angeles) Message-ID: <176594863333.2697267.6772082030907596983@indico.slac.stanford.edu> Please note that the event "Weekly Meeting - data processing focus!" will start on 17 Dec 2025, 09:00 (America/Los_Angeles). You can access the full event here: https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10214&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=YsVHJ5XG4zjHQN-k0uZ2LIURTvNKTdwslwcIi9eIrEaUg0mwUH2REknfSp6l9ccj&s=wauxHGgX5Tehst1_Y2efGSpt5gMsOCGB7EfR-lVB9cU&e= Description ----------- Zoom link:?https://urldefense.proofpoint.com/v2/url?u=https-3A__stanford.zoom.us_j_93402131889-3Fpwd-3DaPDTt6wDZJcMcQQ3AGnAEagIhJEDVF.1&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=YsVHJ5XG4zjHQN-k0uZ2LIURTvNKTdwslwcIi9eIrEaUg0mwUH2REknfSp6l9ccj&s=mx-aw7w_g0wXTNorLi83M0kpdR2Jo98F2bR-cT4t7N0&e= Note ---- Let's use the meeting time to discuss preparations for processing the 2021 dataset. Agenda for Weekly Meeting - data processing focus! -------------------------------------------------- - [09:00 - 09:05] News / EC Report - [09:05 - 09:10] PPC Report - [09:10 - 10:05] Group Summaries: Group Updates - [09:10 - 09:15] Analysis Update - [09:15 - 09:20] Reconstruction Update - [09:20 - 09:25] Monte Carlo Update - [09:25 - 09:30] Detector Update - [09:30 - 10:00] Data processing discussion - [09:55 - 10:00] upcoming meetings / AOB -- Indico :: Email Notifier https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.slac.stanford.edu_e_10214&d=DwIDaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=ErLkbYGWd8oSAZ70ywHxhA&m=YsVHJ5XG4zjHQN-k0uZ2LIURTvNKTdwslwcIi9eIrEaUg0mwUH2REknfSp6l9ccj&s=wauxHGgX5Tehst1_Y2efGSpt5gMsOCGB7EfR-lVB9cU&e= -------------- next part -------------- A non-text attachment was scrubbed... Name: event.ics Type: text/calendar Size: 538 bytes Desc: not available URL: From jlab-scicomp-briefs at jlab.org Wed Dec 17 17:30:00 2025 From: jlab-scicomp-briefs at jlab.org (Wesley Moore via Jlab-scicomp-briefs) Date: Wed, 17 Dec 2025 22:30:00 +0000 Subject: [Hps] [Jlab-scicomp-briefs] December 17 Compute Cluster Power Outage In-Reply-To: References: Message-ID: Dear Users, Services have been partially restored, and recovery efforts are ongoing. Current status / in-progress work: * qcd16p and qcd18p will remain down while support services are restored. * ifarm will be patched and rebooted at 10:00 PM. * Farm node recovery is ongoing and will continue as systems come back online. We will provide additional updates as progress continues and services are fully restored. Thank you for your patience. Best regards, Wesley, on behalf of the Scientific Computing Operations Team ________________________________ From: Wesley Moore Sent: Monday, December 15, 2025 1:26 PM To: Wesley Moore via Jlab-scicomp-briefs ; lqcd-users at jlab.org ; epsci at jlab.org ; Wesley Moore Subject: Re: December 17 Compute Cluster Power Outage Dear Users, This is a gentle reminder of the planned compute cluster outage on December 17, beginning at 6:00 AM, to support contractor work installing new PDUs. Please see previous email for details. Please plan accordingly and ensure any critical work is completed ahead of the outage. As always, we will prioritize restoring core production services as quickly as possible once power is restored. Thank you for your understanding. Best regards, Wesley, on behalf of the Scientific Computing Operations Team ________________________________ From: Jlab-scicomp-briefs on behalf of Wesley Moore via Jlab-scicomp-briefs Sent: Friday, December 12, 2025 9:53 AM To: Wesley Moore via Jlab-scicomp-briefs ; lqcd-users at jlab.org ; epsci at jlab.org Subject: [Jlab-scicomp-briefs] December 17 Compute Cluster Power Outage Dear Users, On December 17th, there will be a power outage affecting all compute clusters to allow a contractor to install new PDUs. The outage window will begin at 6:00 AM. Due to the nature of the work, we can only provide a general timeframe, and we expect the interruption to last most of the day. Full recovery time is still to be determined. As always, we will prioritize restoring core production services first so jobs can resume as quickly as possible. Affected services include, but are not limited to: * General compute cluster availability (Farm & QCD) * JupyterHub * Tape library Systems expected to be unaffected by the outage: * Interactives (ifarm &. qcdi) * Lustre filesystems * Work disk filesystems * code.jlab.org and associated runners To reduce the impact on users, we will keep Maintenance Day (December 16th) very light to help ensure no user-facing disruptions to running jobs ahead of the outage. We will provide updates as more information becomes available. Best regards, Wesley, on behalf of the Scientific Computing Operations Team -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -- This is an announcement-only list for Jefferson Lab Scientific Computing Updates . Subscription and List Archive: https://mailman.jlab.org/mailman/listinfo/jlab-scicomp-briefs For help: https://jlab.servicenowservices.com/scicomp From jlab-scicomp-briefs at jlab.org Thu Dec 18 17:06:13 2025 From: jlab-scicomp-briefs at jlab.org (Wesley Moore via Jlab-scicomp-briefs) Date: Thu, 18 Dec 2025 22:06:13 +0000 Subject: [Hps] [Jlab-scicomp-briefs] December 17 Compute Cluster Power Outage In-Reply-To: References: Message-ID: Dear Users, There was an unexpected power outage in the data center earlier today. Systems have now been restored and are operating in a stable state. We will continue to monitor the situation overnight to ensure continued stability and will provide updates if anything changes. Thank you for your patience. Best regards, Wesley, on behalf of the Scientific Computing Operations Team ________________________________ From: lqcd-users on behalf of Wesley Moore via lqcd-users Sent: Wednesday, December 17, 2025 5:30 PM To: Wesley Moore via Jlab-scicomp-briefs ; lqcd-users at jlab.org ; epsci at jlab.org Subject: Re: [lqcd-users] December 17 Compute Cluster Power Outage Dear Users, Services have been partially restored, and recovery efforts are ongoing. Current status / in-progress work: * qcd16p and qcd18p will remain down while support services are restored. * ifarm will be patched and rebooted at 10:00 PM. * Farm node recovery is ongoing and will continue as systems come back online. We will provide additional updates as progress continues and services are fully restored. Thank you for your patience. Best regards, Wesley, on behalf of the Scientific Computing Operations Team ________________________________ From: Wesley Moore Sent: Monday, December 15, 2025 1:26 PM To: Wesley Moore via Jlab-scicomp-briefs ; lqcd-users at jlab.org ; epsci at jlab.org ; Wesley Moore Subject: Re: December 17 Compute Cluster Power Outage Dear Users, This is a gentle reminder of the planned compute cluster outage on December 17, beginning at 6:00 AM, to support contractor work installing new PDUs. Please see previous email for details. Please plan accordingly and ensure any critical work is completed ahead of the outage. As always, we will prioritize restoring core production services as quickly as possible once power is restored. Thank you for your understanding. Best regards, Wesley, on behalf of the Scientific Computing Operations Team ________________________________ From: Jlab-scicomp-briefs on behalf of Wesley Moore via Jlab-scicomp-briefs Sent: Friday, December 12, 2025 9:53 AM To: Wesley Moore via Jlab-scicomp-briefs ; lqcd-users at jlab.org ; epsci at jlab.org Subject: [Jlab-scicomp-briefs] December 17 Compute Cluster Power Outage Dear Users, On December 17th, there will be a power outage affecting all compute clusters to allow a contractor to install new PDUs. The outage window will begin at 6:00 AM. Due to the nature of the work, we can only provide a general timeframe, and we expect the interruption to last most of the day. Full recovery time is still to be determined. As always, we will prioritize restoring core production services first so jobs can resume as quickly as possible. Affected services include, but are not limited to: * General compute cluster availability (Farm & QCD) * JupyterHub * Tape library Systems expected to be unaffected by the outage: * Interactives (ifarm &. qcdi) * Lustre filesystems * Work disk filesystems * code.jlab.org and associated runners To reduce the impact on users, we will keep Maintenance Day (December 16th) very light to help ensure no user-facing disruptions to running jobs ahead of the outage. We will provide updates as more information becomes available. Best regards, Wesley, on behalf of the Scientific Computing Operations Team -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -- This is an announcement-only list for Jefferson Lab Scientific Computing Updates . Subscription and List Archive: https://mailman.jlab.org/mailman/listinfo/jlab-scicomp-briefs For help: https://jlab.servicenowservices.com/scicomp From stepanya at jlab.org Fri Dec 19 14:11:41 2025 From: stepanya at jlab.org (Stepan Stepanyan) Date: Fri, 19 Dec 2025 19:11:41 +0000 Subject: [Hps] Fwd: [Cuga] Updated DOE Requirements for Jefferson Lab Access by Certain Foreign National - A message from Tuane Young References: Message-ID: <42726342-74A1-475F-ACE7-003CCADFDBD5@jlab.org> In case you have not seen this. Stepan Begin forwarded message: From: Sharice Smith via Cuga Subject: [Cuga] Updated DOE Requirements for Jefferson Lab Access by Certain Foreign National - A message from Tuane Young Date: December 19, 2025 at 10:14:52?AM EST To: "cuga at jlab.org" Reply-To: Sharice Smith December 19, 2025 SUBJECT: Updated DOE Requirements for Jefferson Lab Access by Certain Foreign Nationals CONTACT: Tuane Young, tyoung at jlab.org Dear Colleagues, We are writing to inform you of important changes to the Department of Energy?s Unclassified Foreign National Access (UFNA) Program. The recent changes affect the vetting process that Jefferson Lab needs to follow for certain foreign nationals. This can impact planning for site access, collaborations and events involving international participants. For foreign nationals who are not U.S. citizens, but are from a country listed as a State Sponsors of Terrorism or Countries of Risk (Cuba, North Korea, Iran, Syria, Belarus, Russia and China, including Hong Kong and Macau), the vetting period has been extended from seven days to 45 days. The vetting, which includes an indices check, must now be completed before arrival on-site, with no exceptions. An indices checks are background checks done by U.S. government agencies to find out if they have any information about a specific foreign person. Once completed, the indices check will remain valid for two years. Be advised that individuals from these countries and not U.S. citizens may no longer serve as hosts for other foreign nationals. These updated requirements in general apply to other sensitive country nationals with the exception of those accessing Jefferson Lab under our User Agreement as an Office of Science User Facility. For these individuals, the indices check may be completed after access is granted. Additionally, the extended vetting period applies to any foreign national whose work involves sensitive subjects. The seven-day vetting period for all other foreign nationals remains unchanged. These changes reflect DOE?s commitment to strengthening national security and enhancing vetting protocols for access to DOE facilities. Jefferson Lab remains fully committed to compliance while supporting our user community through this transition. If you are expecting visitors: ? Initiate the access request process well in advance (minimum 45 days). ? Review your hosting responsibilities to ensure individuals from so-called State Sponsors of Terrorism and Countries of Risk are not assigned hosting duties. ? On host and event submission forms, identify whether sensitive information will be shared with the guest. ? Stay in touch with the Access Team at fso at jlab.org if you have any questions regarding access requests. For more information on the current access and registration policies, and your responsibilities as a host, visit the Site Security website. _______________________________________________ Cuga mailing list Cuga at jlab.org https://mailman.jlab.org/mailman/listinfo/cuga -------------- next part -------------- An HTML attachment was scrubbed... URL: