<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle23
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">After killing this process and restarting it, it corrupted the local cache and the “secrets” file that gets created by sssd. This means that no one could login and all Kerberos commands would fail.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In order to fix this, I had to do the following:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">rm -f /var/lib/sss/db/*cache*<o:p></o:p></p>
<p class="MsoNormal">yum reinstall sssd-common<o:p></o:p></p>
<p class="MsoNormal">systemctl restart sssd<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">reboot<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">While is seems like a logical process that I just did, these commands are after a bunch of frogging around and trying various things. I left these steps out an simply give the commands here to fix it in the future.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Anthony<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> IT Service Desk <jlab@servicenowservices.com> <br>
<b>Sent:</b> Thursday, September 21, 2023 4:07 PM<br>
<b>To:</b> Erik Werlau <werlau@jlab.org>; Brian Bevins <bevins@jlab.org>; Theo McGuckin <tsm@jlab.org>; Brad Cumbia <cumbia@jlab.org>; Anthony Cuffe <cuffe@jlab.org>; Ryan Slominski <ryans@jlab.org>; Jessica Zamzow <zamzow@jlab.org>; accadm@jlab.org; Adam Carpenter
<adamc@jlab.org>; Theo Larrieu <theo@jlab.org><br>
<b>Subject:</b> Incident INC0109183 has been assigned - Some process called Kerberos Cache Manager, sssd_kcm, is hogging a lot of CPU on my machine devl02.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p><b><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Subject:</span></b><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black"> Some process called Kerberos Cache Manager, sssd_kcm, is hogging a lot of CPU
on my machine devl02.</span><o:p></o:p></p>
<p><b><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Incident:</span></b><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black"> INC0109183</span><o:p></o:p></p>
</div>
<div>
<p><strong><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif;color:black">An incident has been assigned to Theo McGuckin.</span></strong><o:p></o:p></p>
</div>
<div>
<p><strong><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif;color:black">Additional Details:</span></strong><o:p></o:p></p>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Requestor: Gary Croke (<a href="mailto:gcroke@jlab.org">gcroke@jlab.org</a>) ext: 5097<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Updated By: tsm<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Opened: 09-21-2023 03:21:03 PM
<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Category: Accelerator Computing Environment (ACE)<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Subcategory: Linux Support<o:p></o:p></span></p>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Assigned To: Theo McGuckin<o:p></o:p></span></p>
<div>
<p><strong><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif;color:black">Comments:</span></strong><o:p></o:p></p>
<div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">
<hr size="2" width="100%" align="center">
</span></div>
</div>
<div>
<p class="MsoNormal"><strong><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">09-21-2023 04:06:44 PM EDT - Theo McGuckin</span></strong><sup><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black"> Additional
comments</span></sup><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Stopped and restarted sssd-kcm.socket process as root.<br>
Let me know if the problem returns</span><o:p></o:p></p>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">
<hr size="2" width="100%" align="center">
</span></div>
</div>
<div>
<p class="MsoNormal"><strong><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">09-21-2023 03:21:03 PM EDT - Gary Croke</span></strong><sup><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black"> Additional
comments</span></sup><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">Some process called Kerberos Cache Manager, sssd_kcm, is hogging a lot of CPU on my machine devl02. Can't seem to kill it, restarted, still there.</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black">You can view all the details of the incident by following the link below:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Helvetica",sans-serif"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__jlab.servicenowservices.com_nav-5Fto.do-3Furi-3Dincident.do-3Fsys-5Fid-3De6c3126f1b15b510a552ed3ce54bcbb0&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=39aRN8DBBjQIGUY49byo_w&m=JxWDGsrYLf_LiPPJTOYc2QrrXbWskoInxZYhuKYdbDNk3l4qtAHp650MqAR_9PzF&s=DVX1ttz52oNP81AUGs1MnqdVhoNX3eQZ-DbPYEfc4AQ&e="><span style="font-size:12.0pt;color:white;border:solid #0368D4 1.0pt;padding:4.0pt;background:#278EFC;text-decoration:none">Take
me to the Incident</span></a></span><o:p></o:p></p>
<p><strong><span style="font-size:13.5pt;font-family:"Helvetica",sans-serif;color:#999999">Original Request:</span></strong><o:p></o:p></p>
<p><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:#999999">Some process called Kerberos Cache Manager, sssd_kcm, is hogging a lot of CPU on my machine devl02. Can't seem to kill it, restarted, still there.<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Ref:MSG0936696_R3ScwvzPR9rKrjXQujNo<o:p></o:p></p>
</div>
</div>
</body>
</html>