<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<style type="text/css">
<!--
html{color:#555555;}body{line-height:1.5;font-family:'Trebuchet MS','Helvetica Neue',Arial,Helvetica,sans-serif;font-size:87.5%;}h1{font-size:1.6em;}h2.field-label{display:inline-block;font-size:1em;padding-right:5px;min-width:10em;margin:0.3em;}.problem_report{line-height:1.5;max-width:60em;}fieldset.problem_report.resolved
legend{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAy0lEQVQ4jWP8//8/AyWAiZACd3f3/xYWFrht+f//P1a84t3e/0obff4rbfT5D1GGXR0LuoEr3+/7X3W4n2gvwA0gVSOKAcqbfPGGpImJCU45JgYGBoa7fpsZ22wLSbadgYGBgRE9GrF55Vf2BYbHjx8zYjWB0ljAcAGGExkZ/0MtwuoCggmJEBh4AzBS4pMnT/7fuXOH4dKlSwwnT56EiwcGBv43MDBgMDExYdDX12eQkZGBhAlyiC5YsOA/AwMDUXjLli3/iYoFQgAA+pSxZrXofD0AAAAASUVORK5CYII=);background-repeat:no-repeat;padding-left:18px;}fieldset.problem_report.needs_attention
legend{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABmJLR0QA/wD/AP+gvaeTAAAA9ElEQVR42sWTvUoDQRSFv9wMKWxSBVmzdhZJIwTWv9pyLWxTpbE1kBeJPoLxBazzBgGFKNqlHXAhsITUw1y7sMpmjER0YJrDPWcO556BPzhjYPJjlhfT82LUpxcK6Lo5U0JMgcu56tXy7BQajeBDpkAcAEdz1W6uyrLdYieK2DMmKCArcqczpH/ddc0msy+OkyQJC4h3N0ynx+Q5ALtUNs5q5U+8e+R+VPFi4kjk3NZqd++qUK+TZVnYwSfAOyvejeLXt/2qVG/dYoG1dqseBNco27bs/wXKWhIDB8AhcFLAH4Bn4Al4AUqT7RVC++6mv/JVPwDi3VGzomYvyAAAAABJRU5ErkJggg==);background-repeat:no-repeat;padding-left:18px;}.problem_report div.field-items{display:inline-block;}div.date-vitals p{font-size:87.5%;}a{text-decoration:none;}.Readme a:link,.Readme a:visited,.Readme
a:active{color:red;}
-->
</style>
</head>
<body id="mimemail-body" class="elog-logentry-notify">
<div id="center">
<div id="main">
<style>
<!--/*--><![CDATA[/* ><!--*/
div.field-vitals{
margin: 0.5em 0;
}
div.field-vitals .field-type-taxonomy-term-reference {
margin: 0.1em 0;
}
article.comment {
padding-left: 10px;
}
article.comment.odd {
background-color: #EEEEEE;
}
article.comment.even {
background-color: #DDDDDD;
}
div.node-content.logentry table{
width: auto;
border-collapse: collapse;
border-spacing: 0;
border-width: 1px;
}
div.node-content.logentry th{
border: inherit;
}
div.node-content.logentry blockquote{
background-color: #FFFFFF;
}
div.node-content.logentry caption{
font-size: 1em;
font-weight: normal;
}
table.field-vitals{
margin-top: 1em;
margin-bottom: 1em;
font-size: 87.5%;
}
table.field-vitals th{
vertical-align: middle;
text-align: left;
width: 15%;
padding: 0.1em;
}
table.field-vitals td{
vertical-align: middle;
text-align: left;
width: auto;
padding: 0.1em;
}
table.field-vitals td li {
margin-left: 0;
list-style-type: none;
list-style-image: none;
}
table.downtime {
width: 30em;
margin-bottom: 1em;
border: 1px black dotted;
}
table.downtime th {
text-align: center;
}
table.downtime td {
text-align: center;
}
tr.caption th {
border-bottom: none;
}
table.downtime tfoot{
background-color:#EEEEEE;
}
div.field-name-body{
margin: 1em 0;
font-size: 110%;
}
div.date-vitals p{
margin: .1em 0;
}
article div.ctools-collapsible-container{
margin-left: -5px;
clear: both;
}
#comment-form{
margin-left: 5px;
border: graytext outset medium;
-moz-border-radius: 15px;
border-radius: 15px;
padding: 1em;
}
div.comments-form-box {
margin-top: 2em;
margin-bottom: 5em;
}
h3.comment-title {
/* display: none; */
}
p.author-datetime{
font-weight: bold;
}
/*--><!]]>*/
</style><article id="node-547308" class="node node-logentry article ia-n clearfix" role="article"><header class="node-header"><h1 class="node-title">
<a href="https://logbooks.jlab.org/entry/3474459" rel="bookmark">PXI Downtime</a>
</h1>
</header><div class="date-vitals">
<p class="author-datetime">
Lognumber <a href="https://logbooks.jlab.org/entry/3474459" class="lognumber" data-lognumber="3474459">3474459</a>. Submitted by <a href="https://logbooks.jlab.org/user/beng">beng</a> on <time datetime="2017-06-02T12:31:10-0400" pubdate="pubdate">Fri, 06/02/2017 - 12:31</time>. </p>
<table class="field-vitals"><tr><th>Logbooks: </th><td><a href="https://logbooks.jlab.org/book/hdsolenoid">HDSOLENOID</a></td></tr><tr><th>Entry Makers: </th><td>tlemon</td></tr><tr><th>References: </th><td><a href="https://logbooks.jlab.org/entry/3469077">3469077 - Follow-up Re: Solenoid PLC ethernet comm failure "Duplicate IP addresses"</a><br /><a href="https://logbooks.jlab.org/entry/3406619">3406619 - Follow-up Re: Hall D PXI Upgrades</a></td></tr></table></div>
<div class="logentry node-content">
<p>Hovanes reported that the PXI was down, initial symptoms were that it wasn't even responding to pings.</p>
<p>I went to the hall with Tyler to investigate and initially we found it had nothing on the console indicating an error or why it wasn't running, but upon power cycling it we go the same console message as this comment: <a href="https://logbooks.jlab.org/comment/10881">https://logbooks.jlab.org/comment/10881</a></p>
<p>We changed the controller to use a static IP for NIC1, disabled NIC2, and also upgraded the base software from LabVIEW 2015 to 2016.</p>
<p>Even after doing so we managed to get the same message and/or the controller would reboot itself a few times before going into safe mode (which was the same behavior we saw with 2015).</p>
<p>Thinking this might have something to do with the real-time executable we ran the VI manually from the project, this worked (once). Upon rebooting to test a real-time executable that had debugging disabled (which had the same behavior) running the VI manually resulted in the controller rebooting itself as well.</p>
<p>We redeployed the real-time executable that has debugging enabled and connected to the debug session, initially the VI was stuck initializing, but stopping and starting the VI allowed it to run successfully.</p>
<p>End result: we just rebooted it enough times for it to boot into a working state (without really changing anything from when we started).</p>
<p>As a side note: we rebooted the IOC for the PXI many times (as should be done if the PXI is restarted) and several times it got stuck (all indicators showing disconnected) after a soft reboot which required a hard reboot.</p>
<p>Possible things we could try:<br />
- Add some indication if the VI is running or not (e.g. blink an LED)<br />
- Look into running a serial line to the controller to see if it produces more useful debugging information than on the monitor<br />
- The controller has some diagnostics for the hard drive and memory, but since it said they could take hours to complete we didn't run them</p>
</div>
<div class="attachment-box">
</div>
</article> </div>
</div>
</body>
</html>