<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body>
The crash is unrelated to the endianness. It just led me to uncover
that we're actually byte-swapping.<br>
<br>
Byte-swapping used to be a major bottleneck back in the early 2000s.
Nowadays, not so much, but it still costs you. <br>
<br>
I'm not so much worried about the CPU overhead actually but rather
about EVIO files having one and only one endianness. What Dave
Abbott writes about "not touching user buffers" is a bit concerning.
If different front-ends deliver user buffers in different
endianness, EVIO would not be able to handle this correctly. As the
logbook entry shows, bank data are either always or never
byte-swapped. There is no facility to detect the endianness of
certain banks. Also, I find Dave's explanation rather confusing. I
worry that this could become a mess.<br>
<br>
Ole<br>
<br>
<div class="moz-cite-prefix">On 3.10.21 at 13:31, Andrew Puckett
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:BLAPR09MB6881699AF2A6B57A4C5628C6C3AD9@BLAPR09MB6881.namprd09.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Interesting. So perhaps I’m being naďve
here, but other than the byte-swapping inefficiency Ole
pointed out in processing the raw data on the compute farm
nodes, is there an actual problem here? Do we need to
check/care about this in the software in writing our raw data
decoders?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The cause of Bradley’s crash while
processing GRINCH data doesn’t necessarily seem related to
this…
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Andrew <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span
style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Robert
Michaels <a class="moz-txt-link-rfc2396E" href="mailto:rom@jlab.org"><rom@jlab.org></a><br>
<b>Date: </b>Sunday, October 3, 2021 at 1:21 PM<br>
<b>To: </b>Ole Hansen <a class="moz-txt-link-rfc2396E" href="mailto:ole@jlab.org"><ole@jlab.org></a>, Andrew
Puckett <a class="moz-txt-link-rfc2396E" href="mailto:puckett@jlab.org"><puckett@jlab.org></a>, <a class="moz-txt-link-abbreviated" href="mailto:sbs_daq@jlab.org">sbs_daq@jlab.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:sbs_daq@jlab.org"><sbs_daq@jlab.org></a><br>
<b>Subject: </b>Re: [Sbs_daq] Big endian raw data?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black">I believe there are
byte-swapping routines available in the DAQ libraries
which allow to put the bytes in the right state and be
consistent. But the DAQ expert needs to make this
happen. Below is a snippet of an email from Dave Abbott
about a year ago when I was having some trouble, which I
think is relevant.. Dave is a good person to ask. Can
ask Bryan Moffit or Alexandre, too.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black">----------------------
snippet of email from Dave Abbott ------------------------<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black;background:white">The
CODA data files are written from a Java Event Builder.
JAVA is inherently Big Endian. The EVIO</span><span
style="font-size:12.0pt;color:black">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">files will be by
default in big endian.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">However, ALL Banks
of User data - created in your readout list - will NOT
be swapped. They will stay<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">whatever Endian it
was when it was written.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">Typically the ROC
will run in Linux on Intel which is Little Endian.
Therefore the Data banks you create will stay<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">little endian.
However the Bank headers will be swapped to be
compatible with the rest of the CODA file.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">An even more
confusing possibility is that you might do a DMA from
the VME bus into a CODA data Bank.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">The VME bus is Big
endian. Therefore the data from the VME bus will stay
Big endian in this bank.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">Our general rule
for CODA 3 is that for purposes of DAQ we will not touch
(or modify) the User's data in any way.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black;background:white">We
will only modify the EVIO headers to match the endianess
of whatever System writes the file.</span><span
style="font-size:12.0pt;color:black"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div id="Signature">
<div>
<div id="divtagdefaultwrapper">
<p><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
<div class="MsoNormal" style="text-align:center" align="center">
<hr width="100%" size="0" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span
style="color:black"> Sbs_daq
<a class="moz-txt-link-rfc2396E" href="mailto:sbs_daq-bounces@jlab.org"><sbs_daq-bounces@jlab.org></a> on behalf of Ole Hansen
<a class="moz-txt-link-rfc2396E" href="mailto:ole@jlab.org"><ole@jlab.org></a><br>
<b>Sent:</b> Sunday, October 3, 2021 1:06 PM<br>
<b>To:</b> Andrew Puckett <a class="moz-txt-link-rfc2396E" href="mailto:puckett@jlab.org"><puckett@jlab.org></a>;
<a class="moz-txt-link-abbreviated" href="mailto:sbs_daq@jlab.org">sbs_daq@jlab.org</a> <a class="moz-txt-link-rfc2396E" href="mailto:sbs_daq@jlab.org"><sbs_daq@jlab.org></a><br>
<b>Subject:</b> Re: [Sbs_daq] Big endian raw data?</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Maybe our
various front-ends differ in endianness, so we write
mixed-endian data?!? That would be disastrous since it is
not supported by EVIO. A file can only be one or the other—a
very binary view. (I guess EVIO was written before we became
diversity-aware ;) ).<br>
<br>
Ole<o:p></o:p></p>
<div>
<p class="MsoNormal">On 3.10.21 at 13:03, Andrew Puckett
wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="xmsonormal">Hi Ole, <o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">This is interesting. The GRINCH data
are being read out by the new VETROC modules, I don’t
know if they differ from the other modules in terms of
“endian-ness”. Maybe a DAQ expert can weigh in here?<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Andrew <o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span
style="font-size:12.0pt;color:black">From:
</span></b><span
style="font-size:12.0pt;color:black">Sbs_daq <a
href="mailto:sbs_daq-bounces@jlab.org"
moz-do-not-send="true">
<sbs_daq-bounces@jlab.org></a> on behalf of
Ole Hansen <a href="mailto:ole@jlab.org"
moz-do-not-send="true">
<ole@jlab.org></a><br>
<b>Date: </b>Sunday, October 3, 2021 at 1:00 PM<br>
<b>To: </b><a href="mailto:sbs_daq@jlab.org"
moz-do-not-send="true">sbs_daq@jlab.org</a> <a
href="mailto:sbs_daq@jlab.org"
moz-do-not-send="true">
<sbs_daq@jlab.org></a><br>
<b>Subject: </b>[Sbs_daq] Big endian raw data?</span><o:p></o:p></p>
</div>
<p class="xmsonormal" style="margin-bottom:12.0pt">Hi
guys,<br>
<br>
Bradley reported a crash of the replay (actually in
EVIO) with /adaq1/data1/sbs/grinch_72.evio.0 (see
<a href="https://logbooks.jlab.org/entry/3916105"
moz-do-not-send="true">https://logbooks.jlab.org/entry/3916105</a>).<br>
<br>
When digging into the cause of this crash, I discovered
that these raw data are written in big-endian format.
How can this be? I thought the front-ends are Intel
processors. Are we taking data with ARM chips that are
configured for big-endian mode? Is this a mistake, or is
there some plan to it?<br>
<br>
These big-endian data have to be byte-swapped when
processing them on x86, which is what all our compute
nodes run. That's a LOT of work. It leads to significant
and seemingly completely unnecessary overhead. I.e.
we're burning CPU cycles for nothing good, it seems.<br>
<br>
Please explain.<br>
<br>
Ole<o:p></o:p></p>
</div>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<br>
</body>
</html>