<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
When EVIO opens a file, it looks for a "magic" word in a certain
place early in the file. If it isn't found, it byte-swaps that word
and checks if it matches the magic value then. If it does, it sets a
flag that this file needs byte-swapping, and all header and data
words, except "unknown" data (see the swap_data routine in
evioswap.c), are then swapped. There is no (easy) way to
enable/disable this flag outside of the library; it's hardcoded,
pre-determined behavior.<br>
<br>
I am also not sure if there are any flags in any of the headers to
indicate the endianness of the data. It would just require a single
bit. The headers are not user-accessible anyway, BTW, so it looks
like you're on your own to find out whether you need swapping, on a
per-module or per-ROC basis, I guess. Like I said, it's not very
clear. What does "VME is big endian" really mean? An Intel CPU in a
VME crate produces what endianness ... 🤔<br>
<br>
Ole<br>
<br>
<div class="moz-cite-prefix">On 3.10.21 at 14:21, Alexandre Camsonne
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAP0WL2xW8m=se4XsfT8U-x5w_ELX5=+TXKUzKNM-3F1+RNXRAg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="auto">
<div>Everything is intel besided VTP. Though Dave mentionned VME
was big endian.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Alexandre<br>
<br>
<div class="gmail_quote" dir="auto">
<div dir="ltr" class="gmail_attr">On Sun, Oct 3, 2021, 13:56
Paul King <<a href="mailto:pking@jlab.org"
moz-do-not-send="true">pking@jlab.org</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div dir="auto"
style="direction:ltr;margin:0;padding:0;font-family:sans-serif;font-size:11pt;color:black">I
can comment that when I wrote the helicity scaler
library, I found that I needed to byte swap the data
words (module data and diagnostic counters) on the
crate in order to be decoded correctly.
<br>
</div>
<div dir="auto"
style="direction:ltr;margin:0;padding:0;font-family:sans-serif;font-size:11pt;color:black">I'm
not sure if halladaq8 is an Intel or arm cpu.<br>
<br>
</div>
<div dir="auto"
style="direction:ltr;margin:0;padding:0;font-family:sans-serif;font-size:11pt;color:black">Does
Podd or evio2xml do a dynamic check of endianness and
then byteswap, or is that explicitly enabled?
<span
id="m_-188557246818857469ms-outlook-android-cursor"></span></div>
<div><br>
</div>
<div
id="m_-188557246818857469ms-outlook-mobile-signature">Sent
from my mobile device.<br>
Get <a
href="https://urldefense.proofpoint.com/v2/url?u=https-3A__aka.ms_ghei36&d=DwMFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=FxpSg1qAggzMg8YCjcvm4w&m=W_Py47zEQFMF4YLSxKlQ6YvCZkjxlCgMIaViW0IvmiA&s=HtjtqwZf9TY9s3jftpebziiNiD97Y3BAaURjA_dYHjc&e="
target="_blank" rel="noreferrer"
moz-do-not-send="true">Outlook for Android</a></div>
<hr style="display:inline-block;width:98%">
<div id="m_-188557246818857469divRplyFwdMsg" dir="ltr"><font
style="font-size:11pt" face="Calibri, sans-serif"
color="#000000"><b>From:</b> Sbs_daq <<a
href="mailto:sbs_daq-bounces@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq-bounces@jlab.org</a>>
on behalf of Andrew Puckett <<a
href="mailto:puckett@jlab.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">puckett@jlab.org</a>><br>
<b>Sent:</b> Sunday, October 3, 2021 1:31:05 PM<br>
<b>To:</b> Robert Michaels <<a
href="mailto:rom@jlab.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">rom@jlab.org</a>>;
Ole Hansen <<a href="mailto:ole@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">ole@jlab.org</a>>; <a
href="mailto:sbs_daq@jlab.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">sbs_daq@jlab.org</a>
<<a href="mailto:sbs_daq@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq@jlab.org</a>><br>
<b>Subject:</b> Re: [Sbs_daq] Big endian raw data?</font>
<div>Â </div>
</div>
<div link="blue" vlink="purple"
style="word-wrap:break-word" lang="EN-US">
<div>
<p>Interesting. So perhaps I’m being naïve here, but
other than the byte-swapping inefficiency Ole
pointed out in processing the raw data on the
compute farm nodes, is there an actual problem
here? Do we need to check/care about this in the
software in writing our raw data decoders?</p>
<p>Â </p>
<p>The cause of Bradley’s crash while processing
GRINCH data doesn’t necessarily seem related to
this…
</p>
<p>Â </p>
<p>Andrew </p>
<p>Â </p>
<div style="border:none;border-top:solid #b5c4df
1.0pt;padding:3.0pt 0in 0in 0in">
<p style="margin-bottom:12.0pt"><b><span
style="font-size:12.0pt;color:black">From:
</span></b><span
style="font-size:12.0pt;color:black">Robert
Michaels <<a href="mailto:rom@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">rom@jlab.org</a>><br>
<b>Date: </b>Sunday, October 3, 2021 at 1:21
PM<br>
<b>To: </b>Ole Hansen <<a
href="mailto:ole@jlab.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">ole@jlab.org</a>>,
Andrew Puckett <<a
href="mailto:puckett@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">puckett@jlab.org</a>>,
<a href="mailto:sbs_daq@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq@jlab.org</a>
<<a href="mailto:sbs_daq@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq@jlab.org</a>><br>
<b>Subject: </b>Re: [Sbs_daq] Big endian raw
data?</span></p>
</div>
<div>
<p><span style="font-size:12.0pt;color:black">I
believe there are byte-swapping routines
available in the DAQ libraries which allow to
put the bytes in the right state and be
consistent. But the DAQ expert needs to make
this happen. Below is a snippet of an email
from Dave Abbott about a year ago when I was
having some trouble, which I think is
relevant.. Dave is a good person to ask. Can
ask Bryan Moffit or Alexandre, too.</span></p>
</div>
<div>
<p><span style="font-size:12.0pt;color:black">Â </span></p>
</div>
<div>
<p><span style="font-size:12.0pt;color:black">----------------------Â
snippet of email from Dave Abbott
------------------------</span></p>
</div>
<div>
<p><span style="font-size:12.0pt;color:black">Â </span></p>
</div>
<div>
<p><span
style="font-size:12.0pt;color:black;background:white">The
CODA data files are written from a Java Event
Builder. JAVA is inherently Big Endian. The
EVIO</span><span
style="font-size:12.0pt;color:black">
</span></p>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">files
will be by default in big endian.</span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">Â </span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">However,
ALL Banks of User data - created in your
readout list - will NOT be swapped. They
will stay</span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">whatever
Endian it was when it was written.</span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">Â </span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">Typically
the ROC will run in Linux on Intel which is
Little Endian. Therefore the Data banks you
create will stay</span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">little
endian. However the Bank headers will be
swapped to be compatible with the rest of
the CODA file.</span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">Â </span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">An even
more confusing possibility is that you might
do a DMA from the VME bus into a CODA data
Bank.</span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">The VME
bus is Big endian. Therefore the data from
the VME bus will stay Big endian in this
bank.</span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">Â </span></p>
</div>
<div>
<p style="background:white"><span
style="font-size:12.0pt;color:black">Our
general rule for CODA 3 is that for purposes
of DAQ we will not touch (or modify) the
User's data in any way.</span></p>
</div>
<p><span
style="font-size:12.0pt;color:black;background:white">We
will only modify the EVIO headers to match the
endianess of whatever System writes the file.</span><span
style="font-size:12.0pt;color:black"></span></p>
</div>
<div>
<div>
<p><span style="font-size:12.0pt;color:black">Â </span></p>
</div>
<div id="m_-188557246818857469x_Signature">
<div>
<div
id="m_-188557246818857469x_divtagdefaultwrapper">
<p><span
style="font-size:12.0pt;color:black">Â </span></p>
</div>
</div>
</div>
</div>
<div style="text-align:center" align="center">
<hr width="100%" size="0" align="center">
</div>
<div id="m_-188557246818857469x_divRplyFwdMsg">
<p><b><span style="color:black">From:</span></b><span
style="color:black"> Sbs_daq <<a
href="mailto:sbs_daq-bounces@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq-bounces@jlab.org</a>>
on behalf of Ole Hansen <<a
href="mailto:ole@jlab.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">ole@jlab.org</a>><br>
<b>Sent:</b> Sunday, October 3, 2021 1:06 PM<br>
<b>To:</b> Andrew Puckett <<a
href="mailto:puckett@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">puckett@jlab.org</a>>;
<a href="mailto:sbs_daq@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq@jlab.org</a>
<<a href="mailto:sbs_daq@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq@jlab.org</a>><br>
<b>Subject:</b> Re: [Sbs_daq] Big endian raw
data?</span> </p>
<div>
<p>Â </p>
</div>
</div>
<div>
<p style="margin-bottom:12.0pt">Maybe our various
front-ends differ in endianness, so we write
mixed-endian data?!? That would be disastrous
since it is not supported by EVIO. A file can
only be one or the other—a very binary view. (I
guess EVIO was written before we became
diversity-aware ;) ).<br>
<br>
Ole</p>
<div>
<p>On 3.10.21 at 13:03, Andrew Puckett wrote:</p>
</div>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p>Hi Ole, </p>
<p>Â </p>
<p>This is interesting. The GRINCH data are
being read out by the new VETROC modules, I
don’t know if they differ from the other
modules in terms of “endian-ness”. Maybe a
DAQ expert can weigh in here?</p>
<p>Â </p>
<p>Andrew </p>
<p>Â </p>
<div style="border:none;border-top:solid
#b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p style="margin-bottom:12.0pt"><b><span
style="font-size:12.0pt;color:black">From:
</span></b><span
style="font-size:12.0pt;color:black">Sbs_daq
<a
href="mailto:sbs_daq-bounces@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">
<sbs_daq-bounces@jlab.org></a>
on behalf of Ole Hansen <a
href="mailto:ole@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">
<ole@jlab.org></a><br>
<b>Date: </b>Sunday, October 3, 2021 at
1:00 PM<br>
<b>To: </b><a
href="mailto:sbs_daq@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">sbs_daq@jlab.org</a>
<a href="mailto:sbs_daq@jlab.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">
<sbs_daq@jlab.org></a><br>
<b>Subject: </b>[Sbs_daq] Big endian
raw data?</span></p>
</div>
<p style="margin-bottom:12.0pt">Hi guys,<br>
<br>
Bradley reported a crash of the replay
(actually in EVIO) with
/adaq1/data1/sbs/grinch_72.evio.0 (see
<a
href="https://logbooks.jlab.org/entry/3916105"
target="_blank" rel="noreferrer"
moz-do-not-send="true">https://logbooks.jlab.org/entry/3916105</a>).<br>
<br>
When digging into the cause of this crash, I
discovered that these raw data are written
in big-endian format. How can this be? I
thought the front-ends are Intel processors.
Are we taking data with ARM chips that are
configured for big-endian mode? Is this a
mistake, or is there some plan to it?<br>
<br>
These big-endian data have to be
byte-swapped when processing them on x86,
which is what all our compute nodes run.
That's a LOT of work. It leads to
significant and seemingly completely
unnecessary overhead. I.e. we're burning CPU
cycles for nothing good, it seems.<br>
<br>
Please explain.<br>
<br>
Ole</p>
</div>
</blockquote>
<p>Â </p>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Sbs_daq mailing list<br>
<a href="mailto:Sbs_daq@jlab.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">Sbs_daq@jlab.org</a><br>
<a
href="https://mailman.jlab.org/mailman/listinfo/sbs_daq"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://mailman.jlab.org/mailman/listinfo/sbs_daq</a><br>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>