[Sbs_daq] Big endian raw data?

[Ole Hansen]

The crash is unrelated to the endianness. It just led me to uncover that 
we're actually byte-swapping.

Byte-swapping used to be a major bottleneck back in the early 2000s. 
Nowadays, not so much, but it still costs you.

I'm not so much worried about the CPU overhead actually but rather about 
EVIO files having one and only one endianness. What Dave Abbott writes 
about "not touching user buffers" is a bit concerning. If different 
front-ends deliver user buffers in different endianness, EVIO would not 
be able to handle this correctly. As the logbook entry shows, bank data 
are either always or never byte-swapped. There is no facility to detect 
the endianness of certain banks. Also, I find Dave's explanation rather 
confusing. I worry that this could become a mess.

Ole

On 3.10.21 at 13:31, Andrew Puckett wrote:
>
> Interesting. So perhaps I’m being naïve here, but other than the 
> byte-swapping inefficiency Ole pointed out in processing the raw data 
> on the compute farm nodes, is there an actual problem here? Do we need 
> to check/care about this in the software in writing our raw data decoders?
>
> The cause of Bradley’s crash while processing GRINCH data doesn’t 
> necessarily seem related to this…
>
> Andrew
>
> *From: *Robert Michaels <rom at jlab.org>
> *Date: *Sunday, October 3, 2021 at 1:21 PM
> *To: *Ole Hansen <ole at jlab.org>, Andrew Puckett <puckett at jlab.org>, 
> sbs_daq at jlab.org <sbs_daq at jlab.org>
> *Subject: *Re: [Sbs_daq] Big endian raw data?
>
> I believe there are byte-swapping routines available in the DAQ 
> libraries which allow to put the bytes in the right state and be 
> consistent.  But the DAQ expert needs to make this happen.  Below is a 
> snippet of an email from Dave Abbott about a year ago when I was 
> having some trouble, which I think is relevant..  Dave is a good 
> person to ask.  Can ask Bryan Moffit or Alexandre, too.
>
> ---------------------- snippet of email from Dave Abbott 
> ------------------------
>
> The CODA data files are written from a Java Event Builder. JAVA is 
> inherently Big Endian. The EVIO
>
> files will be by default in big endian.
>
> However, ALL Banks of User data - created in your readout list - will 
> NOT be swapped. They will stay
>
> whatever Endian it was when it was written.
>
> Typically the ROC will run in Linux on Intel which is Little Endian. 
> Therefore the Data banks you create will stay
>
> little endian. However the Bank headers will be swapped to be 
> compatible with the rest of the CODA file.
>
> An even more confusing possibility is that you might do a DMA from the 
> VME bus into a CODA data Bank.
>
> The VME bus is Big endian. Therefore the data from the VME bus will 
> stay Big endian in this bank.
>
> Our general rule for CODA 3 is that for purposes of DAQ we will not 
> touch (or modify) the User's data in any way.
>
> We will only modify the EVIO headers to match the endianess of 
> whatever System writes the file.
>
> ------------------------------------------------------------------------
>
> *From:*Sbs_daq <sbs_daq-bounces at jlab.org> on behalf of Ole Hansen 
> <ole at jlab.org>
> *Sent:* Sunday, October 3, 2021 1:06 PM
> *To:* Andrew Puckett <puckett at jlab.org>; sbs_daq at jlab.org 
> <sbs_daq at jlab.org>
> *Subject:* Re: [Sbs_daq] Big endian raw data?
>
> Maybe our various front-ends differ in endianness, so we write 
> mixed-endian data?!? That would be disastrous since it is not 
> supported by EVIO. A file can only be one or the other—a very binary 
> view. (I guess EVIO was written before we became diversity-aware ;) ).
>
> Ole
>
> On 3.10.21 at 13:03, Andrew Puckett wrote:
>
>     Hi Ole,
>
>     This is interesting. The GRINCH data are being read out by the new
>     VETROC modules, I don’t know if they differ from the other modules
>     in terms of “endian-ness”. Maybe a DAQ expert can weigh in here?
>
>     Andrew
>
>     *From: *Sbs_daq <sbs_daq-bounces at jlab.org>
>     <mailto:sbs_daq-bounces at jlab.org> on behalf of Ole Hansen
>     <ole at jlab.org> <mailto:ole at jlab.org>
>     *Date: *Sunday, October 3, 2021 at 1:00 PM
>     *To: *sbs_daq at jlab.org <mailto:sbs_daq at jlab.org>
>     <sbs_daq at jlab.org> <mailto:sbs_daq at jlab.org>
>     *Subject: *[Sbs_daq] Big endian raw data?
>
>     Hi guys,
>
>     Bradley reported a crash of the replay (actually in EVIO) with
>     /adaq1/data1/sbs/grinch_72.evio.0 (see
>     https://logbooks.jlab.org/entry/3916105
>     <https://logbooks.jlab.org/entry/3916105>).
>
>     When digging into the cause of this crash, I discovered that these
>     raw data are written in big-endian format. How can this be? I
>     thought the front-ends are Intel processors. Are we taking data
>     with ARM chips that are configured for big-endian mode? Is this a
>     mistake, or is there some plan to it?
>
>     These big-endian data have to be byte-swapped when processing them
>     on x86, which is what all our compute nodes run. That's a LOT of
>     work. It leads to significant and seemingly completely unnecessary
>     overhead. I.e. we're burning CPU cycles for nothing good, it seems.
>
>     Please explain.
>
>     Ole
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/sbs_daq/attachments/20211003/0a56ba26/attachment.html>