[Moller_daq] [EXTERNAL] MOLLER raw data file size

[Paul King]

Michael,

I think we would probably just keep the final running sum value, and if there is a mismatch of the sum of the subblocks and the full window we would mark that channel as bad for the entire event.

Based on past performance of the VQWK modules, I am anticipating very few events will get lost due to a sum-of-blocks and full-window-sum mismatch.  If we see more errors occurring, we could change the data filtering to keep the running sums for each block.

Paul.

________________________________________
From: Moller_daq <moller_daq-bounces at jlab.org> on behalf of mgericke <mgericke at jlab.org>
Sent: Friday, September 5, 2025 12:41 PM
To: moller_daq at jlab.org
Subject: Re: [Moller_daq] [EXTERNAL] MOLLER raw data file size

Hi Paul,

In that case, you need the first 3 and a subset of 4 (the
minimum/maximum values, plus maybe trigger information), plus the 3
final window block running sum values.
Could there be an advantage to keep the running sums, rather than just
the final total value ? If there is data loss, it could tell your in
which block it occurred. But maybe
we don't care and just throw away the entire sequence at that point.

Thanks,

Michael


On 2025-09-05 11:12 a.m., Paul King wrote:
> Dear Michael,
>
> We would store the raw data from the ADC boards in binary form.  However, I had thought to do a data filtering step either in the CODA readout list on the CPU, since we  will not need the top longword of the 64-bits on the sum of all samples, for example, and would not need to keep each instance of the running sum.
>
> Paul.
>
> ________________________________________
> From: Michael Gericke <mgericke at physics.umanitoba.ca>
> Sent: Friday, September 5, 2025 12:01 PM
> To: Paul King
> Cc: moller_analysis at jlab.org; moller_daq at jlab.org
> Subject: [EXTERNAL] MOLLER raw data file size
>
> Hi Paul,
>
> Is the plan for the MOLLER data to write the raw data (as it comes from
> the ADC boards) to file in binary form,
> or only the "condensed" root files?
>
> Right now, including the total helicity window data (which I added to
> the firmware last night) as well as the block
> data, the raw data files size for a 1 minute run is 445440000 bytes.
> This is for a reversal rate of 2 kHz and 4 blocks
> per window. Each block of data within a given helicity window is a
> separate packet, which is written to file.
>
> Each block includes 16x7 (16 channels times 7 data items) 64 bit data
> registers, plus a 4x64 bit packet header, including
> block number, time stamp, and total sample count (over all 16 channels).
>
> Within each block we have the following data items:
>
> 1) 64 bit sum of all samples per block for each of 16 channels
> 2) 64 bit sum-of-squares per block for each of 16 channels
> 3) 64 bit number of samples per block for each of 16 channels
> 4) 64 bit:  20 bit maximum + 20 bit minimum  sample values + 2 bit
> trigger state (both input trigger Lemo signals) + 22 bits empty/free per
> block for each of 16 channels
> 5) 64 bit running sum of all samples over the sequence of blocks in a
> helicity window for each of 16 channels
> 6) 64 bit running sum-of-squares over the sequence of blocks in a
> helicity window for each of 16 channels
> 7) 64 bit running sample number over the sequence of blocks in a
> helicity window for each of 16 channels
>
> The running sums over all blocks per helicity window are cleared with
> each new trigger, but are kept for all
> block packets rather than only sending the final values with the last
> block packet. I think doing the latter would
> require a more complicated setup for the firmware with some sort of
> interleaved packet transmission (since they
> would have different sizes).
>
> I tested the transmission in longer runs last night and with regard to
> that there doesn't seem to be a problem (e.g. dead
> time). From a data storage point of view this is a little wasteful, but
> we could throw away the intermediate running
> values before writing it to file.
>
> Right now, I am just dumping every packet as I receive it to a binary file.
>
> Of course, if you change the number of blocks per window, the file size
> changes. Right now, the maximum number
> of blocks that can be specified per helicity window is 16 (based on the
> 8 bit register size). So if that were to be chosen
> for some reason, it would increase the file size by a factor of 4
> relative to size I stated at the beginning.
>
> Thanks,
>
> Michael
>
> _______________________________________________
> Moller_daq mailing list
> Moller_daq at jlab.org
> https://mailman.jlab.org/mailman/listinfo/moller_daq
_______________________________________________
Moller_daq mailing list
Moller_daq at jlab.org
https://mailman.jlab.org/mailman/listinfo/moller_daq