[Halld-offline] subtle bug in ROOT

[Richard Jones]

Hello Paul,

Thank you, that is a good suggestion. I use weighted spectra for just about
all spectra I create, so for that TH1I is not so useful. However in some of
the use cases I covered in my message, a TH1I would do just as well.

For example, simply using a tagger_weight in filling your histograms would
immediately give you automatic tagger accidentals subtraction. That is how
I do it in my analyses. The non-Poisson errors are automatically accounted
for by the TH1D. If you think that would be of general interest, I can
offer to add a method to the DBeamPhoton that returns a tagger weight. It
would be part of the calibration cycle to establish the correct bounds for
the tagger trues and accidentals for each run.

-Richard Jones

On Sat, Feb 25, 2017 at 8:33 AM, Paul Mattione <pmatt at jlab.org> wrote:

> I recommend using TH1I, since it uses half the memory of TH1D, unless of
> course you need to fill a bin more than 2 billion times (or the content
> could be that large after merging, or if you anticipate dividing
> histograms, etc.).
>
>
>  - Paul
>
> On 02/25/2017 08:23 AM, Richard Jones wrote:
>
> correction: "That, plus the fact that TH1F::Fill() is a couple orders of
> magnitude slower than TH1F::Fill()" should be
>
> That, plus the fact that TH1F::Fill() is a couple orders of magnitude
> slower than TH1D::Fill().
>
> On Sat, Feb 25, 2017 at 8:14 AM, Richard Jones <richard.t.jones at uconn.edu>
> wrote:
>
>> Dear colleagues,
>>
>> Maybe this is common knowledge to people more versed in ROOT than I am,
>> but this was a surprise to me, that took me quite some effort to discover
>> why my TTree analysis was producing nonsense.
>>
>> >>  TH1F histograms silently truncate their bin contents at 16,777,216
>> (2^28) <<
>>
>> unless you fill with non-unity weights, in which case they truncate at
>> OTHER SMALL VALUES. To be specific, if you create a TH1F histogram h1 and
>> then do h1.Fill(0.) repeatedly then this bin will silently stop
>> incrementing when the bin content reaches 1.677216e+7. If you fill it with
>> a non-unity weight then it will gradually lose precision as the number of
>> Fill calls on that bin exceeds some threshold like a few M, and then
>> silently stop incrementing altogether when the bin content reaches some
>> limit. For w=100 I found this limit to be 2.147e+9. This was unexpected
>> because the letter F in TH1F implies "float" which has a max value about
>> 3.4e+38. What use is a histogram object that silently discards entries as
>> soon as the count reaches some small value that we expect to commonly hit
>> in high-statistics analysis? They must be doing some kind of
>> range-truncating-compression in the storage of TH1F bin contents.
>> Personally, I would rather get the right answer, even if it means using
>> more memory, but that's just me.
>>
>> A workaround would be never to use TH1F, always TH1D. I have not been
>> able to discover a similar silent truncation in TH1D. That, plus the fact
>> that TH1F::Fill() is a couple orders of magnitude slower than TH1F::Fill().
>> Apparently it takes a lot of cpu time to generate bugs of this kind?
>>
>> Meanwhile, beware. This is especially insidious because the command
>> tree.Draw("px") in your interactive ROOT session implicitly creates and
>> fills a TH1F, not a TH1D, even if px is declared double in your tree. In my
>> present analysis, my tree has 200M rows, but in principle that will bite
>> you even if you have only 20M rows in your tree.
>>
>> -Richard Jones
>>
>
>
>
> _______________________________________________
> Halld-offline mailing listHalld-offline at jlab.orghttps://mailman.jlab.org/mailman/listinfo/halld-offline
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20170225/5ab2b251/attachment-0002.html>