[Halld-offline] subtle bug in ROOT

[Richard Jones]

correction: "That, plus the fact that TH1F::Fill() is a couple orders of
magnitude slower than TH1F::Fill()" should be

That, plus the fact that TH1F::Fill() is a couple orders of magnitude
slower than TH1D::Fill().

On Sat, Feb 25, 2017 at 8:14 AM, Richard Jones <richard.t.jones at uconn.edu>
wrote:

> Dear colleagues,
>
> Maybe this is common knowledge to people more versed in ROOT than I am,
> but this was a surprise to me, that took me quite some effort to discover
> why my TTree analysis was producing nonsense.
>
> >>  TH1F histograms silently truncate their bin contents at 16,777,216
> (2^28) <<
>
> unless you fill with non-unity weights, in which case they truncate at
> OTHER SMALL VALUES. To be specific, if you create a TH1F histogram h1 and
> then do h1.Fill(0.) repeatedly then this bin will silently stop
> incrementing when the bin content reaches 1.677216e+7. If you fill it with
> a non-unity weight then it will gradually lose precision as the number of
> Fill calls on that bin exceeds some threshold like a few M, and then
> silently stop incrementing altogether when the bin content reaches some
> limit. For w=100 I found this limit to be 2.147e+9. This was unexpected
> because the letter F in TH1F implies "float" which has a max value about
> 3.4e+38. What use is a histogram object that silently discards entries as
> soon as the count reaches some small value that we expect to commonly hit
> in high-statistics analysis? They must be doing some kind of
> range-truncating-compression in the storage of TH1F bin contents.
> Personally, I would rather get the right answer, even if it means using
> more memory, but that's just me.
>
> A workaround would be never to use TH1F, always TH1D. I have not been able
> to discover a similar silent truncation in TH1D. That, plus the fact that
> TH1F::Fill() is a couple orders of magnitude slower than TH1F::Fill().
> Apparently it takes a lot of cpu time to generate bugs of this kind?
>
> Meanwhile, beware. This is especially insidious because the command
> tree.Draw("px") in your interactive ROOT session implicitly creates and
> fills a TH1F, not a TH1D, even if px is declared double in your tree. In my
> present analysis, my tree has 200M rows, but in principle that will bite
> you even if you have only 20M rows in your tree.
>
> -Richard Jones
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20170225/860b0e4b/attachment-0002.html>