[Halld-offline] subtle bug in ROOT

[Richard Jones]

Dominik,

Such limits exist of course, but this is NOT what I am reporting. The limit
of IEEE 32-bit floats is 3.4e+38. TH1F is not using ordinary float-type
values. It must be converting the floats to something less than 32 bits
(maybe 16 bits?) and trying to save space that way.

-Richard J

On Sat, Feb 25, 2017 at 9:15 AM, Dominik Werthmueller <werthm at jlab.org>
wrote:

> Dear Richard
>
> This issue is not really a bug in ROOT but related to general floating
> point arithmetics on computers. Basically the difference between two
> representable numbers increases as their values increase and it will exceed
> 1 at the limit you mentioned. Hence adding 1 does not change the number
> anymore.
>
> https://root-forum.cern.ch/t/maximum-histogram-bin-content/5891/2
> https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
>
> Cheers,
> Dominik
>
> > Am 25.02.2017 um 13:14 schrieb Richard Jones <richard.t.jones at uconn.edu
> >:
> >
> > Dear colleagues,
> >
> > Maybe this is common knowledge to people more versed in ROOT than I am,
> but this was a surprise to me, that took me quite some effort to discover
> why my TTree analysis was producing nonsense.
> >
> > >>  TH1F histograms silently truncate their bin contents at 16,777,216
> (2^28) <<
> >
> > unless you fill with non-unity weights, in which case they truncate at
> OTHER SMALL VALUES. To be specific, if you create a TH1F histogram h1 and
> then do h1.Fill(0.) repeatedly then this bin will silently stop
> incrementing when the bin content reaches 1.677216e+7. If you fill it with
> a non-unity weight then it will gradually lose precision as the number of
> Fill calls on that bin exceeds some threshold like a few M, and then
> silently stop incrementing altogether when the bin content reaches some
> limit. For w=100 I found this limit to be 2.147e+9. This was unexpected
> because the letter F in TH1F implies "float" which has a max value about
> 3.4e+38. What use is a histogram object that silently discards entries as
> soon as the count reaches some small value that we expect to commonly hit
> in high-statistics analysis? They must be doing some kind of
> range-truncating-compression in the storage of TH1F bin contents.
> Personally, I would rather get the right answer, even if it means using
> more memory, but that's just me.
> >
> > A workaround would be never to use TH1F, always TH1D. I have not been
> able to discover a similar silent truncation in TH1D. That, plus the fact
> that TH1F::Fill() is a couple orders of magnitude slower than TH1F::Fill().
> Apparently it takes a lot of cpu time to generate bugs of this kind?
> >
> > Meanwhile, beware. This is especially insidious because the command
> tree.Draw("px") in your interactive ROOT session implicitly creates and
> fills a TH1F, not a TH1D, even if px is declared double in your tree. In my
> present analysis, my tree has 200M rows, but in principle that will bite
> you even if you have only 20M rows in your tree.
> >
> > -Richard Jones
> > _______________________________________________
> > Halld-offline mailing list
> > Halld-offline at jlab.org
> > https://mailman.jlab.org/mailman/listinfo/halld-offline
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20170225/3591f16a/attachment-0002.html>