[Halld-offline] subtle bug in ROOT

[Paul Mattione]

No, he's right.  With floats you only have 23 bits for the significand 
(1 for sign, 8 for exponent), meaning you only have 6 or 7 significant 
digits (2^23).  Then you can't increment by one any more.

  - Paul

On 02/25/2017 09:20 AM, Richard Jones wrote:
> Dominik,
>
> Such limits exist of course, but this is NOT what I am reporting. The 
> limit of IEEE 32-bit floats is 3.4e+38. TH1F is not using ordinary 
> float-type values. It must be converting the floats to something less 
> than 32 bits (maybe 16 bits?) and trying to save space that way.
>
> -Richard J
>
> On Sat, Feb 25, 2017 at 9:15 AM, Dominik Werthmueller <werthm at jlab.org 
> <mailto:werthm at jlab.org>> wrote:
>
>     Dear Richard
>
>     This issue is not really a bug in ROOT but related to general
>     floating point arithmetics on computers. Basically the difference
>     between two representable numbers increases as their values
>     increase and it will exceed 1 at the limit you mentioned. Hence
>     adding 1 does not change the number anymore.
>
>     https://root-forum.cern.ch/t/maximum-histogram-bin-content/5891/2
>     <https://root-forum.cern.ch/t/maximum-histogram-bin-content/5891/2>
>     https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
>     <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html>
>
>     Cheers,
>     Dominik
>
>     > Am 25.02.2017 um 13:14 schrieb Richard Jones
>     <richard.t.jones at uconn.edu <mailto:richard.t.jones at uconn.edu>>:
>     >
>     > Dear colleagues,
>     >
>     > Maybe this is common knowledge to people more versed in ROOT
>     than I am, but this was a surprise to me, that took me quite some
>     effort to discover why my TTree analysis was producing nonsense.
>     >
>     > >>  TH1F histograms silently truncate their bin contents at
>     16,777,216 (2^28) <<
>     >
>     > unless you fill with non-unity weights, in which case they
>     truncate at OTHER SMALL VALUES. To be specific, if you create a
>     TH1F histogram h1 and then do h1.Fill(0.) repeatedly then this bin
>     will silently stop incrementing when the bin content reaches
>     1.677216e+7. If you fill it with a non-unity weight then it will
>     gradually lose precision as the number of Fill calls on that bin
>     exceeds some threshold like a few M, and then silently stop
>     incrementing altogether when the bin content reaches some limit.
>     For w=100 I found this limit to be 2.147e+9. This was unexpected
>     because the letter F in TH1F implies "float" which has a max value
>     about 3.4e+38. What use is a histogram object that silently
>     discards entries as soon as the count reaches some small value
>     that we expect to commonly hit in high-statistics analysis? They
>     must be doing some kind of range-truncating-compression in the
>     storage of TH1F bin contents. Personally, I would rather get the
>     right answer, even if it means using more memory, but that's just me.
>     >
>     > A workaround would be never to use TH1F, always TH1D. I have not
>     been able to discover a similar silent truncation in TH1D. That,
>     plus the fact that TH1F::Fill() is a couple orders of magnitude
>     slower than TH1F::Fill(). Apparently it takes a lot of cpu time to
>     generate bugs of this kind?
>     >
>     > Meanwhile, beware. This is especially insidious because the
>     command tree.Draw("px") in your interactive ROOT session
>     implicitly creates and fills a TH1F, not a TH1D, even if px is
>     declared double in your tree. In my present analysis, my tree has
>     200M rows, but in principle that will bite you even if you have
>     only 20M rows in your tree.
>     >
>     > -Richard Jones
>     > _______________________________________________
>     > Halld-offline mailing list
>     > Halld-offline at jlab.org <mailto:Halld-offline at jlab.org>
>     > https://mailman.jlab.org/mailman/listinfo/halld-offline
>     <https://mailman.jlab.org/mailman/listinfo/halld-offline>
>
>
>
>
> _______________________________________________
> Halld-offline mailing list
> Halld-offline at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-offline

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20170225/740bd166/attachment-0002.html>