[Halld-offline] subtle bug in ROOT

[Richard Jones]

Dear colleagues,

Maybe this is common knowledge to people more versed in ROOT than I am, but
this was a surprise to me, that took me quite some effort to discover why
my TTree analysis was producing nonsense.

>>  TH1F histograms silently truncate their bin contents at 16,777,216
(2^28) <<

unless you fill with non-unity weights, in which case they truncate at
OTHER SMALL VALUES. To be specific, if you create a TH1F histogram h1 and
then do h1.Fill(0.) repeatedly then this bin will silently stop
incrementing when the bin content reaches 1.677216e+7. If you fill it with
a non-unity weight then it will gradually lose precision as the number of
Fill calls on that bin exceeds some threshold like a few M, and then
silently stop incrementing altogether when the bin content reaches some
limit. For w=100 I found this limit to be 2.147e+9. This was unexpected
because the letter F in TH1F implies "float" which has a max value about
3.4e+38. What use is a histogram object that silently discards entries as
soon as the count reaches some small value that we expect to commonly hit
in high-statistics analysis? They must be doing some kind of
range-truncating-compression in the storage of TH1F bin contents.
Personally, I would rather get the right answer, even if it means using
more memory, but that's just me.

A workaround would be never to use TH1F, always TH1D. I have not been able
to discover a similar silent truncation in TH1D. That, plus the fact that
TH1F::Fill() is a couple orders of magnitude slower than TH1F::Fill().
Apparently it takes a lot of cpu time to generate bugs of this kind?

Meanwhile, beware. This is especially insidious because the command
tree.Draw("px") in your interactive ROOT session implicitly creates and
fills a TH1F, not a TH1D, even if px is declared double in your tree. In my
present analysis, my tree has 200M rows, but in principle that will bite
you even if you have only 20M rows in your tree.

-Richard Jones
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jlab.org/pipermail/halld-offline/attachments/20170225/b5257def/attachment.html>