[Halld-pid-upgrade] Fwd: Re: thoughts after PID session [from Mike]

[Michael Williams]

The standard FOM in HEP is S/sqrt(S+B), but it's important to think about why that is.   The reason is that in HEP most FOM usage is to optimize a selection where the goal is to obtain the highest signal significance.   S/sqrt(S+B) is vaguely related to significance ... but vastly underestimates the significance if S >> B.    

Some people prefer S/sqrt(B) which works well unless B is small at which point it overestimates the significance. The S/sqrt(S+B) FOM also has the drawback that it requires knowing the relative cross sections to calculate it.  

Punzi's FOM, which is epsilon/(a/2 + sqrt(B)), where epsilon is the signal efficiency and a is how many "sigma" you want to observe the signal at minimum, does not require knowing the S cross section and behaves well at all values of B and S.   

There are others on the market as well.   

In practice I have found that they mostly are maximized at about the same selection, even for very small signal and background.   So which one is used from that prospective seems to be mostly academic.  

For the PID study, the important thing will be, to me, to make sure (again) that what is shown is understood by those viewing it.    If you choose S/sqrt(S+B) as B gets small w.r.t. S the FOM will not change much in value.   Eg, if S = 10000, B = 1000 then FOM = 95.   If the PID cut removes 90% of the bkgd then FOM = 100.   So we went from 10% bckd to 1% but the FOM changes by very little.    Even the more extreme S = B = 10k gives FOM = 71 then if B = 100 you get 100.   So S/B = 1 to S/B = 100 the FOM changes by 30%.

This is OK (it is not a problem) but needs to be explained carefully when presenting it.   Whatever FOM you choose here, if it changes by a factor of X it doesn't mean that we will be X times more sensitive to the presence of a hybrid in the decay.  

Mike

On Oct 9, 2012, at 7:49 AM, Elton Smith wrote:

> I agree with Matt that as far as the PID system is concerned, one should 
> focus on S and B. I also agree that S/B is not the right figure of merit 
> and we should use something like S/sqrt(S+B).
> 
> Elton.
> 
> 
> On 10/8/12 6:47 PM, Matthew Shepherd wrote:
>> On Oct 8, 2012, at 5:58 PM, Michael Williams wrote:
>> 
>>> One final question: Do we have any decent predictions for the size of the hybrid rates?  Paul is looking at S/B but we don't really care about S, we care about S*f where f is the fraction of the signal that is from a hybrid.   If the bkgd is of comparable size to the hybrid contribution, then it could be a problem (regardless of how big the non-hybrid signal is).
>> Yes, it can be a problem, but it is more complicated than just a PID problem.
>> 
>> Definitions:
>> 
>> S:  number of events with some topology of stable hadrons
>> f: fraction of these events that come from exotic resonance decay
>> B: number of events that are not of the signal topology
>> 
>> The PID system is going to help us with S/B.  We hope that nature helps us with f so that S*f is bigger than B.  The idea that S*f is big is one of the arguments for building a photoproduction experiment -- we have to test it.  I don't think we have a good idea of how big f is.
>> 
>> There are some measurements for S.  See:
>> 
>> http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=809
>> 
>> which is an old compilation of cross sections.
>> 
>> I would tend to focus on S/B in the PID studies, since that is what the PID system will do.  If we want to go a step further we can use PID system to maximize S/B (or some other FOM.. maybe S^2/(S+B)) and then use the size of B to infer some sensitivity limit on f or the hybrid production cross section.
>> 
>> It may be more clear in a presentation to say a system gives us sensitivity down to a certain value for S*f, than picking an f, which has a big uncertainty, and quoting S*f/B.  We can ask:  which system gives us the most sensitivity to exotics?  All assumptions that go into such a study would be based on some data or a Pythia model.
>> 
>> Matt
>> 
>> 
>> 
>> _______________________________________________
>> Halld-pid-upgrade mailing list
>> Halld-pid-upgrade at jlab.org
>> https://mailman.jlab.org/mailman/listinfo/halld-pid-upgrade
> 
> _______________________________________________
> Halld-pid-upgrade mailing list
> Halld-pid-upgrade at jlab.org
> https://mailman.jlab.org/mailman/listinfo/halld-pid-upgrade