[Halld-physics] 3pi fits updated

[Richard Jones]

Jake,

Thanks for these plots.  I offer two comments based on my reflections on what you are showing, which we all need to think hard about because they will strongly affect the scientific conclusions we are able to draw from this kind of analysis.

   1. *There is no theorem protecting us from significant bias in the best-fit amplitudes returned by a maximum likelihood fit. * The maximum-likelihood estimator is a /_consistent,_ _minimum-variance_ estimator. /The following two properties can be proved for maximum likelihood estimators/, but they are not unbiased -- one should expect systematic bias that shifts best-fit values from being centered around the true value. / Experience//shows that these shifts are often small, but we should expect them to be present, and measure them./
      /
          * /consistency -- / the estimator expectation value converges to the true parent distribution value, in the limit where the number of events /N /in an individual fit goes to infinity.  Sometimes in the literature this property is also called "/asymptotically unbiased/".
          * /minimum-variance /-- if you repeated the same experiment /R/ times with the same finite number /N/ of events per experiment, the fluctuations in the best-fit values from /R/ fits to data from these /R/ independent experiments would be the lowest possible among all consistent estimators. (Here, think small fluctuations, but around what value?!?)
   2. *These intrinsic PWA biases are dependent on the sample size and decrease with increasing sample size **like 1/sqrt(/N/), the same as the statistical error. *This means that increasing /N/ will not increase the significance of the bias, as the bias scales with /N/ at the same rate as the statistical error.  As an experiment pushing toward unprecedented statistics, this would be a very valuable property if we could make sure it works for us.

Assuming that Jake's initial results hold up to further scrutiny, we can interpret the background amplitude coming from the fits to perfect-resolution fits as an estimate of this fundamental bias for /N/=1,000,000 (for this channel, for this simulated detector, for these physics inputs).  Repeating fits with independent sets of 1M events would allow us to nail down this bias to arbitrary precision, and predict it for /N/=10M or /N/=100M.  If we explicitly correct for it with a calibrated delta/sqrt(N) factor, then the residual bias scales like 1/N, even better!   We might correct for it or include it as a systematic error if it is small enough -- looks promising.

More troubling is the much larger bg amplitude returned when detector smearing is included. /In this case the MLE estimators do not even satisfy the consistency test./  This is because we are generating the sample based on generated angles/momenta, but fitting the sample based on reconstructed angles/momenta. *So we have no theorems to protect us in this case*, only hopes that the violations are small in some sense.  Looking at Jake's results, it looks to me like these MLE estimator biases are on the order of 10% (distressingly large) in this channel.

When they are not small, an innovative approach may be needed that regains some of the protection of the MLE theorems.  These theorems would be extremely valuable to have working for us.  I have some ideas for a direction to head in seeking this, but no concrete results so far.  Whatever we do, we would be relying on Monte Carlo to describe the smearing effects and undo them, in some sense, when the fit is carried out.

-Richard J.






On 6/21/2011 3:50 PM, Jake Bennett wrote:

> At the meeting on Monday I showed a few plots of fits for pi+pi-pi+n with 100% polarized data.  I have included a few more plots to compare fits with actual detector and perfect acceptance and resolution.  The flat background component of the fit is still present when the detector resolution is removed, but it is reduced compared to that with detector resolution.
>
> You can find the updated slides at http://www.jlab.org/Hall-D/software/wiki/images/3/3b/Update_6_21.pdf.
>
> I am working out a few issues with the fits to unpolarized data, but I should have those soon.
>
> Jake
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.jlab.org/pipermail/halld-physics/attachments/20110622/63a12532/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4092 bytes
Desc: S/MIME Cryptographic Signature
Url : https://mailman.jlab.org/pipermail/halld-physics/attachments/20110622/63a12532/attachment.bin