<html>
<head>
</head>
<body bgcolor="#ffffff" text="#000000">
Jake,<br>
<br>
Thanks for these plots. I offer two comments based on my
reflections on what you are showing, which we all need to think hard
about because they will strongly affect the scientific conclusions
we are able to draw from this kind of analysis.<br>
<ol>
<li><b>There is no theorem protecting us from significant bias in
the best-fit amplitudes returned by a maximum likelihood fit. </b>
The maximum-likelihood estimator is a <i><u>consistent,</u> <u>minimum-variance</u>
estimator. </i>The following two properties can be proved
for maximum likelihood estimators<i>, but they are not unbiased
-- one should expect systematic bias that shifts best-fit
values from being centered around the true value. </i>
Experience<i> </i>shows that these shifts are often small, but
we should expect them to be present, and measure them.<i><br>
</i></li>
<ul>
<li><i>consistency -- </i> the estimator expectation value
converges to the true parent distribution value, in the limit
where the number of events <i>N </i>in an individual fit
goes to infinity. Sometimes in the literature this property
is also called "<i>asymptotically unbiased</i>".<br>
</li>
<li><i>minimum-variance </i>-- if you repeated the same
experiment <i>R</i> times with the same finite number <i>N</i>
of events per experiment, the fluctuations in the best-fit
values from <i>R</i> fits to data from these <i>R</i>
independent experiments would be the lowest possible among all
consistent estimators. (Here, think small fluctuations, but
around what value?!?)<br>
</li>
</ul>
<li><b>These intrinsic PWA biases are dependent on the sample size
and decrease with increasing sample size </b><b>like 1/sqrt(<i>N</i>),
the same as the statistical error. </b>This means that
increasing <i>N</i> will not increase the significance of the
bias, as the bias scales with <i>N</i> at the same rate as the
statistical error. As an experiment pushing toward
unprecedented statistics, this would be a very valuable property
if we could make sure it works for us.<br>
</li>
</ol>
Assuming that Jake's initial results hold up to further scrutiny, we
can interpret the background amplitude coming from the fits to
perfect-resolution fits as an estimate of this fundamental bias for
<i>N</i>=1,000,000 (for this channel, for this simulated detector,
for these physics inputs). Repeating fits with independent sets of
1M events would allow us to nail down this bias to arbitrary
precision, and predict it for <i>N</i>=10M or <i>N</i>=100M. If
we explicitly correct for it with a calibrated delta/sqrt(N) factor,
then the residual bias scales like 1/N, even better! We might
correct for it or include it as a systematic error if it is small
enough -- looks promising.<br>
<br>
More troubling is the much larger bg amplitude returned when
detector smearing is included. <i>In this case the MLE estimators
do not even satisfy the consistency test.</i> This is because we
are generating the sample based on generated angles/momenta, but
fitting the sample based on reconstructed angles/momenta. <b>So we
have no theorems to protect us in this case</b>, only hopes that
the violations are small in some sense. Looking at Jake's results,
it looks to me like these MLE estimator biases are on the order of
10% (distressingly large) in this channel.<br>
<br>
When they are not small, an innovative approach may be needed that
regains some of the protection of the MLE theorems. These theorems
would be extremely valuable to have working for us. I have some
ideas for a direction to head in seeking this, but no concrete
results so far. Whatever we do, we would be relying on Monte Carlo
to describe the smearing effects and undo them, in some sense, when
the fit is carried out.<br>
<br>
-Richard J.<br>
<br>
<br>
<br>
<br>
<br>
<br>
On 6/21/2011 3:50 PM, Jake Bennett wrote:<br>
<br>
<blockquote type="cite" cite="mid:BANLkTimgGJ1BrNvK7HRE_0O3SLEWHhVkUw@mail.gmail.com">
<pre wrap="">At the meeting on Monday I showed a few plots of fits for pi+pi-pi+n with 100% polarized data. I have included a few more plots to compare fits with actual detector and perfect acceptance and resolution. The flat background component of the fit is still present when the detector resolution is removed, but it is reduced compared to that with detector resolution.
You can find the updated slides at <a href="http://www.jlab.org/Hall-D/software/wiki/images/3/3b/Update_6_21.pdf">http://www.jlab.org/Hall-D/software/wiki/images/3/3b/Update_6_21.pdf</a>.
I am working out a few issues with the fits to unpolarized data, but I should have those soon.
Jake
</pre>
</blockquote>
<br>
</body>
</html>