Why nominal audit-completion weights fail under outcome-dependent returns¶

W3 R92 methods companion · Independently verified synthetic source and full-archive replay · 22 October 2026

Three distinct events¶

The initial missingness mask identifies the realized finite target group of interest: records originally unavailable for analysis. A response-selective secondary validation phase can measure some of these values. For the remaining unavailable group \(U\), independently randomized audit invitation has probability \(g\).

Successful completion of a selected audit is another process, with return probability \(c_i\). If it depends on the still-unseen outcome, the probability of actually observing that outcome is \(g c_i\), not \(g\). No method can replace this distinction by treating every invitation as an observation.

The experiment uses a prespecified transformed bounded response \(Z_i=\operatorname{clip}((Y_i+1)/2,0,1)\). Its finite-population target, \(\mu_M=N^{-1}\sum_{i\in M}Z_i\), is conditional on the realized originally missing set \(M\). \(S\) denotes the already observed subset of \(M\) and \(U=M\setminus S\) the still-unobserved target.

Estimators and conditional oracle biases¶

With \(I_i\sim\mathrm{Bernoulli}(g)\) independently assigned and actual response \(R_i\sim\mathrm{Bernoulli}(c_i)\) independently generated by a separate stream conditional on the frozen source, the assignment-only and nominal-adjusted estimates use observed \(Z\) values only:

\[ \widehat\mu_{\rm assignment} =\frac{1}{N}\left[\sum_{i\in S}Z_i+ \sum_{i\in U}\frac{I_iR_iZ_i}{g}\right], \]

\[ \widehat\mu_{\rm nominal} =\frac{1}{N}\left[\sum_{i\in S}Z_i+ \sum_{i\in U}\frac{I_iR_iZ_i}{g(1-\delta)}\right]. \]

Conditional on the complete realized source and previous measured subsets, random invitation and return produce \(E[I_iR_i]=g c_i\). Hence

\[ E(\widehat\mu_{\rm assignment})-\mu_M =-\frac{1}{N}\sum_{i\in U}(1-c_i)Z_i, \]

\[ E(\widehat\mu_{\rm nominal})-\mu_M =\frac{1}{N}\sum_{i\in U} \left(\frac{c_i}{1-\delta}-1\right)Z_i. \]

Those oracle expressions use latent outcomes to verify numerical arithmetic in a synthetic simulator only. They are not observed-data estimators or an identified sensitivity parameter in real eye-tracking data.

The frozen actual return laws are \(c_i=1\) at complete return; \(c_i=1-\delta\) under uniform noncompletion; and \(c_i=1-\delta(0.5+Z_i)\) under selective noncompletion, with \(\delta=0.01, 0.05, 0.15\). The selectively missing simulator rule is deliberately tied to the unknown transformed outcome, with completion probability bounded between \(1-1.5\delta\) and \(1-0.5\delta\). A nominal value \(\delta\) is not an empirical completion probability estimated from data; it is a prespecified policy value for this falsification experiment.

For genuine uniform noncompletion, the second conditional bias is zero by algebraic cancellation. For outcome-dependent completion,

\[ B_{\rm nominal} =\frac{\delta}{N(1-\delta)} \sum_{i\in U}(0.5-Z_i)Z_i, \]

which need not vanish. This is why even a small failure probability can invalidate a weighting identity; a positive, negative or near-zero aggregate bias is determined by the whole unobserved finite source, not by the invitation probability alone.

Coupled simulations, denominators and metric interpretation¶

R92 newly generated 32 independent X/Y primitive conditions per native study design: independent curves and repeated-trial participants. Paired MAR/MNAR first-stage masks, outcome-dependent secondary validation, eight within-source random invitation/response replicates, two nested assignment probabilities and seven completion laws produced 14,336 records, not 14,336 independent source draws.

The original source/simulation streams are disjoint from R91. Identical uniforms are shared across responses and assignment policies within each conditional replicate, permitting direct matched comparisons rather than drawing new outcome sources for different response laws. Increasing nominal failure causes nested actual noncompletion under each return law; increasing assignment fraction produces nested invitations. A mismatch in X/Y, mask or assignment digests across matched scenarios is a failure, not a reason to exclude a source.

The prespecified fixed-cell descriptive summaries include assignment-only and nominal corrected signed error, RMSE of both, simulator-only analytic conditional design bias, actual return and failure counts, and sharp observed-only finite target interval width \((N-n)/N\). None of these represents a confidence interval or a simultaneous functional-band calibration. As the study contains 32 primitive blocks per native design and eight conditional repeats per block, no pseudo-independent 256-source inference is warranted.

An exact oracle conditional design bias and a finite-simulation mean realized error will generally be different: the former averages over the theoretical random assignment-and-return distribution for each fixed source, while the latter averages only eight particular repeated realizations per source. Small finite-simulation error does not imply the true return model is uniform; nor does a source-level empirical RMSE comparison establish general estimator efficiency.

Reproducibility and source-control gates¶

Original R92 science run #39837488254 was 12/12 PASS, archiving all original source-method cases with exact checkout and SHA256 members. Separately coded R92-A independent replay #39838411816 was 4/4 PASS, including the actual immutable original 14,336-case archive. See the R92 original and independent evidence ledger for the original-generating checkout, artifact digests and complete failure-inclusive CI provenance.

Both original and independent research source PRs remain draft and unmerged. Documentation-only publication cannot itself license use of the synthetic return model on actual gaze data. Validity of real MNAR correction, raw gaze recovery, function-on-scalar inference and nominal 90% simultaneous functional bands remains unestablished. Scientific blockers #296 and #289 stay open; stable 1.1.0 unchanged, v1.2/GitHub/PyPI HOLD.