Simulation-based calibration and measurement-resolution guards

Bayesian/process estimators can be computationally wrong even when they return plausible-looking results. sbc_rank_diagnostics() provides a lightweight diagnostic layer for rank-based simulation-based calibration (SBC). The function deliberately does not fit a Bayesian model itself: users provide the ranks generated by a correctly specified simulation/inference loop. This keeps the diagnostic separate from the estimator and avoids pretending that SBC establishes substantive model validity.

ranks <- read.csv(system.file("extdata", "sbc_rank_demo.csv", package = "eyeprocess"))
sbc <- sbc_rank_diagnostics(ranks$rank, n_draws = unique(ranks$n_draws), bins = 10)
print(sbc)
plot(sbc)
sbc_ecdf_deviation(sbc)

SBC asks whether posterior computation is calibrated under the declared generative model; it does not establish that the generative model is scientifically correct for real participants or tasks. The architecture follows the rank-calibration logic described by Talts et al. (2018) and current Stan documentation.

Measurement resolution poses a separate problem. An analysis may request temporal or spatial distinctions finer than the empirical recording quality can credibly support. analysis_resolution_guard() combines an observed/declared event duration and effective sampling frequency with optional spatial feature size and radial error. The thresholds are researcher-declared compatibility rules, not universal eye-tracking quality cutoffs.

analysis_resolution_guard(
  event_duration_ms = 100,
  effective_hz = 60,
  spatial_feature_size = .20,
  radial_error = .04,
  min_samples = 3,
  max_error_fraction = .5
)

For pupil analyses, audit_pupil_preprocessing_order() and pupil_baseline_sensitivity() make the preprocessing sequence and baseline-window dependence inspectable. They report consequences of declared choices rather than automatically selecting a preferred baseline.

pupil <- read.csv(system.file("extdata", "pupil_baseline_demo.csv", package = "eyeprocess"))
pupil_baseline_sensitivity(
  pupil,
  time = "time_ms",
  pupil = "pupil",
  by = c("person_id", "trial_id"),
  windows = list(W500 = c(-500, 0), W300 = c(-300, 0), W200 = c(-200, 0))
)

Interpretation boundary

A successful SBC diagnostic supports the computational calibration of a declared Bayesian workflow under simulation. A passing resolution guard indicates compatibility with user-declared numerical rules. Neither result, alone, validates a psychological construct or a universal measurement threshold.

Methodological anchors

• Talts S, Betancourt M, Simpson D, Vehtari A, Gelman A (2018). Validating Bayesian Inference Algorithms with Simulation-Based Calibration. arXiv:1804.06788.
• Niehorster DC, Nyström M, Hessels RS, et al. (2026). The fundamentals of eye tracking, Part 7: Determining data quality. Behavior Research Methods 58, 183. DOI: 10.3758/s13428-026-03039-4.
• Mathôt S, Fabius J, Van Heusden E, Van der Stigchel S (2018). Safe and sensible preprocessing and baseline correction of pupil-size data. Behavior Research Methods 50, 94–106. DOI: 10.3758/s13428-017-1007-2.