Workflow · Reproducibility

Reproducible publication workflow

This workflow is for analyses intended to support a manuscript, validation record, benchmark, or other durable scientific claim.

Workflow overview

1. Record software identity

For release 0.1.0:

python -m pip install gazeaudit==0.1.0

The version-specific archive DOI is 10.5281/zenodo.22757340.

For analyses run from development main, record the exact Git commit rather than describing the code only as “latest”.

2. Preserve source identity

Depending on the source, preserve one or more of:

• DOI and version;
• immutable repository commit;
• download URL and retrieval timestamp;
• cryptographic checksum;
• source manifest;
• participant/file inventory;
• preprocessing provenance supplied by the upstream ecosystem.

The level of source locking should match the strength of the claim. Formal validation evidence benefits from stricter source control than an exploratory tutorial.

3. Freeze the scientific specification

Before final classification, preserve:

• endpoint definition;
• AOI geometry;
• measurement-error model and grouping;
• specification factors and levels;
• invalid-combination rules;
• perturbation grids;
• reference effect and rationale, if used;
• ConclusionRule, if used;
• expected software/version constraints when external packages are scientifically material.

4. Execute without post-hoc filtering

Preserve every valid specification that was declared. If a specification fails technically, record the failure and its reason rather than silently deleting it.

This distinction is central to the GazeBase validation record, where completeness itself is part of the scientific gate.

5. Build a conclusion audit bundle

from gazeaudit import ConclusionRule, build_conclusion_audit_bundle

rule = ConclusionRule(
    relative_tolerance=0.20,
    require_sign=True,
    minimum_recovery_fraction=0.90,
)

bundle = build_conclusion_audit_bundle(
    results,
    reference_effect=reference_effect,
    rule=rule,
    title="Study robustness audit",
    endpoint="predeclared scientific endpoint",
    source_description="immutable source description",
)

Only use a reference effect when its scientific meaning is independently justified.

6. Preserve deterministic outputs

Useful bundle outputs include:

bundle.summary
bundle.scientific_fingerprint
bundle.bundle_fingerprint
bundle.manifest_json()
bundle.markdown

The scientific fingerprint binds scientific inputs/outputs. The bundle fingerprint additionally binds recorded execution context.

7. Verify before publication

from gazeaudit import verify_publication_audit_bundle

verify_publication_audit_bundle(bundle)

If bound content has changed, verification should fail rather than silently bless the modified bundle.

8. Archive methods and complete evidence

Recommended archive layout:

analysis/
  protocol/
    scientific-specification.json
    source-lock.json
  results/
    specifications.csv
    sensitivity-curves.csv
    robustness-summary.json
  publication/
    manifest.json
    methods.md
    report.md
    fingerprints.txt
  environment/
    software-versions.txt

This is an illustrative structure, not a required GazeAudit filesystem schema.

For a more complete reviewer-facing example—including decision history, execution-status accounting, Methods/Results wording, limitations, provenance, fingerprints, and a human-readable archive manifest—see the publication and archive handoff example. Its effect values are synthetic teaching material, not validation evidence.

9. Run the manuscript-readiness gate

Before submission, test whether the manuscript and archive can be reconstructed without private lab context.

Use the manuscript readiness checklist to verify:

1. one reconstructable scientific endpoint;
2. the full declared/valid/successful/failed denominator;
3. researcher decisions separated from software diagnostics;
4. Results wording that matches the complete observed pattern;
5. agreement among manuscript, decision log, limitations, and archive manifest;
6. exact software identity and claim-supporting files.

Then work through the reviewer reconstruction example to see why six directionally consistent estimates do not license a complete-space robustness claim when a seventh valid branch remains unresolved.

10. Preserve reviewer-requested amendments

Peer review can legitimately add new sensitivity analyses, thresholds, exclusions, measurement assumptions, or endpoints. Those analyses belong in the final scientific record, but they should extend the submitted audit rather than retrospectively rewrite it.

Use the reviewer-requested amendments guide to record:

• the reviewer item and review round;
• whether the original outcomes had already been inspected;
• the scientific rationale for the added work;
• the amendment-specific specification space and execution status;
• the software identity used for the amendment;
• the relationship between amendment evidence and the revised manuscript claim.

Keep temporal denominators explicit. For example, a submitted 8 / 8 audit followed by a reviewer-requested 4 / 4 sensitivity amendment remains 8 / 8 submitted + 4 / 4 post-review. It is not “12 pre-specified analyses.”

Work through the reviewer-requested reanalysis example for a complete synthetic revision exercise, including response-letter wording, revised Results and limitations, amendment archive structure, and failed-branch handling.

11. Bind the reviewer response to the evidence

Once revision analyses are complete, keep the response letter synchronized with the manuscript and revision archive.

Use the reviewer response letter guide so each reviewer item records:

• the request and response category;
• whether the relevant decision was introduced after outcome inspection;
• the analytical action, including no-change decisions;
• the submitted and post-review denominators separately;
• the manuscript locations changed;
• the revision files that support the response.

Then work through the revision response package example to see a clarification, a stricter-threshold sensitivity amendment, and a different-endpoint request carried through one synthetic response matrix and archive.

12. Finalize the resubmission and editor handoff

Before sending the revised manuscript, bind the response package to the final version transition.

Use the version-change manifest guide to give each material manuscript change a stable identifier and distinguish:

• editorial text changes;
• documentation clarifications;
• corrections with superseded evidence;
• sensitivity and analytical amendments;
• endpoint and measurement amendments;
• reporting-boundary changes.

Then apply the resubmission readiness and editor handoff guide to verify that:

• every reviewer item has a stable disposition;
• submitted and post-review evidence remain temporally distinct;
• the response letter and revised manuscript use the same denominators and endpoint definitions;
• every manifest entry points to real revision evidence;
• software identity and fingerprints correspond to final claim-supporting files;
• unresolved failures and uncertainty remain visible;
• an outsider can follow the editor-facing evidence map from final claim to source evidence.

The submission-to-accepted-record worked example demonstrates the entire synthetic sequence from an 8 / 8 submitted audit through a 4 / 4 review amendment, separate endpoint amendment, response matrix, version manifest, and final archive.

13. Report claims at the right scope

A robust conclusion under one frozen protocol does not imply universal robustness to every conceivable pipeline. A fragile conclusion under one protocol does not imply the source dataset is unusable.

Report:

• what was varied;
• what was held fixed;
• the endpoint;
• the conclusion rule, if any;
• the observed stability/fragility pattern;
• the execution denominator and unresolved valid failures;
• any post-review amendment and its separate denominator;
• the response-letter relationship to the final manuscript claim;
• material version changes that affect claim scope;
• the protocol boundary;
• known untested uncertainty dimensions.

14. Cite reproducibly

For GazeAudit 0.1.0, cite the version DOI and record the software version or commit:

Balaskas, S. (2026). GazeAudit: Measurement uncertainty and inferential robustness for eye-tracking research (Version 0.1.0) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.22757340

Use the concept DOI 10.5281/zenodo.22757339 only when you intentionally want a reference that resolves to the latest archived version.

GazeAudit’s own frozen evidence

The package’s validation programme applies stronger archive-before-reveal controls to its canonical case studies. See the validation matrix and scientific methods for the authoritative records.