Python Suite research package
GazeForge¶
Auditable AI for eye-tracking research¶
Machine learning, computer vision, temporal event modelling, semantic AOIs, scanpaths, validation, and provenance — designed so AI can assist eye-tracking research without silently rewriting the empirical record.
The scientific contract¶
AI may propose, score, classify, embed, or flag. It must not silently alter the empirical record.
GazeForge keeps predictions, confidence, model identity, sampling-rate assumptions, review decisions, and benchmark provenance visible as ordinary research data.
Explore by what you need to do¶
Start safely with research data¶
See what GazeForge actually does
Run one deterministic source → QC → events → AOIs → scanpaths → provenance workflow before choosing a specialist method.
2 · ImportBring a tracker export in without guessing
Freeze participant/trial identity, timestamp units, coordinate basis, geometry, nominal rate, observed cadence, duplicates, and source fingerprints.
3 · ReviewSeparate QC evidence from exclusions
Keep flags, human review, denominators, prespecified criteria, exploratory sensitivity rules, and the primary-analysis derivative distinct.
4 · AnalyzeChoose the method that matches the research task
Route to transparent events, learned-model validation, static/dynamic AOIs, scanpaths, study freeze, reporting, or empirical-evidence review.
The GazeForge research path¶
Start from your task¶
Do the work
Run a workflow¶
Follow one executable path from source gaze through canonicalisation, QC, transparent events, AOIs, scanpaths, provenance, and reviewable exports.
Understand the methods
Choose a method¶
Match the scientific question, available evidence, grouping/generalisation unit, validation requirement, and intended claim before dropping into API details.
Check the claims
:material-shield-search-outline: Inspect evidence¶
Start with the generated evidence layer, then drill into the benchmark whose provenance, labels, sampling rate, and split design match your question.
New here? Start with the GazeForge Tour · Follow a first study · Freeze outcomes & estimands · Choose from the Documentation Map · Choose a method · Understand outputs · Build an evidence bundle · Audit what a gaze measure supports · Audit denominators & exposure · Document missing-data assumptions · Audit grouping & pseudoreplication · Audit model diagnostics · Audit uncertainty & multiplicity · Audit sensitivity & robustness · Report without overclaiming · Share with a reviewer · Troubleshoot a workflow · Also useful: Research recipes · Import real data · Worked tracker import/QC · Review QC & exclusions · Validate event models · Study lifecycle · Study-design templates · Worked dynamic-AOI study · Methods overview · Runnable examples · Validation reporting · Research terminology · Benchmark guide · Reproducible reporting
From research question to publication¶
Plan the study
Follow the study lifecycle¶
Move from an observable research construct through acquisition, import, QC, events, AOIs, scanpaths, validation, provenance, analysis freeze, and reporting.
Learn by example
Run a worked study¶
Use a deterministic static advertising/interface demonstration with explicit brand, claim, disclosure, and product AOIs, reviewable outputs, and source fingerprints.
Prepare model inputs
Reconcile denominators & exposure¶
Keep observed zero, missing, absent-by-design, undefined denominators, partial coverage, and right-censored no-fixation latency distinct before modelling.
Check fitted models
Audit convergence & diagnostics¶
Treat non-convergence, singular/boundary states, separation or invalid covariance, and missing diagnostics as explicit stop conditions before interpretation.
Check inferential reporting
:material-chart-error: Audit uncertainty & multiplicity¶
Keep effect scale, units, interval method/level, confirmatory-family membership, and raw versus adjusted inferential fields explicit before manuscript wording.
Stress-test the analysis
Audit sensitivity & robustness¶
Reconcile preregistered and executed variants, preserve denominator/exposure changes, keep non-evaluable and non-converged cases visible, and separate changed-estimand deviations from direct robustness comparisons.
Before submission
Audit publication readiness¶
Check acquisition provenance, exclusions, model identity, split design, native/derived wording, evidence status, archive identity, and reproducible reporting before freezing manuscript claims.
Freeze the archive
Build the evidence bundle¶
Assemble source identity, pre-review QC, reviewed decisions, the primary-analysis derivative, event/AOI/scanpath outputs, provenance, fingerprints, and a human-readable archive map.
Share the archive
Hand off to a reviewer or replicator¶
Map claims to artifacts and API routes, declare what can be rerun, record software identity and access requirements, and carry limitations forward without treating reproducibility as scientific validity.
The worked study uses synthetic/demo data only and does not establish consumer effects, empirical validation, native-device validity, native 60 Hz validity, Gazepoint validity, or GP3 validity.
See the workflow before you run it¶
The previews below use synthetic/demo data only. They illustrate software behaviour and review surfaces; they are not empirical validation evidence and do not establish native-device, native 60 Hz, Gazepoint, or GP3 validity.
1 · Inspect quality
Non-destructive QC
Flag suspicious samples and review trial quality without silently deleting observations.
2 · Validate predictions
Events & calibration
Keep held-out identity, probabilities, sample/event metrics, calibration, and confidence/coverage visible.
3 · Inspect meaning
AOIs & scanpaths
Review semantic regions and sequence structure while preserving explicit labels and fixation order.
Synthetic/demo preview · not empirical validation evidence. Open all visual diagnostics →
What GazeForge covers¶
-
Eye-event AI
Transparent I-VT and angular I-VT baselines, Random Forest classification, temporal-context MLPs, calibrated probabilities, confidence/coverage analysis, and event-level temporal matching.
-
Semantic AOIs
Human-defined and AI-proposed AOIs, optional OWL-ViT open-vocabulary detection, explicit review/correction, dynamic keyframes, bounded interpolation, and fixation assignment.
-
Scanpaths & sequences
Semantic scanpaths, motifs, TF-IDF/SVD embeddings, similarity, and clustering without requiring an opaque generative model.
-
Validation & provenance
Participant-held-out folds, leave-one-dataset-out validation, calibration, event-level metrics, evidence-aware dataset cards, fingerprints, and protected frozen reports.
Public alpha release¶
GazeForge 0.1.0a2 is the current public alpha release. It is published on PyPI through Trusted Publishing and frozen as the v0.1.0a2 GitHub Release. The release remains intentionally alpha: APIs may change while native 60 Hz/GP3-class validation, broader external benchmark qualification, and remaining dynamic-detection validation are completed.
Release & install guidance → · Citation & attribution → · Changelog & releases →
First frozen empirical checkpoint¶
GazeForge now contains its first reviewed external evidence suite from Lund2013, pinned to an exact upstream commit and verified file-by-file before analysis. The primary lower-rate analysis derives a 60 Hz human-reference condition from the native 500 Hz expert labels and evaluates all methods on identical participant-held-out folds.
| Model | Balanced accuracy | Macro-F1 | Event-F1 |
|---|---|---|---|
| I-VT | 0.388 | 0.287 | 0.626 |
| RandomForest | 0.670 | 0.595 | 0.440 |
| ContextMLP | 0.679 | 0.649 | 0.535 |
The result is intentionally multi-criterion: ContextMLP leads sample-level multiclass classification, while I-VT leads event segmentation and boundary fidelity. The independent MN annotator sensitivity analysis reproduces that broad pattern. Human MN–RA agreement remains high from native 500 Hz (κ = 0.815) to derived 60 Hz (κ = 0.799).
Derived 60 Hz is not native GP3 validation
Lund2013 is a native 500 Hz corpus. These lower-rate results quantify a controlled derivation from expert annotations; they do not establish device-specific validity for a native 60 Hz Gazepoint GP3 recording. Native GP3-class expert-labelled event validation remains a major open evidence gate.
Open the visual results gallery → · Inspect all verified Lund tables → · Read the generated evidence status →
A workflow designed for scientific review¶
vendor / raw gaze
│
▼
canonical gaze schema
│
├── QC anomaly flags ────────────────┐
├── event probabilities ─────────────┤
├── semantic AOI proposals ─ review ─┤
└── scanpath representations ────────┤
▼
reviewed analytic table
│
▼
model-ready statistical handoff
│
▼
statistics / models / report
The package does not infer diagnoses, emotions, personality, protected traits, or unsupported latent mental states from gaze.
Run the practical end-to-end workflow → · Build model-ready tables → · Audit measurement interpretation → · Report without overclaiming → · Choose a research workflow →
Validation is visible, not implied¶
The generated Evidence status page is the canonical public status layer. This summary uses the same status classes while retaining key benchmark-specific boundaries.
| Benchmark | Human reference | Native rate | Current role |
|---|---|---|---|
| Lund2013 | paired expert event labels | 500 Hz | Frozen empirical evidence: native/derived human agreement, derived 60 Hz modelling, annotator and sampling/purity sensitivity |
| Hollywood2EM | sequential student labels with expert-corrected final labels | ≈500 Hz | Frozen empirical evidence: aggregate derived-60-Hz source-token-held-out evidence; token-disjoint only, not participant-disjoint; exact annotation-repository licence and token→participant mapping remain unresolved |
| Gaze-in-the-Wild | distributed trained human labellers | published 120 Hz acquisition; exact ProcessData nominal 300 Hz | Reviewed empirical evidence: exact-distribution participant-disjoint evidence on a derived 60-Hz task-agnostic grid; the complete authoritative numeric TrIdx→task mapping, task-stratified validation, native-60-Hz/GP3 validity, acquisition-hardware cadence verification, and quarantine exit remain open |
| VISUS | one published curated dynamic-AOI annotation process involving two contributors | 60 Hz | Bounded empirical evidence: verified partial public-derivative Tobii 60 Hz observations; the full 25-participant × 11-stimulus benchmark is not recovered, original source licensing remains unresolved, and no full-dataset model-validation, human-human-agreement, Frozen Evidence, or native-GP3 claim is created |
GazeForge never silently upgrades evidence strength. Resampled lower-rate evidence remains labelled as derived, human-human agreement is not treated as an error-free ceiling, and unresolved coordinate or identity evidence blocks stronger cross-dataset claims. For Gaze-in-the-Wild, the reviewed participant-disjoint benchmark remains task-agnostic: first-party task-separated extraction structure and secondary numeric corroboration do not substitute for a complete authoritative TrIdx→task mapping, and TrIdx 4 → Tea_Making is not inferred by elimination. Published 120 Hz acquisition provenance also remains distinct from the official 300 Hz processed-stream target and from the actual analysis cadence from timestamps, which is separately resolved before constructing the derived analysis grid. For VISUS, the current public derivative supports bounded empirical observations only; it does not recover the original full benchmark or establish model validity, independent human-human agreement, Frozen Evidence, unrestricted source redistribution, or native GP3 validity.
Evidence status → · See the full validation matrix → · Gaze-in-the-Wild task-mapping evidence → · VISUS public partial evidence → · See frozen empirical evidence →
Quick start¶
Install the exact public alpha from PyPI:
python -m pip install "gazeforge==0.1.0a2"
For development or commit-pinned research work:
git clone https://github.com/stefanosbalaskas/GazeForge.git
cd GazeForge
python -m pip install -e ".[dev]"
pytest
from gazeforge import ai_flag_anomalies, canonicalize_gaze, simulate_gaze
raw = simulate_gaze(n_participants=3, n_trials=2, samples_per_trial=180)
gaze = canonicalize_gaze(raw, sampling_rate_hz=60)
flagged = ai_flag_anomalies(gaze.data, sampling_rate_hz=60)
Take the GazeForge Tour → · Import real study data → · Run the worked tracker import/QC → · Review QC/exclusions → · Validate a learned event model → · Run the worked validation study → · Continue with the getting-started guide → · Run the practical end-to-end workflow →
Current project phase¶
GazeForge is public alpha research software with Frozen, Reviewed, and Bounded external empirical evidence under explicitly different scientific scopes. Version 0.1.0a2 is published through PyPI Trusted Publishing and frozen in an exact GitHub Release with wheel, source distribution, and SHA-256 manifest; a stable scientific-performance claim still requires broader independent validation.
- CI spans Python 3.10, 3.12, and 3.14 on Linux, Windows, and macOS.
- Documentation is built strictly and deployed through GitHub Pages.
- Public evidence status is generated from versioned policy plus exact evidence bindings and is available as Markdown and machine-readable JSON.
- External benchmark files are not silently bundled or relicensed.
- Frozen benchmark reports carry deterministic SHA-256 fingerprints and are revalidated before website display.
- The highest-priority event-model evidence gap is a native 60 Hz/GP3-class expert-labelled corpus.
Scientific governance → · Citation & attribution → · Evidence status → · Benchmark evidence → · Roadmap on GitHub →