Validation Status & Study Plan
No independent validation study of this specific implementation has been published. This page records what is known, what is not yet established, and what a future report must include.
Status date: July 26, 2026
Current conclusion: results are model-based educational estimates. The site does not claim a fixed real-world margin of error, criterion validity, diagnostic validity, or equivalence to an official CEFR, GRE, SAT, IELTS, or TOEFL score.
Current method
The main vocabulary test combines a recognition checklist, decoy words intended to detect overclaiming, definition questions, a two-parameter logistic IRT model, Bayesian EAP ability estimation, Fisher Information item selection, and an internal SEM stopping rule. The word banks draw on published vocabulary-frequency research and CEFR-oriented learning resources.
What the internal metrics do not prove
An SEM value produced by the model measures uncertainty conditional on the model, item parameters, and responses. It does not by itself establish a percentage error in total vocabulary size. Referencing a large external corpus also does not validate the scoring implementation, item calibration, stopping rule, or CEFR mapping.
Required validation study
- Sample: preregistered target size, age, education, first language, geographic distribution, and English-learning history.
- Calibration: documented item difficulty and discrimination estimates, item-fit checks, differential item functioning, and held-out evaluation.
- Reliability: test–retest estimates, alternate-form reliability, conditional SEM, and uncertainty across the score range.
- Criterion validity: comparison with an appropriate independently administered vocabulary measure and, separately, official exam outcomes where relevant.
- Confidence intervals: empirically justified intervals with coverage testing, not intervals inferred solely from an internal stopping threshold.
- Limitations: sampling bias, device effects, repeated exposure, guessing, overclaim correction, language background, and the definition of a word family.
Publication standard
A future validation report should publish the protocol, anonymized aggregate results, analysis code where possible, effect sizes, confidence intervals, exclusions, missing-data handling, conflicts of interest, and reviewer identity. Claims on product pages must match the report and be revised when later evidence changes the estimate.
Change log
July 26, 2026: removed the unsupported fixed ±3 percent accuracy claim, separated internal model precision from real-world validity, and published this status page.