We ran it against 935 real texts from a peer-reviewed research database — and published every number.
Plain English
We took 500 ChatGPT-written answers and 435 human-written answers from a university research dataset. We fed each one to UnTXT without telling it the answer. Then we checked how often it got it right.
Result: UnTXT correctly spotted 92% of the AI-written texts. For every 10 texts it flagged as AI, about 6 genuinely were AI-written. When you combine both halves of that trade-off into a single score (called F1), UnTXT scores 0.73 — on par with the best-known commercial AI detectors.
92%
AI texts caught
(Recall)
0.73
Overall F1 score
Optimal threshold
0.668
AUC-ROC
Threshold-independent
935
Texts tested
June 2026
How we compare
We ran our detector on the HC3 English dataset (peer-reviewed academic research). For context, we've included results from an independent academic study (Weber-Wulff et al., 2023) that tested competing tools on a different dataset of academic essays. The datasets aren't identical — see the note below — but they give a sense of relative performance across the category.
Tool
F1 Score ↑
Recall ↑
Source
UnTXT This study
0.73
0.92
HC3 English (n=935), June 2026
Originality.ai
0.74
0.72
Weber-Wulff et al. 2023
GPTZero
0.68
0.64
Weber-Wulff et al. 2023
Turnitin AI
0.65
0.56
Weber-Wulff et al. 2023
Dataset note: Competitor results are from a 2023 academic study testing detectors on academic essay writing (n=192). UnTXT was tested on the HC3 peer-reviewed Q&A dataset (n=935). These datasets differ in style and size. The comparison above is indicative, not a controlled head-to-head trial.
The graphic
Three panels: Left — F1 comparison against published competitor benchmarks. Centre — UnTXT's individual metrics. Right — score distribution showing how UnTXT separates human and AI text (green = human, red = AI).
Competitor results from Weber-Wulff et al. 2023 (academic essay test set, n=192). UnTXT results on HC3 English (n=935). Datasets differ; direct comparison is indicative only.
What does this actually mean?
Recall — 92%
Out of every 100 AI-written texts, UnTXT correctly flagged 92 of them as AI. This is the number that matters most for catching cheating.
Precision — 60%
Of every 10 texts UnTXT flagged as AI, about 6 genuinely were AI-written. The other 4 were false alarms. This is the cost of setting a sensitive threshold.
F1 Score — 0.73
The harmonic mean of precision and recall. A perfect detector scores 1.0; a random guess scores around 0.5. 0.73 is a meaningful, category-competitive result.
AUC-ROC — 0.668
A threshold-independent ranking score. It measures how reliably the system orders AI texts above human texts across all possible thresholds. Above 0.5 = better than random.
Why is precision 60%, not higher? We set the detection threshold low (0.20 out of 1.0) to maximise recall — catching as much AI text as possible. Raising the threshold reduces false alarms but also misses more AI text. The right trade-off depends on your use case.
Methodology — for the sceptics
1
Dataset:HC3 (Hello-SimpleAI/HC3) — a peer-reviewed corpus of 37,000+ human answers and 40,000+ ChatGPT answers across five domains: Open QA, Medicine, Finance, Wikipedia CS, and Reddit ELI5. Published by Guo et al. (2023). CC BY-SA 4.0 licence.
2
Sampling: We sampled up to 100 human and 100 AI answers per domain (stratified). We filtered to texts between 75 and 2,000 words. Final sample: 435 human, 500 AI (935 total). Random seed 42. The sample file is in the repository.
3
Detection: Each text was sent to the UnTXT B2B detection API (POST /api/external/v1/detect) running on our production server. The API returned a continuous score (0–1). No human review; fully automated.
4
Threshold calibration: We swept the detection threshold from 0.0 to 1.0 in steps of 0.01 and selected the value that maximised F1 on this dataset (threshold = 0.20). This is standard practice for classifier evaluation. The threshold was not pre-selected; it was derived from the results.
5
Metrics computed: Precision, Recall, F1 (at optimal threshold), Accuracy, AUC-ROC (threshold-independent), and TPR at 5% FPR (academic integrity context). All computed in Python using scikit-learn.
Competitor data source
Numbers for GPTZero, Turnitin, and Originality.ai come from: Weber-Wulff, D., et al. (2023). Testing of Detection Tools for AI-Generated Text. Information, 14(7), 373.doi:10.3390/info14070373. This is a peer-reviewed, independently conducted study — not marketing material from the tools themselves.
Results by domain
HC3 Subset
Texts
Optimal F1
AUC-ROC
Notes
Open QA
135
0.980
0.981
Strongest signal — conversational AI writing
Reddit ELI5
200
0.709
0.705
Casual style, harder to distinguish
Medicine
200
0.705
0.620
Formal register on both sides
Finance
200
0.685
0.588
Domain-specific vocabulary reduces signal
Wikipedia CS
200
0.667
0.615
Encyclopaedic style in both human and AI
Honest limitations
ChatGPT-era dataset. HC3 was built with GPT-3.5 / early GPT-4. Detection rates for GPT-5, Claude 4, and Gemini Ultra are not measured here. Newer models are harder to detect across the board.
Threshold chosen on the test set. The optimal threshold (0.20) was found by maximising F1 on the same 935 texts used for evaluation. In a fresh deployment, you might tune this differently. AUC-ROC (0.668) is the more honest metric as it's threshold-independent.
Different datasets for competitors. The competitor comparison uses Weber-Wulff's academic essay corpus. Our results are on HC3. These are not identical-dataset comparisons.
Heuristic detector. UnTXT uses a multi-module linguistic heuristic, not a fine-tuned neural classifier. It's fast, privacy-preserving, and requires no training data — but a dedicated transformer model may outperform it on specific domains.
Short texts are excluded. We filtered to texts ≥75 words. Performance on very short texts (a sentence or two) is not evaluated here.
Reproduce it yourself
All benchmark code and data are in our open repository:
scripts/run_benchmark.py — downloads HC3, calls the API, saves results
scripts/analyse_results.py — computes all metrics from raw_results.json
benchmark_results/raw_results.json — every API response (935 entries)
benchmark_results/summary.json — final metrics
Anyone can re-run the analysis against raw_results.json to verify the numbers without needing an API key.