User:Alaexis/AI Source Verification
| Description | Use AI to verify whether sources support the claims they're cited for |
|---|---|
| Authors | Alaexis, Polygnotus, LuisVilla |
| Updated | August 14, 2026 (1 day ago) |
| Source | User:Alaexis/AI_Source_Verification.js |
AI Source Verifier uses large language models to check whether a Wikipedia citation actually supports the claim it's attached to. Click any reference, and the tool will analyze the source content and tell you if the claim is supported, partially supported, or not supported.
Installation
[edit]First install User:Enterprisey/script-installer, then return to this page and click the "Install" button in the infobox. Alternatively, add this line to your Special:MyPage/common.js:
{{subst:iusc|User:Alaexis/AI Source Verification.js}}
Usage
[edit]- Click the "Verify" tab (next to Article / Talk) to open the sidebar. On Vector Legacy and MinervaNeue skins, click the "More" menu and select "Verify" instead.
- With the sidebar open, click any inline citation (e.g., [1]) in the article
- Select an AI provider from the dropdown (optional)
- Click "Verify" to check if the source supports the cited claim
- The tool will fetch the source content and return a verdict with an explanation
Supported AI providers
[edit]Open-weights: GPT-OSS-20B hosted on HuggingFace Wikimedia account (default), Qwen3-14B-FP8/Qwen3.6-27B-FP8 hosted on Wikimedia LiftWing platform, Qwen Sea Lion hosted by PublicAI; Commercial: Claude (AI), ChatGPT, Gemini (chatbot).
Change Log
[edit]August 7, 2026
[edit]- The tool shows quotes from sources for Not Supported - Contradiction, Partially Supported and Supported verdicts.
August 7, 2026
[edit]- Now feedback lives on-wiki rather than in Google Forms.
August 4, 2026
[edit]- New UI that centers the most commonly used verification actions.
July 2026 Wikimania Push 🇫🇷🇫🇷🇫🇷
[edit]Fixes and features driven by feedback from new and existing users trying the tool during Wikimania.
- Qwen3.6-27B added via Wikimedia's own LiftWing infrastructure - you can now select LiftWing as a provider option.
- PDF upload for offline sources - upload a PDF directly in the verification sidebar for sources that aren't freely available online.
- Thinking budget has been increased to 16k tokens which should improve verdict quality. On longer or more nuanced claims, the model was running out of room to reason before it could finish, sometimes forcing a rushed or truncated verdict.
- Fixed ISBN links being treated as sources.
July 2, 2026 - Checking adjacent citations together
[edit]Adjacent citations are now checked together. The tool gives both the collective verdict and individual verdicts for each citation.
June 24, 2026 - Checking adjacent citations together FINALLY (in beta)
[edit]The beta version of the script now checks adjacent citations together. This functionality has not been extensively tested so please try it out and leave feedback at the talk page.
June 20, 2026 - HuggingFace restored
[edit]HuggingFace-hosted model has been restored as the default.
June 19, 2026 - HuggingFace deactivated temporarily
[edit]The default option is again PublicAI. Expect some degradation of the quality. Once HF becomes available again it will be re-added.
June 16, 2026 - More reliable fetching from Internet Archive via API
[edit]Now the tool uses Internet Archive API to fetch data from Internet Archive. This gives us cleaner data and hopefully less rate limiting issues. If a citation contains both the original and archived links, the tool prefers the archived version as it's not subject to link rot. For citations with only a live link, if the fetch fails the tool attempts to check the Wayback Machine for an archived snapshot as a fallback.
June 8, 2026 - Distinguishing omission from contradiction
[edit]The "Not supported" verdict now has two variants: "Omission" and "Contradiction". Contradiction is a stronger signal that requires action even when it appears in just one source out of a group, or in a source that has not been fully loaded.
May 10, 2026 - Grouping citations, default model change
[edit]Multiple citations supporting the same claim are now grouped together in the Verify All report. You can see at a glance when a "partially supported" or "not supported" verdict applies to just one source within a group - the claim itself may still be covered by its other sources. This is the first step towards joint evaluation of citation groups.
The default free model is now GPT-OSS-20B (an open-weights model developed by OpenAI), hosted on HuggingFace Wikimedia account. The reason for this change is that Qwen3-32B had trouble following the JSON schema and tended to start its responses with "Okay, let's tackle this claim".
May 8, 2026 - Default model change
[edit]The default LLM provider is now HuggingFace (Qwen3-32B, hosted on Wikimedia account), replacing PublicAI. This model is also available for free and is much better in terms of accuracy. Existing users who had PublicAI selected will be automatically migrated to HuggingFace. PublicAI is still selectable from the provider dropdown if you'd like to keep using it.
Feature requests
[edit]Something missing? Feel free to drop a note at the talk page. There is also an issue tracker here.
Performance and limitations
[edit]The tool successfully detects about 70% of citations that don't support the claims they are supposed to support. For "good" citations, it misfires in <15% of cases. Since the overwhelming majority of citations support the sources they're attached to, most hits will be false positives and thus manual review is always necessary. That said, the underlying problem is real: across all references checked (not just flagged ones), roughly 1 in 20–50 contains a genuine accuracy issue. See here for more details.
Limitations
[edit]- Online sources only: The tool can only verify sources available on the web. For paywalled or offline sources, you can paste the text manually.
- False positives: many accurate citations are flagged as not supported due to the limitations of the underlying models or due to technical limitations (e.g., paywalled or video sources)
- Accuracy varies by model: Commercial models (Claude, GPT, Gemini) significantly outperform the free option.
- Limited support of PDF sources
Out of scope
[edit]- Reliability of sources: The tool only checks whether a source supports the claims made.
- Copyvio: The tool does not check whether article text is copied verbatim or closely paraphrased from a source.
Credits
[edit]This script is based on Polygnotus's AI Source Verification script. Thanks to User:LuisVilla for multiple improvements and to many other editors who have been testing it.
The free inference is powered by open-weight models hosted on PublicAI and on Wikimedia HuggingFace account.
Technical details
[edit]Source content is fetched via a proxy that also handles rate limiting (code: ). API keys are stored locally in your browser (localStorage) and can be removed at any time.