I have a new app out, VerusCite. With the recent rise in popularity of GenAI tools like ChatGPT and Claude, this has also come along with academics writing slop articles.

One of the ways to check that slop is via looking at the articles citations. LLMs have some predictable failure modes in writing papers whole cloth – they tend to get details like complicated author lists wrong, or swap out incorrect journal titles. VerusCite is a tool for editors and reviewers to use to verify citations in a fast and cheap application.

It costs $2 to review a paper (and you get two free reviews on sign-up). If you want to see the output of a single example though, check out https://veruscite-data.com/share/E1-rn3TBwksENO_3hlmsRD0IG-EnXd6vfQg-upQZWQM

In addition to hallucinations, I have made many parts of the application just useful to editors in general. Many papers have minor errors in their bibliographies; typos, years off, author swaps, bad URLs, etc. Here is an example – no hallucinations that signal poor writing, but has seven different errors in the bibliography.

This is par for the course (it is quite possible 5% of citations have errors that look like this). The website has convenient tools to edit citations and export the fixed citations (whether minor errors or gross hallucinations) in various formats.

This makes much lighter work of the tedious job of formatting and checking citations for editors. One of the ways I think is critical to build generative AI tools is to consider the human in the loop from the start. My tool will ultimately make some errors (I error rate estimates in my public benchmark). I want it to be as fast for a human to confirm (or refute) the LLM label.

If you are an editor or a reviewer, I highly suggest you check the application out. Peer review journals (and pre-print servers that review the applications before posting), will need to use a tool like this as a first pass to ensure slop is not being posted.