Researcher builds open-source AI text detector to study detector limitations and verifier-based LLM applications

Sebastian Raschka, PhD, describes an end-to-end project to build an AI text detector from scratch, including dataset construction, model training, local deployment, and a verifier-based application. The project is framed as an educational demonstration of how AI detectors work and their limitations.

The detector uses a fine-tuned DistilBERT classifier to output a 0–100 score representing the estimated probability that a given text is AI-generated, similar to approaches like Pangram models that underlie Substack’s AI detector feature. The classifier’s score is not a general probability but an estimate based on its training distribution.

The project also explores training a small language model (SLM) with a verifier to produce text that avoids detection, illustrating a use case beyond traditional reasoning models. This verifier-based approach is intended to study how detectors and LLMs interact in a cat-and-mouse dynamic, where detectors identify patterns in AI-generated text and LLMs adapt to avoid those patterns.

The detector is implemented as an API and includes a user-friendly local browser interface that returns a whole-text AI score and highlights scores for individual text chunks. The project is positioned as a practical template for researchers and developers interested in building similar tools or studying detector limitations.

Raschka emphasizes that AI checkers are inherently iterative and prone to false positives, where human-written text may be flagged as AI-generated. The project aims to educate users on these limitations and provide a tool for improving personal writing without over-polishing it into AI-like text that triggers detectors.