For example, if we identify that 50% of a document is likely written by an AI tool, it could contain as much as 65% AI writing. We’re comfortable with that since we do not want to incorrectly highlight human-written text as AI-written. In order to maintain this low rate of 1% for false positives, there is a chance that we might miss 15% of AI written text in a document. To bolster our testing framework and diagnose statistical trends of false positives, in April 2023 we performed additional tests on 800,000 additional academic papers that were written before the release of ChatGPT to further validate our less than 1% false positive rate.
In other words, we might flag a human-written document as AI-written for one out of every 100 fully-human written documents. We strive to maximize the effectiveness of our detector while keeping our false positive rate - incorrectly identifying fully human-written text as AI-generated - under 1% for documents with over 20% of AI writing.
