The author argues that perplexity, a measure of how well a model predicts the next token on average, is a key metric to compare between model developers and inform shared safety standards in AI. Perplexity has been used to track progress in pretraining models since GPT-3.5, with significant advancements seen at log-perplexities below 0.8.