What your AI strength score of 175 reveals about the data behind LLMs
a new tool 'in the weights' measures how strongly ai models recall specific people without web search, with scores ranging from 0 to 996. this exposes which individuals are encoded in model weights and highlights the dependency of smaller models on high-relevance training data.
joey flynn and thomas dimson, both ex-openai, released 'in the weights', a search tool that queries several llms to detect if a specific person is stored within the model's weights rather than retrieved via search. the tool aggregates results to generate a 'strength score' on a leaderboard where famous figures like taylor swift rank near 996, whereas an average journalist might score 175. the creators note that smaller models, such as meta's llama, possess tighter capacity constraints, meaning they only store individuals considered highly relevant during training. this utility offers transparency into who is represented in foundation model training data and demonstrates the varying memorization capacities across different model architectures.