ai可解释性突破引研究者争议

nick cammarata, a researcher at goodfire and former co-author of the paper 'zoom in', argues that neural networks are more interpretable than previously thought. he claims that famous examples of polysemanticity, such as the cat-car neuron in inceptionv1, were misinterpreted. cammarata now sees the cat-car neuron as a v-shape detector, activated by pointed shapes and inhibited by round shapes. he also reinterprets the superposition circuit from 'zoom in' as ordinary generalization. this challenges the superposition hypothesis, which suggests that neural networks pack unrelated concepts into the same neurons. cammarata's new perspective has sparked debate among researchers, with some, like openai's roon, calling it a historic moment, while others, like google deepmind's neel nanda, warn against overhyping it.