Discord AI, 체스판 이미지 때문에 8,000명 이상 사용자 정지
discord는 스프레드시트 같은 무해한 이미지를 아동 보호 관련 자료로 잘못 표시한 버그로 인해 8,000명 이상의 사용자를 정지했다가 복구했습니다. 이는 자동화된 모더레이션의 위험성을 보여줍니다.
discord fixed a bug in its automated moderation system that wrongfully banned more than 8,000 users over two months after flagging benign images such as game textures and grid patterns as illegal content. the glitch bypassed the standard human review step, causing immediate account suspensions for content that vaguely resembled known harmful material in databases. discord stated it is implementing safeguards to prevent recurrence, but the incident sparked outrage among users who rely on the platform for livelihood and community. this case signals a growing challenge for tech companies relying on ai to police content at scale, proving that false positives in automated safety systems can have severe consequences for user trust.