Original Summary

This is being thrown around a lot, including the dreaded &quot;10% chance by the end of this decade&quot;. Every conversation I have seen about this, mostly on twitter, not once have I seen anyone connecting the dots on concretely and plausibly how this might pan out. It is usually 1. Train powerful AI 2. ??? 3. Extinction<p>I personally find it implausible, but some quite smart people have talked about it so I am also not dismissing it as pure fear-mongering.<p>The reasons I do not think it is plausible:<p>1. Limited real world projection: an exceptionally smart AI is less powerful than a dumb person with the capability of shutting off AI&#x27;s power<p>2. Confinements: The AI &#x27;escaping&#x27; incidents that got media attention were really the failure of infra, probably due to the scenarios being novel. But it is probably a lot better PR to say &#x27;our model escaped&#x27; than &#x27;our infra was poorly thought out&#x27;. A fully confined model has a significantly less blast radius (claude ruined this term)<p>3. The game theoretic aspect: &#x27;If all else stays the same and AI kept getting powerful, scenario X will materialize&#x27; is tempting to think, but all else never stays the same. Especially with AI, there has already been enough fear and paranoia spread that people are unwilling to give an un-sandboxed AI control of a laptop, let along more consequential outcomes. And the already spread AI FUD almost guarantees that the outcomes where AI controls substantial parts of the infra&#x2F;economy autonomously are very unlikely to materialize, at least in short&#x2F;medium term.<p>I would love to hear competing hypotheses where an AI can cause something resembling extinction and how.

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Hacker News 新项目
  • 发布时间：2026/9/20 23:16:06