In If Anyone Builds It, Everyone Dies, Eliezer Yudkowsky and Nate Soares contend that superhuman artificial intelligence developed using today's machine learning methods will result in human extinction. Yudkowsky and Soares believe that conventional prudence has failed and thus write in terms intended to portray delay and compromise as morally reprehensible. This review examines the book in terms of how it addresses the relationship among science-based knowledge, certainty and the role of government. Why does that matter? Because the authors' recommended policies rely upon the degree to which they can document their empirical findings, and the two, the findings and the proposed policies, are typically evaluated by different people. A major structural weakness in the book is that it spends the most time on the authors' weakest argument, that extinction is obviously possible, rather than their stronger one; the authors' strongest case does not appear until very late in the book. When a system has the capacity to inflict permanent damage, the burden of establishing safety is placed on those who create such systems. The book is most persuasive when distinguishing between interpretability and control and when arguing that the developers of risky systems should demonstrate their safety before deploying those systems. On the other hand, the book is least persuasive when asserting that extinction is all-but-certain despite acknowledging uncertainty regarding the specific mechanisms, timeframes and defense strategies. Ultimately, we believe that this book serves as a valuable warning, but that its advocacy position would be strengthened had it been decoupled from the certainty asserted in its title.
Scroll to read the preview. Download for the complete document.
Grace, K., Stewart, H., Sandkuehler, J. F., Thomas, S., Weinstein-Raun, B., Brauner, J., & Korzekwa, R. C. (2024). Thousands of AI authors on the future of AI (arXiv:2401.02843). arXiv. https://doi.org/10.48550/arXiv.2401.02843
Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J., Belonax, T., Chen, J., Duvenaud, D., Khan, A., Michael, J., Mindermann, S., Perez, E., Petrini, L., Uesato, J., Kaplan, J., Shlegeris, B., Bowman, S. R., & Hubinger, E. (2024). Alignment faking in large language models (arXiv:2412.14093). arXiv. https://doi.org/10.48550/arXiv.2412.14093
Sastry, G., Heim, L., Belfield, H., Anderljung, M., Brundage, M., Hazell, J., O’Keefe, C., Hadfield, G. K., Ngo, R., Pilz, K., Gor, G., Bluemke, E., Shoker, S., Egan, J., Trager, R. F., Avin, S., Weller, A., Bengio, Y., & Coyle, D. (2024). Computing power and the governance of artificial intelligence (arXiv:2402.08797). arXiv. https://doi.org/10.48550/arXiv.2402.08797
United Nations. (2015). Transforming our world: The 2030 agenda for sustainable development (A/RES/70/1). United Nations General Assembly.
Yudkowsky, E., & Soares, N. (2025). If anyone builds it, everyone dies: Why superhuman AI would kill us all. Little, Brown and Company.
Metrics are updated in real time as the article is accessed and downloaded.
Comments
Leave a Comment
