{"id":20654,"date":"2025-01-23T10:00:15","date_gmt":"2025-01-23T11:00:15","guid":{"rendered":"http:\/\/medexperts.pro\/?p=20654"},"modified":"2025-01-23T11:24:22","modified_gmt":"2025-01-23T11:24:22","slug":"a-test-so-hard-no-ai-system-can-pass-it-yet","status":"publish","type":"post","link":"https:\/\/medexperts.pro\/?p=20654","title":{"rendered":"A Test So Hard No AI System Can Pass It \u2014 Yet"},"content":{"rendered":"
<\/div>\n
\n
\n

If you\u2019re looking for a new reason to be nervous about artificial intelligence, try this: Some of the smartest humans in the world are struggling to create tests that A.I. systems can\u2019t pass.<\/p>\n

For years, A.I. systems were measured by giving new models a variety of standardized benchmark tests. Many of these tests consisted of challenging, S.A.T.-caliber problems in areas like math, science and logic. Comparing the models\u2019 scores over time served as a rough measure of A.I. progress.<\/p>\n

But A.I. systems eventually got too good at those tests, so new, harder tests were created \u2014 often with the types of questions graduate students might encounter on their exams.<\/p>\n

Those tests aren\u2019t in good shape, either. New models from companies like OpenAI, Google and Anthropic have been getting high scores on many Ph.D.-level challenges, limiting those tests\u2019 usefulness and leading to a chilling question: Are A.I. systems getting too smart for us to measure?<\/p>\n<\/div>\n<\/div>\n

<\/div>\n
\n
\n

This week, researchers at the Center for AI Safety and Scale AI are releasing a possible answer to that question: A new evaluation, called \u201cHumanity\u2019s Last Exam,\u201d that they claim is the hardest test ever administered to A.I. systems. <\/p>\n

Humanity\u2019s Last Exam is the brainchild of Dan Hendrycks, a well-known A.I. safety researcher and director of the Center for AI Safety. (The test\u2019s original name, \u201cHumanity\u2019s Last Stand,\u201d was discarded for being overly dramatic.)<\/p>\n

\n
\n
<\/div>\n
\n
\n
\n

We are having trouble retrieving the article content.<\/p>\n

Please enable JavaScript in your browser settings.<\/p>\n<\/div>\n<\/div>\n

\n
\n

Thank you for your patience while we verify access. If you are in Reader mode please exit and\u00a0log into<\/a>\u00a0your Times account, or\u00a0subscribe<\/a>\u00a0for all of The Times.<\/p>\n

\n<\/div>\n
\n
\n

Thank you for your patience while we verify access.<\/p>\n

Already a subscriber?\u00a0Log in<\/a>.<\/p>\n

Want all of The Times?\u00a0Subscribe<\/a>.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

If you\u2019re looking for a new reason to be nervous about artificial intelligence, try this: Some of the smartest humans in the world are struggling to create tests that A.I. systems can\u2019t pass.For years, A.I. systems were measured by giving new models a variety of standardized benchmark tests. Many of these tests consisted of challenging, S.A.T.-caliber problems in areas like math, science and logic. Comparing the models\u2019 scores over time served as a rough measure of A.I. progress.But A.I. systems eventually got too good at those tests, so new, harder tests were created \u2014 often with the types of questions graduate students might encounter on their exams.Those tests aren\u2019t in good shape, either. New models from companies like OpenAI, Google and Anthropic have been getting high scores on many Ph.D.-level challenges, limiting those tests\u2019 usefulness and leading to a chilling question: Are A.I. systems getting too smart for us to measure?This week, researchers at the Center for AI Safety and Scale AI are releasing a possible answer to that question: A new evaluation, called \u201cHumanity\u2019s Last Exam,\u201d that they claim is the hardest test ever administered to A.I. systems. Humanity\u2019s Last Exam is the brainchild of Dan Hendrycks, a well-known A.I. safety researcher and director of the Center for AI Safety. (The test\u2019s original name, \u201cHumanity\u2019s Last Stand,\u201d was discarded for being overly dramatic.)We are having trouble retrieving the article content.Please enable JavaScript in your browser settings.Thank you for your patience while we verify access. If you are in Reader mode please exit and\u00a0log into\u00a0your Times account, or\u00a0subscribe\u00a0for all of The Times.Thank you for your patience while we verify access.Already a subscriber?\u00a0Log in.Want all of The Times?\u00a0Subscribe.<\/p>\n","protected":false},"author":1,"featured_media":20656,"comment_status":"close","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-20654","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/medexperts.pro\/index.php?rest_route=\/wp\/v2\/posts\/20654","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/medexperts.pro\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/medexperts.pro\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/medexperts.pro\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/medexperts.pro\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=20654"}],"version-history":[{"count":2,"href":"https:\/\/medexperts.pro\/index.php?rest_route=\/wp\/v2\/posts\/20654\/revisions"}],"predecessor-version":[{"id":20657,"href":"https:\/\/medexperts.pro\/index.php?rest_route=\/wp\/v2\/posts\/20654\/revisions\/20657"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/medexperts.pro\/index.php?rest_route=\/wp\/v2\/media\/20656"}],"wp:attachment":[{"href":"https:\/\/medexperts.pro\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=20654"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/medexperts.pro\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=20654"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/medexperts.pro\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=20654"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}