AI versus human beings.
Barnabas62
Shipmate, Host Emeritus
in Purgatory
This thread is a spin-off from The Styx plus some personal research into chess engines.
For some time now chess engines have been vastly superior to even the very best human chess players. Self-learning plus immensely superior calculation capability means that even the best player in the world has zero chance of winning or even drawing against the current generation of chess engines.
Chess games represent a closed though very complex challenge. But I wonder if they foretell the future for the solution of all complex human challenges.
In short, I’m not sure whether “closed” or “open” will represent real distinctions in the immediate future.
Are we creating artificial capabilities which will inevitably end up being better than us at all manner of problem solving?
For some time now chess engines have been vastly superior to even the very best human chess players. Self-learning plus immensely superior calculation capability means that even the best player in the world has zero chance of winning or even drawing against the current generation of chess engines.
Chess games represent a closed though very complex challenge. But I wonder if they foretell the future for the solution of all complex human challenges.
In short, I’m not sure whether “closed” or “open” will represent real distinctions in the immediate future.
Are we creating artificial capabilities which will inevitably end up being better than us at all manner of problem solving?
Comments
Question: Are we creating artificial capabilities which will inevitably end up being better than us at well-bounded problem solving?
Answer: Somewhere between possibly and probably.
The chess example is interesting. 60 years ago, in the infancy of computing, chess was generally thought of as too complex for computers to play well.
I was county standard, certainly well able to beat the first programs available commercially. But not any more. Chess Engines like Stockfish 18 have a rating of over 4,000, compared with the highest human player (Magnus Carlsen) at over 2800. They play games against one another with combinations of astounding complexity and beauty, in human terms.
@pease do you know if there is any theoretical research into the kinds of open-ended challenges which can be put to IT AI engines? I appreciate also that there are ethical boundaries.
It occurs to me that a question such as “What are the most promising areas of research into finding cures for human cancers” might require feeding very large amounts of data into the AI engines. And some of that data might be regarded as commercially confidential. It strikes me as a very useful question.
I think Turing/Gödel style proofs relate to formal systems, whereas what humans mean by "problem" encompasses a much wider and woollier range of possibilities.
Yes. Wider and woolier indeed. And your analysis of board games like chess is spot on. Yet ….
I’m uncomfortable with restrictive extrapolations. I’m not entirely sure why. A feeling that well bounded and unbounded may be edges of a spectrum. Also that there may also be a spectrum defined at its edges by calculation/learning capabilities and what we call self-awareness.
I’m not sure I’ve explained that very well and I hope you catch my drift. A self-aware AI engine seems possible and in the process illuminate what we mean by self-awareness.
Or maybe I’m just being wild and woolly. I’d feel more comfortable if Asimov’s laws of robotics could somehow be programmed in!
It should be possible to program the three laws into an AI engine: Asimov envisaged them as an ethical basis for a robot, provided the robot would not “go rogue”. Maybe a triumvirate of robots would be needed to prevent harm to humans?
The problem would be that a genuine AI would probably want to work around the laws. (A number of Asimov's stories turn on robots that can do so.)
From the thread title what I’m also getting at is that the chess superiority of current chess engines is essentially harmless - except possibly to some egos.
But it doesn’t follow outside the board games arena that harmlessness applies.
Leaving self-awareness of one side awaiting future developments, there seems to be a growing awareness that AI capabilities in the office environment are destroying entry level jobs for young people entering the work place. That’s not socially harmless. Nor do I see it as preventable given present business competitive environments. But the trend of replacing human jobs with AI capabilities seems likely to continue.
I suppose artisan jobs and a lot of service industry jobs are safer but I’m not even sure about that.
They are not really, outside a few niches. What is happening is that large parts of the economy are either flat or in decline (or being milked by PE), the only difference between the US and the rest of the Anglo world is that the US stockmarket is being bouyed up by AI spending.
Here’s a quote from the Reuters report.
@chrisstiles if course you’re right about the flat economy. That must also be having an impact. But there is a significant AI impact.
There are of course some new jobs being created and some evidence of folks moving to jobs where required skills are not (yet) easily replaced by AI.
There is no evidence of that in the report. There are some claims that the total number of jobs available is lower than it would have been if not for AI, but there's no attempt to quantify this. Instead there are vague blanket statements like the following:
"due in part to increased use of artificial intelligence and other forms of automation, a survey showed on Wednesday."
Further, for nearly a year now there have been a series of stories in the business press regarding internal AI investment not showing any returns, e.g
https://fortune.com/2026/08/22/executives-ai-productivity-layoffs-study/
Or
https://www.ft.com/content/b4b60d00-2e8c-4db0-b3ed-9988dc0eeb5c
or this report from MIT:
https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf
But of course framing your layoffs in terms of 'AI investment' shows you are a forward thinking CEO, as opposed to the captain of an enterprise that is circling the plughole.
"I have argued in the past that people are overestimating the organisational level benefits of AI because they are extrapolating from individual experiences, and speeding up production behind a bottleneck doesn’t increase output (although it might reduce it). But one thing I haven’t emphasised enough is that bottlenecks are not natural obstacles – they are, in most cases, the consequence of increasing production until you hit a bottleneck. If AI removes a bunch of bottlenecks, that won’t be used to produce the same output faster and cheaper, it will be used to produce a lot more output until a new bottleneck is reached and requires human intervention. (Weirdly, there was a two week period after the announcement of DeepSeek when all the techbros were wailing at their share prices and shouting “its Jevons Paradox you idiots”, but this got really quickly forgotten)."
via https://backofmind.substack.com/p/finally-we-have-created-the-silver
This is even before accounting for the amount of 'workslop' that AI enables. I'm sure I'm not the only person here who has received borderline incoherent documents from colleagues or suppliers who have drunk the koolaid.
I am persuaded that they are not.
I think artificial self-awareness is even more unlikely that artificial consciousness. For me, the idea that self-awareness can be achieved by combining computational complexity and masses of data ignores the millions of years of bodily experience that were involved in the evolution of human self-awareness, along with the evolution of the ability of individual human beings to develop self-awareness as we grow up.
No, that's undoubtedly going to happen, both because there is AI that makes it easier to fake people's voices, and because generative AI can generate superficially plausible text at scale.
The generally degraded information environment won't help either.
Re extrapolation. The increasing analytical capability of AI engines to both learn and solve problems. Can we at this stage say that we know what the limits of that increasing capability might be? I’m not sure we can.
Re self awareness. Are you sure that the read across from the millions of years’ journey to human self awareness is appropriate to growing AI capability?
Following up on your very helpful posts, I have been looking for additional serious sources of information and came across this.
From the Office of National Statistics..
It suggests amongst other things that the accurate measurement of business impacts is itself a work in progress. Also that the impact on employment has been generally marginal apart from some niche sectors (as you said).
I also noted from some other links to consultancies the following expected line.
“You may need our expert consultancy if you want to maximise the advantages of AI implementation in your business”.
This strongly supports the suggestion that, so far at least, implementation of AI has not delivered general P & L benefits. I liked the argument from one of your links about moving bottlenecks without overall improvement. That’s certainly something I saw re implementing process changes in the work place.
Producers of AI capabilities continue to make very positive noises about further significant improvements in those capabilities in the work place. Well, they would.
I guess it’s early days to have a clear view about the long term impact of AI, and not just on employment.
A couple of examples:
“Liar” (1941!) shows what happens when you have an articulate robot that gathers data about you and, in order to please you, steers your thoughts further in the direction in which they were going, leading you to take unwise action which you falsely believe is completely logical. This is not a sentient robot. It is the stage which AI in common use has already reached. This is a fairly short story, included in “I Robot” and “The Complete Robot”.
“The Bicentennial Man” (1976) is pretty much a sentient robot with a wish to become as human as possible (it concludes that it must become mortal, interestingly).
They are, but their valuations and investments are based on the idea that GDP ends up being 3-4x what it is currently (that's GDP growth of 300-400% -- note that historically even completely transformative technologies have added around 1% to growth per year).
Unless there's a sharp discontinuity in history the likes of which we've never seen before most of those investors are going to lose their shirts.
The product we receive on our screens is one thing, but the process of getting that information is mind blowing.
I'm afraid I find that a disingenuous question. What's the justification for thinking that it's computational complexity that's the key to self-awareness and that other aspects of human evolutionary development can be ignored? To my mind, any "reading across" comes from the untested, unproven assumption that self-awareness is a stand-alone quality.
Meanwhile, I think most of the rest of your post was in response to points that were raised by chrisstiles.
Re specialist and generalist. That’s a fair point for now. I don’t know enough about how the software coding works both for specific problem solving and learning. For all I know there may already be a category of transferable problem solving capability. Or it may be round the corner.
Re self-awareness. Why do you think I’m asking a disingenuous question? It was just a question that occurred to me. Thinking about why, I find my prior assumption is that self awareness is a an emergent consequence of a highly complex neural network. What has clearly emerged via evolution is increasingly complex brain development. The human brain is a highly complex neural network which happens to sit in a particular organic body which provides input to it and also responds to it.
I think that’s a fairly common understanding but I may be wrong.
In terms of AI therefore, AI engines have different input and output connections to those which have evolved in the human body. So I’m not sure that the evolutionary argument applies in the same way.
Of course there are other ways of looking at the seat of conscious awareness in human beings. Maybe @chrisstyles was thinking of those? I don’t know.
I'm not sure what you are referring to here -- I didn't speak to consciousness at all.
A key plot point in Liar is that the robot is telepathic. I think the most significant contribution of Liar is to introduce the word "robotics" to the English language (according to the Oxford English Dictionary).
I perceive a pervasive underlying assumption that AI is somehow behaving like a developing human being. It isn't. Not in the slightest.
And I find much of what is written and said about "Artificial Intelligence", including the words themselves, to be disingenuous rhetoric.
Currently, the biggest proof or argument we have regarding the future capability of AI is the amount of money that is being invested in it. On Thursday (a couple of days ago), Nvidia's market value increased by $442 billion, following their second-quarter earnings report (which included favourable revenue guidance). This was the second-largest one-day gain in history (after Microsoft, a few weeks ago). Nvidia is the most highly-valued company in history.
The human brain doesn't "happen" to sit in an organic body - it is part of that body. The links between the rest of the body and the brain are I think much more extensive and complicated than the links between a group of computing chips and whatever device the chips are sitting in.
I think the most significant way this makes a difference is that the brain has evolved out of systems that adapt an organism's behaviour in moving around an environment, satisfying the organism's needs, and avoiding threats. Our self-awareness and consciousness is that of a body in an environment, not that of a brain that happens to be in a body.
Certainly the market believes that “Artificual Intelligence” (I agree with your view that it is a misleading label) will eventually turn out to be profitable for those who invest intelligently in its capabilities.
Currently the specialist IT capabilities are not the same as human general-purpose intelligence. And currently they may give the misleading impression that they are intelligent, because of their problem-solving capabilities. “They walk like ducks and quack like ducks but they are not ducks.”
However (!) I’m not sure things will stay like that. How far we are away from producing software plus machines with more general purpose problem-solving capabilities I’m not sure. At any rate it seems likely that such machines would need a complex neural network in order to process and learn. To what extent they may be conscious or indeed self-aware remains to be seen.
Taking up your marketing point, I’m pretty sure there is already a potential market for such complex machines. “If they walk like a duck and quack like a duck it won’t really matter whether they are ducks or not.”
I hope my metaphor doesn’t confuse the discussion.
@chrisstiles
I’ll clarify later. The issue of whether consciousness and self awareness can be separated from the evolution of living creatures is fascinating and has been discussed a lot. It needs a more careful post than the one I produced. Sorry about that.
Even within the domains they have been applied to these systems aren't really 'learning'. generally 'machine learning' so called is - at the commercial level - a large amount of p-hacking and HARKing.
Even the FT is sounding caution here, they had an analysis of the finances of the hyperscalers a few months back that showed that only Amazon was likely to turn a (fairly modest) profit on the investments they had made (and significantly, they were the only one who wasn't trying to also build their own model).
That's probably changed since their latest investment announcements though.
I suppose the Nvidia massive market value increase (@pease quoted) is a sign of optimism? But maybe it just shows how successful they have been in selling future hope?
Does that follow, or does the market "believe" that the stock price is going up and therefore there is money to be made. Tech stocks have a tendency to have valuations decoupled from their ability to pay dividends. There also appears to be some evidence that a big chunk of AI "investment" is actually rather incestuous movement of cash piles from hardware firms to AI developers back to hardware firms. There might be a future in which AI is a profitable business, but I think we're going to have at least one "dot com bubble" moment before then.
Speculation mania? Maybe. That’s the trouble with future hope. No doubt we’ll all watch with interest.
Anyway. If it turns out to be BS hype AI won’t threaten the job market. unlike the way a flat economy will. (I suppose an AI bubble could precipitate a bear market? )
Yes, the unwinding will cause a huge amount of economic harm, not least because of the amount of misallocated real resources that would have been wasted on the way up.
It could and would threaten the job market, long before we realize that it's a bubble / other misleading pile of never-mind.
One way this would happen is by people being fired as "replaced by AI," which I understand has already happened at at least one major tech company--and if/when it becomes clear that this was a mistake, still, those people aren't going to get their jobs back again. They will have already scattered to the four winds, trying to find something, anything, to keep them going financially. And rebuilding what was destroyed will also take a huge amount of time and effort. It's almost always easier and faster to destroy than to rebuild--and that's leaving out all the time for leadership to faff around saying, "Should we? Shouldn't we?" and having arguments about it, and whose fault it was.
Yeah, there's quite a bit of this around, usually the service gets generally worse and it disappears into general enshittification.
The prospectus of SpaceX, OpenAI and Anthropic are all based around the idea that they'll have revenues of $30 trillion each, total revenue of the S&P 500 is currently $20 trillion.
Indeed, particularly because when they fire people, they don't bother to ask who's carrying the institutional knowledge right out the door with them--and if/when they try to rebuild what they destroyed, those people are elsewhere, looking on with bitter satisfaction and saying "I told you so." (Ask me now I know...)
https://www.penguin.co.uk/books/462159/we-are-not-machines-by-oconnor-sarah/9780241704226
There seems to me to be a reduction in the kinds of entry jobs that young people and/or graduates would have had in previous generations. For example I was only yesterday hearing about a lab that is planning to pivot to much more mechanisation and robotics. So a technician in the past might have been able to do x repetitive tasks in a day and the plan is for a "robot" to do far more with no breaks.
Strictly speaking this example us not about AI/LLMs but it seems to me that many routine tasks are being parcelled out to some kind of LLM which is reducing human employment.
The second aspect of this I heard about in New Zealand. I cannot speak to the absolute numbers, but I heard the anecdote several times; young people cannot get the "graduate" positions in the NZ economy so after university leave the country if the have no family connections to get a job. They then get the years of experience in Australia, Europe or wherever and if they are still wanting to move back to NZ they do so as more experienced and seasoned employees.
My belief is that this has not been a common feature of life in the UK, particularly since Brexit. It happens abs has always happened a bit, but we have not had migration as a normal part of national life to get experience. The complicated new feature is that the graduate jobs are collapsing everywhere all at once.
I also reflect that if a large number of jobs can be swept away in this way then it probably supports David Graeber's contention that many are actually Bullshit.
Putting it another way, AI has entered the arms race in a big way, and national security more generally. In that regard, the money going into AI isn't just about the financial return on investment.
A related aspect are stories about AI agents "breaking out" out of their sandboxes (which happen to be connected to the internet, for some reason) and pursuing their goals more creatively and aggressively and rule-bendingly than envisaged. These stories do not harm narratives about AI and national security.
Meanwhile, as far as capitalism is concerned, we *are* machines. In the Guardian's review, Mythili Rao takes a Guardian-friendly view: We have always shared our work with machines, ever since we first became tool-users. ("Tool", "machine" and "technology" are all words for “a system or device that performs or assists in the performance of a human task”.)
And from the Penguin blurb: Tools and technology change (shape) the people that use them - widespread use shapes society. Most of the ways are unseen and unplanned. Thinking back to Hannah Arendt's book The Human Condition, I suspect it's the distinction between labour (labor) and work that's more significant in relation to our humanity.
That's has been more supply side than demand side, with technology companies (not just AI specific ones, there was spacex and the hyperscalers before that) seeking defence/government contracts because they are lucrative, large and somewhat immune to the wider economy.
On the theme of graduate/new start jobs; the US economy has largely been kept afloat on the back of AI related investment, meanwhile the UK economy has been mostly flat, with annualised GDP rarely hitting 1%. Both those environments make for an economy in which major sectors are flat or shrinking year on year. A graduate program is generally funded centrally as a cost to the entire business, and once it's closed it's a really easy decision to postpone its re-opening for yet another year (meanwhile divisions/programs will look for people with 1 or 2 years experience because they can't bear the individualised cost of training someone up).
Yes, I heard her chatting about this on a podcast. She was amused by it.
My father and his father nearly 50 years before him were products of the traditional indentured apprentice system, which is interesting for the purposes of this discussion because it is quite hard to see what benefit there was to the companies involved. My father spent four years working through various departments at a company that no longer exists; when he had finished he was working in a range of other companies. There may have been some expectation of a few years of continued employment, I cannot now ask him. I do know that for most of the rest of his working life he worked elsewhere.
In 2026 that looks very strange.
I think the indenture itself (a piece of paper that is framed and on the wall of many homes in that era) only covered the period of the apprenticeship. I do not think it covered time after the apprenticeship was finished, that was more to do with social convention. I think that is correct.