Originally revealed in Strange Loop Canon, April 23, 2024
On aim drift and decrease reliability. Or, why can’t LLMs play Conway’s Game Of Life?
Every time over the previous few years that we got here up with issues LLMs can’t do, they handed them with flying colors. But at the same time as they handed them with flying colors, they nonetheless can’t reply questions that appear easy, and it’s unclear why.
And so, over the previous few weeks I’ve been obsessed by attempting to determine the failure modes of LLMs. This began off as an exploration of what I discovered. It is admittedly a little bit wonky however I feel it’s fascinating. The failures of AI can train us much more about what it can do than the successes.
The start line was larger, the need for job by job evaluations for lots of the roles that LLMs will finally find yourself doing. But then I began asking myself how can we determine the bounds of its means to purpose in order that we can belief its means to study.
LLMs are arduous to, as I’ve written a number of occasions, and their means to purpose is troublesome to separate from what they’re skilled on. So I wished to discover a method to take a look at its means to iteratively purpose and reply questions.
To proceed studying this text, click on right here.
https://www.predictiveanalyticsworld.com/machinelearningtimes/what-can-llms-never-do/13507/
