> Demis Hassabis, the CEO of Google’s DeepMind, said the goal is to create “models that are able to understand the world around us.”
> These statements betray a conceptual error: Large language models do not, cannot, and will not “understand” anything at all.
While I find lots to agree with in TFA, this pedantry over "understanding" irritates.
While the "understanding" of LLMs is (I believe) a very different affair from human understanding, I wonder if the author can find a better word to succinctly convey what Hassabis is saying?
The problem here is not word choice but failure to recognise that the meaning of a word has changed when the context of its use changes. If I say "it thinks the db is down because it doesn't know the cache is out of date", no-one (or at least very very few!) would take issue with "thinks" and "knows" on the basis that a simple computer program doesn't have human-level knowledge and thought. The words serve to convey meaning, and the context is clear.
If the author doesn't recognise this typical feature of language, extending the meaning of existing words when they are used in new contexts, the "conceptual error" is on them.
Yet another example of failing to grasp the computer vs program paradigm.
For contemporary AI the “statistical engine” for next token generation is the VM. The vast n-dimensional vector matrix of weights is the software.
The software is -not- a token prediction algorithm. It is a machine interpretable compilation of human culture.
The prediction algorithm is analogous to the CPU. The weights are analogous to the software. A cpu does not exhibit the capabilities of software, it enables software to exhibit its capabilities.
> Demis Hassabis, the CEO of Google’s DeepMind, said the goal is to create “models that are able to understand the world around us.”
> These statements betray a conceptual error: Large language models do not, cannot, and will not “understand” anything at all.
While I find lots to agree with in TFA, this pedantry over "understanding" irritates.
While the "understanding" of LLMs is (I believe) a very different affair from human understanding, I wonder if the author can find a better word to succinctly convey what Hassabis is saying?
The problem here is not word choice but failure to recognise that the meaning of a word has changed when the context of its use changes. If I say "it thinks the db is down because it doesn't know the cache is out of date", no-one (or at least very very few!) would take issue with "thinks" and "knows" on the basis that a simple computer program doesn't have human-level knowledge and thought. The words serve to convey meaning, and the context is clear.
If the author doesn't recognise this typical feature of language, extending the meaning of existing words when they are used in new contexts, the "conceptual error" is on them.
Yet another example of failing to grasp the computer vs program paradigm.
For contemporary AI the “statistical engine” for next token generation is the VM. The vast n-dimensional vector matrix of weights is the software.
The software is -not- a token prediction algorithm. It is a machine interpretable compilation of human culture.
The prediction algorithm is analogous to the CPU. The weights are analogous to the software. A cpu does not exhibit the capabilities of software, it enables software to exhibit its capabilities.