Thirith on 28/4/2023 at 10:49
With Eliza, the cracks in the illusion usually showed within one or two exchanges. It was never particularly convincing, unless you fabricated an example playing to its limited strength. No comparison to what you can get out of ChatGPT, at least in terms of surface convincingness.
Cipheron on 28/4/2023 at 12:40
I think the point being made was about people seeing more than is actually there, and how this goes back to the very earliest and most primitive chatbot ever made.
It's not even sentience, people wrongly ascribe some "decision making" process to many chatbots, i.e. there's some algorithm in there that's using logic to decide what to write.
When in fact it's actually a deceptively simple "word salad machine" that's just been trained to fool another computer program as to whether the word salad is fake word salad or from real texts. The thing is, neither the program generating the word salad, nor the training program that assesses the output understand English or the topics it's being trained to produce texts about.
Starker on 28/4/2023 at 15:15
Exactly. I remember in particular the anecdote where the secretary of Eliza's author asked him to leave the room so that she could have a real conversation with Eliza. Maybe it's because the answers ChatGPT generates are remarkably human-like, but people can't seem to help but anthropomorphise it or at the very least think it has a brain of "some sort". Like, saying that it can "come up" with things or "create" something. And I'm not saying it to disparage anyone, as I myself have been compelled to do the very same, especially before I read up on how it actually works.
demagogue on 28/4/2023 at 18:21
Just to put on my old philosophy of language cap for a bit, while it's not directly simulating the brain, it's effectively recreating the same representations the brain would have to use to get the same outputs in the same context. So I think it's still fair to say it's internalizing human ways of thinking, not in the way humans do it, but in its own way.
But it's not able to recreate everything. But I think that's mostly just because it didn't have access to other modalities of experience, like vision, audition, or all the emotional and affective inputs that human brains trigger instinctively, not in the training and not in the chats. But if it did have those inputs, and probably you'd need a lot more density in the hidden units to handle that, then I think it'd internalize how the brain represents those things as well, at least to the extent that inputs match the outputs in the way training structured it, no more and no less.
I think the argument I'm handwaving at is something like, while I agree it'd be wrong to over-anthropomorphize ChatGPT, I think there's a problem with under-anthropomorphizing what it's actually internalizing from human behavior in its hidden units also. It's very different from Eliza & GOFAI (good old fashioned AI), which didn't represent the functionality of "meanings" at all, just brute-force stim/responsed in "blind" data arrays. Hidden units aren't just blind data arrays; functional representations are being built in the hidden units. They're opaque and we can't see them in there, but that doesn't mean they aren't there.
That's the thumbnail version. I think I'd have to spend a lot of time unpacking what I think I mean by that to give it justice though.
Starker on 28/4/2023 at 19:20
I don't really know how we use language on the brain level, but I know we use it extremely creatively. We are constantly playing around with it in the Wittgensteinian sense of language-game. As such, any meaning is infinitely malleable and depends on the current game being played. And not only that, we even change the rules mid-game and invent whole new ones as we are playing it.
We certainly don't go, "Based on the accumulated texts I have read, this is statistically the most plausible word I should use in the sentence I've been saying so far. Now, let's run a lot more probabilities and check which one of these different end results best corresponds to the user prompt according to my training data and gets the best reward from the reward model."
Because it's the last bit that gives GhatGPT the most human-like appearance, that the output has been carefully selected to appease human trainers and weighted towards the most plausible-sounding ones.
demagogue on 28/4/2023 at 21:36
Re: [[We certainly don't go, "Based on the accumulated texts I have read, this is statistically the most plausible word I should use in the sentence I've been saying so far.]]
We certainly do exactly that on the first pass (well not exactly but virtually), which actually gets put out when we're not paying attention or are distracted, etc. (A: "Hey, what's going on?" B: "Fine, and you?"). When we are paying attention, there's a check on the statistically-likely-reply that's first put forward against the context & our goals, etc. That also happens with what we hear other people saying to us, cf. the famous N400 event in EEGs (a negative polarization bump 400 ms after you hear a thing) when a person's comment diverges from what we expected them to say in the statistically-likely-model that first gets pushed out within the first 100~200ms.
But I wasn't even talking about that. What I was talking about is that functional features may be getting built into the hidden layers via the way back-training works on the feed-forward models. It's like when you train a model to identify faces, operations necessary to discriminate faces like discriminating sub-features like lines and curves actually get built into the model. Or representations of the sub-features are getting burned into the model, so the vanilla ML algorithm "operates" on them. It's like the way natural selection selects fit genes inside DNA code that "operated on" give you fit phenotypes, back-training selects weights in feed-forward ML hidden layer code that "operated on" gives you cognition. Yes it's still a statistical model, but it's effectively simulating operations of line and curve discrimination inside the guts of that model.
Or back up; you're saying "it's just computing statistics blindly". Literally it's a gradient descent algorithm to a statistical nadir (a peak flipped on its head). But if the path of the descent itself effectively simulates the mechanics of cognition, then that "blind path" also happens (not coincidentally) to "operate the mechanics of cognition" in a gradient descent packaging that statistically lands on the same statistically likely peak/response because, after a billion training runs blindly carve at the geometry, that geometry is going to always converge on to the only "statistical" path that's going to consistently get you to the right peak, and that's going to be by actually baking a simulation of the cognition required to get you there into the path geometry itself.
You're actually building algorithms of cognition into the geometry of the model. That's the argument. So "just computing the statistics" is also simulating something like what the brain does, at least the parts that link inputs to outputs. But you're not seeing it because you're not looking at it as actual operations being done by the gradient descent path as it winds its way around. You're just seeing a gradient descent to a nadir and calling that "statistics", a blind road that happens to statistically land on the right answer. (I mentioned lots of little things that aren't getting baked in because there aren't enough cues in the outputs to bake them in. But if they're not making a difference to the output at all, you'd have to look at what role they're playing at all. That's a worthwhile thing to follow-up on, but that's another discussion.)
Since it is so opaque what's happening in the gradient descent space (it's this vast dimensional space we usually just bracket and dismiss as random meandering), there's room for people to argue what's actually happening down there. I think there are a lot of smart people that would disagree with this take, so I don't want to say it as if it's some consensus position. For the purposes of this post I'll just say this is one way that some people look at it that, to the extent it's on to something, gives a view where ChatGPT has more human-like thought inside it than people are giving it credit for.
And the thing about that argument is, the argument isn't really that baking statistics into the geometry of a gradient descent that ChatGPT does can simulate the cognition like a brain does. The real argument is that the algorithms of cognition like the brain does are made by baking them into the geometry of gradient-descent-like operations that the brain's neural networks are doing that are a lot more like what ChatGPT is doing than people think. There are still big differences, I still agree with that; and to get at some of them I'd have to talk about, e.g., Grossbergian neuroscience (e.g., the models are dynamic in the brain whereas they're static in ML/ChatGPT, etc.), but the differences aren't the kind I think people are thinking about when they say ChatGPT is nothing like the brain.
All of this is again just an idea I'm throwing out there as food for thought.
---
Edit: I'm not fully responding to your response though. ChatGPT doesn't answer like it's playing a Wittgenstenian language game because it's often not playing a language game with its responses the way it's set up, at least not as much as humans do. (Some of that also gets baked into its model whatever query it's responding to, I think.) But I think that's more to do with how it "understands" its role in giving responses to queries than the structure of the model itself. It probably would be even more human-like if it was more set up to take on the role of "acting" or "playing a game" in its responses than it does now. I mean, I think that's not as fundamental a change as one might think, like the difference between a bot that explains what a game bot would do in a map vs. an actual game bot playing in a map.
Cipheron on 29/4/2023 at 07:38
I think it's more deeply different than that.
1. When humans write there's more going on, in between each word. With ChatGPT there just *isn't*.
This explains examples where ChatGPT was asked to add up a bunch of numbers such as "13+63+22+144+55+37" and gives the wrong answer. Each time you regenerate the result, it would give a close, but effectively random answer.
It's asked to give the result of that operation, so it just pushes the whole thing into it's statistical-analyzer, and that gives it a probability cloud of possible next tokens. It then picks one of the tokens randomly. This in no way resembles how a reasonable human would approach the problem.
However if you give it the same task, but tell it to add them up one at a time, then it would output the partial sums, and give the RIGHT answer.
Now, OpenAI patched this *particular* exploit, probably by hard-wiring in something that would detect sums and manually insert the correct answer, but it effectively shows that, inbetween words, ChatGPT isn't actually capable of doing logic that takes more than a single step.
2. Humans are able to "look ahead" more than one word when thinking about which word to use. ChatGPT basically is incapable of doing this, at all. So it lacks ANY sort of overall directed or goal-driven stuff to dictate what it decides to write. The very next word is purely dictated by the previous words plus some dice-rolls.
3. The human brain has "state" other than the words it just wrote. ChatGPT doesn't have this, it only has the words it just wrote: that's it's entire memory.
These three things are all connected. Lack of any state or higher functions explains the attempt to just assign results of math calculations at random, and the lack of "look ahead" ability when planning out a sentence.
So, saying the human brain is also a neural network like ChatGPT. I think that's not really a helpful way to understand how this works and the relationship to a real person trying to write something.
Cipheron on 9/5/2023 at 14:59
Just got back onto the ChatGPT subreddit. Immediately hit by the sheer amount of people not understanding how AI works.
The latest person is upset because he asked Bing AI to summarize some videos and it didn't get it right. Like, they expect GPT to watch videos now, and understand what's going on?
Seriously, these people really do think it's like the Oracle of Delphi.
demagogue on 9/5/2023 at 17:37
A few points I want to make.
1. About what I was arguing above, I'm not contesting any of the criticism Ciph gave to it. There is obviously a ton of capacities still missing from ChatGPT. I was trying to make a weaker philosophical claim, something like that "mental representations" tend to converge, which is just a claim or hypothesis I'm making based on stuff I've read. It's like how eyesight keeps independently evolving in a lot of different species under natural selection pressure; I think a mental representation in an LLM can start to converge to the complementary representation in the brain with feedback weighting. That's still very far from saying LLM's cognition is "like" humans; maybe at most that it's more a difference of degree (orders & orders of magnitude) and application (not combining vision & language, no memory, etc.) than type per se. So I don't think I'd deny anything about the response to what I posted. But to push back a little maybe, there are humans that are also missing vision and memory, and we can say they aren't neurotypical, but we wouldn't say what they think "the cat ran across the yard" means is too fundamentally different. It's just that the way LLMs like ChatGPT are designed right now, they don't have the resources available to actually act on that meaning, like bona fide free action, where they can actually imagine the cat and the yard, have some emotion or motivation based on it, and take some actions based on that motivation. I think the core of the representations are in there, but they haven't figured out how to use them like humans do. That's just my feeling about where things are now or will be soonish.
2. Regarding what was just posted above, I think the next step is going to be chaining AI. We saw it with that monk video which chained I think three different systems together--converting player voice to text, feeding the text to ChatGPT with appropriate pre-instructions, and the script or whatever in the game that parses the ChatGPT text and converts it to the speech and animations of the NPC. I think the people designing Bing AI and the like are going to starting chaining AI like that to start addressing questions like the above, because it's natural people will think of AI as doing those kinds of tasks, and they'll design the systems to the demand.
3. The trend I've really been noticing is how many videos are AI scripted and voiced, and how apparently aggressively these kinds of videos are being pushed on to me in different platforms. The way the commenters take it so for granted makes me think that the trend here is that our world is about to become universally and ubiquitously curated and commentated on by AI, where things happening just get automatically processed in this endless stream of AI updates constantly linking us to "the world" and giving us calculated opinions and questions that get people automatically engaged. It's something along those lines. I don't like the trend. I feel like there should be some resistance to this constant AI curation popping up, like some push for "authenticity", but I'm not sure there is or will be. I guess we're starting to see that in the visual arts and maybe music where people are taking sides, but I feel like even those are exceptions that prove the rule and are going to be submerged by the flood of AI content coming.
