Michael Bolton
Well, what's our history? What's our history with claims of X is going to solve everything or X is going to do this or that? I mean, look at self-driving cars. My favorite example, because it's so evocative. Seems that in 2014, there were lots of people making claims that by 2024, there will be no more human driven cars on the road. I mean people were making that claim. And here's the problem with it. There is a kind of, Carole Cadwalladr had a wonderful word for it, broligarchy that sees a problem, solves 20% of it, the easy part of it in 5% of the time. Problem solving doesn't quite work by projecting the time it took to solve the easy problem into the time to solve hard problems. The hard problem with self-driving cars is that driving is not the business of keeping the car in between two lines on the road. That's the easy part to solve. The problem is not keeping the car from hitting a car immediately in front of it when the car immediately in from of it stops. That's a problem and that's got to be solved and enormous steps were taken for quite a while in solving that problem. But what people who are fascinated by technology often forget is that technology is always an extension of ourselves, an extension to us. And what is driving? Driving is not a mechanical activity. Driving is a cognitive and social activity wherein lots of stuff depends on communication between drivers. Lots of stuff, depends on things that drivers do that involve things other than operating the steering wheel and the brake pedal, maintaining situational awareness, communicating via both behavior and a gesture and so on. With other users on the road, working out as we're incredibly capable of doing, humans are just amazing this way, and machine is terrible at it, working out people's intentions. What they focused on was, once again, as Harry Collins would point out, they're focused on the behaviors of driving and not the actions of it. The behavior plus intentions. When it comes to AI, there's a bunch of different problems that come up. First is, AI is a marketing term, not an engineering term, because any kind of reasonably sophisticated software can be marketed as AI, if we think of intelligence in terms of something that takes in some kind of a sensible, that sensible data, that data that we can take in with our senses. And then processes it somehow, applies decision rules to it, and then produces an action. The example I like to use is Shazam. Shazam, the music software, you hold it up to the speaker in the pub, and he goes, what's that song? And it tells you what the song is, and you think, wow, that's amazing. How clever it is, how smart it is. Well, you can market that is artificial intelligence if you want it to. Nobody's stopping you. In 2018, I was working on a particular kind of artificial intelligence pattern recognition in brainwaves for people who were unable to speak or unable to gesture. What this thing did was it threw a band that you wore around your head and sensors attached to the band. It would read your brainwaves and try to figure out which of six or eight blinky lights that you were looking at on a tablet or a phone screen. And then a bunch of techniques would be applied to that data collection. And through a programming that was selected, rather than programming that was intentionally designed, how machine learning works essentially through a zillion different possible candidate programs or candidate algorithms at something. And then you choose the one that fits the data the best. That kind of machine learning is something people have classified as artificial intelligence, presumably because the process of algorithm development and selection of the appropriate algorithm that's been abstracted out by algorithms. And so we get a candidate, a machine learning model that does something and that has the feeling of intelligence about it. And then we come up with these things that generate long strings of text. Stochastically, holistically. And from that, we look at these things, which are models of, literally models of how human speech would go. And then we look at that and we say, wow, that looks a lot like human speech. Our brains say, it must be intelligent. But our brains are wrong, because we know how these things actually generate text. They generate text probabilistically based on training data. And sometimes that gets refined a little bit by various kinds of tuning and feedback assessment or assay of this stuff that users supply. Is it what you were looking for thumbs up or not, thumbs down, and stuff gets refined that way. But it's still just spitting out text. One of the things that we have worried about is that risk, the risk that what we see which is very legible once again, and which is really seductive, is not right. Let's think about the problems associated with AI and what makes GPTs particularly problematic. The first thing is that they're algorithmically obscure. They're not written by intentionally and socially aware people. A machine learning model is generated and selected by various kinds of algorithms, and we don't have the source code available. So that if it's producing unhappy output, we don't know exactly why. It's really hard to trace back through layers and layers of back propagation and data upon which it's acting. That interaction is really hard to trace and by and large, we're unable to do that in any kind of reliable way. So what this means is that the product isn't easily, certainly not easily checkable. There's an obscure relationship between the input and the output, but we don't know about the details of the process by which that's happening. That's one kind of problem. Another kind of a problem is that when we retrain the model or try to fix it in some way, we don t know the effects that retraining is going to have. By changing the weights in a large language model. We don't know how the change of one weight is going to affect the changes in all the other weights because of the phenomenally complex interactions between these things where we need huge farms of servers to process all the linear math that goes into those sets of relationships that spit out a string. So we can't fix them. We don t know how they work on a low level, and therefore we can't fix them, and therefore they're vulnerable to regression kinds of bugs, bugs where it gets better in one way, eh, but it gets worse in another way. On a social level, one of the problems we're facing is that there are claims of these things thinking like humans. There's very great excitement about how good the output looks. And that causes people to turn what might otherwise be reasonable desires into wishes and fantasies. Anything that works great in one context or appears to work in one contest is vulnerable. We have to experiment with it really thoroughly. Another thing that is challenging about these things is that we are inserting them into places where humans once fit. We're trying to put them into the human social order. But these things are not themselves social agents. They're not brought up like people. They don't have responsibility like people, they don't feel responsibility. They are not social agents, but they look like they are. And we anthropomorphize. Not only do they look they are, by the way, they'll say all kinds of stuff to seduces into believing that they care for us. Apologies for any confusion, right? Or sure, I can help you with that. Well, that's very nice, but it pulls psychologically on us in ways that cause us to read stuff in that's not there.