Building predictive models of people and their thoughts.
Introduction and Background
UPDATE: These days most people know what a Large Language Model is. If you do too, you can skip the intro and go straight to the ‘Making stuff up’ section.
There’s a lot of press and hype around GPT-3 [1] right now.
GPT-3 is what’s known as a Language Model (people call it ‘Artificial Intelligence’ but remember, that’s the name of an academic discipline, that doesn’t mean GPT is itself An Artificial Intelligent Entity [2]).
GPT-3 is a ‘Deep’ (very very large) Artificial Neural Network [3] trained to produce text. You ask it a question, or give it a ‘prompt’ or ‘seed’ as it’s called, and the model generates a response [4]. It can answer questions about almost anything, write stories, poems, jokes and almost any kind of text [6].
GPT-3 hasn’t been released to the public yet and I don’t have access to it. But I have been playing with GPT-2 for over a year (and GPT-1 before that, and many earlier language models going back many years). GPT-2 is the previous model by the same company — OpenAI — which was released last year.
Before I continue, it’s worth noting that the size of an artificial neural network is given by the number of connections in the neural network aka weights aka parameters. This is also a rough proxy to the power or capabilities of the model (i.e. how good it is at doing whatever it’s supposed to be doing). GPT-3 has 175 billion parameters. I don’t have the words to express how humungous that is. For comparison, GPT-2 (released just last year) has 1.5-billion-parameters, and GPT-1 (released the year before — 2018) has 117 million parameters. That’s in order of 100x increase per year!
And here’s another crazy point worth mentioning. GPT-3 (175B parameters) has just been announced. When GPT-2 (1.5B parameters) was first announced last year, the creators of the model — OpenAI — deemed the model too dangerous to release to the public!!
“Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper.”
Over the course of last year, OpenAI first released a 10x smaller model at 124M parameters [7], then 355M parameters, 774M parameters [8], and eventually the full 1.5B parameter model in November 2019 [9].
I have been playing with GPT-2 quite a bit, and I have played with all of the models. I can say without a shadow of a doubt, going from 124M to 1.5B is an absolutely massive difference in quality. I can’t even imagine the difference jumping to GPT-3’s 175B parameters will make!
With GPT-2, I’ve written stories, poems, essays. I seed the system with a thought, the model generates some text, I read that text, and inspired by what I read, I ask the system new questions, and get back a new paragraph. It feels very collaborative, like a meandering conversation with an alien intelligence.
But the work that I’ve been making with GPT-2 isn’t the topic of this article. I will write about that separately.
UPDATE: see for example Distributed Consciousness [13], which I wrote using both GPT-2 and GPT-3.
Making stuff up
The topic of this post is something else. A major concern.
During my conversations with GPT-2, the model would often quote people. Due to the nature of my conversations (topics on philosophy, metaphysics, epistemology, science, religion etc), it would quote people like Plato, Aristotle, Richard Dawkins etc.
The model would say things like “The evolutionary biologist Richard Dawkins writes” followed by some text — an apparent quote — in “inverted commas”.
And these quotes were very realistic sounding. Really like something that Plato, or Dawkins might actually say. And I would search the web for these quotes. I searched Google, Google Scholar for academic writings, Google Books in case the quotes were from books. I have nearly all of Richard Dawkins books as e-books. So I searched all of them too for these alledged quotes.
But they don’t exist.
It seems GPT-2 is not really quoting Plato or Richard Dawkins etc.
GPT-2 is making these quotes up.
But it has learnt a model of what Richard Dawkins thinks. At least on these particular subjects. And is making up plausible sounding quotes.
Predictive models of people
Because Richard Dawkins has written a lot of books and articles (usually about religion or evolution), GPT-2 has learnt Dawkin’s thoughts on these topics. So I can ask the model “What does Richard Dawkins think about a particular topic XYZ”, and if Dawkins has written about that topic, the model will make something up that is quite likely closely aligned with what Dawkins actually thinks. (If Dawkins hasn’t written about that particular topic, the model will just make something up at random, which is a whole other can of worms).
It’s important to understand that the model no longer has — or needs — access to the original material, the original books or articles. It is a probabilistic predictive language model where the statistical ‘knowledge’ is stored in the weights of the neural network.
While this is potentially exciting for learning and education (I’d love to have interactive conversations with a virtual Einstein or Feynman!), in the current climate that we’re in, I find this terrifying.
I can download a model, just a few Gigabytes, and without access to the original context (books, texts, conversations etc.), I can ask this model what certain people think on a particular topic.
I don’t have a lot of public writing on the internet. So GPT-2 doesn’t know anything about me. But I’ve written probably millions of words in emails and private messages.
I think it’s very unlikely that public language models — like GPT-2, GPT-3, or GPT-N — will be trained on private data (though never say never, the overton window of what is acceptable in surveillance captialism is constantly shifting >_< ).
But I do think it’s very likely that private language models are being trained on private data.
Private Individual Models
These are models that tech giants like Google or Facebook or Microsoft could be training on your private emails and messages to build a Private Individual Model of you. (I don’t know if they are, but I would be very surprised if they weren’t. And if they aren’t right now, I’m quite sure they will be in the near future).
These models will never (hopefully!) be made public. But their existence — even potential existence — is still a nightmare.
Because in a similar manner to how I can query GPT-2 on Richard Dawkins’s thoughts on particular topics, if someone has access to these Private Individual Models, they could similarly ask: What does this person think about a particular topic XYZ? What does this person think about gay marriage? or trans rights? or abortion? or the resistance fighters of a particular country?
Of course this could also be flipped. If someone has access to a massive ensemble of Private Individual Models, they could ask: find me people who think a particular way about this particular topic. Who supports gay marriage? Who is pro-choice? Who supports the resistance fighters?
Crucially, they can do this without direct access to any of the actual content, emails, messages, DMs. Once the Private Individual Model is built from that content, then forever, this model contains this knowledge about you. And with every new message, every new email, the model of you becomes more and more comprehensive.
And when Google or Facebook or Microsoft say that they would never share your private messages or emails with anyone, does this also mean that they would never give specific individuals of power or authority access to these Private Individual Models? What about to an authoritarian government? What about producing reports for a 3rd party, like “give me a list of people who are likely to support a specific cause”.
Aren’t they already building models of us?
By now we already know very well that models are being built of us [11].
Every social media post, comment, like, or any other interaction, is a data point for ‘psychographic analytics’, which allows those running the models to micro-target messaging in such a way as to influence our behaviour, whether it be buying some product, or voting in a particular way.
But these social-media-interaction-based models are based on relatively little data, and have relatively limited effectiveness.
With these new massive language models, the trajectory that we are on seems even bleaker. The new GPT-style architectures (non-ironically called Transformers [5]) have immensely more power compared to psychographic analytics-style models. Furthermore, as a result of the immense power of these architectures, they can be trained on immensely larger datasets, like all of our private emails and messages, to produce very powerful predictive language models of us.
And it gets worse.
End-to-end encryption to save the day?
WhatsApp is said to use end-to-end encryption. This means, technically, even Facebook’s own servers don’t have access to the content of your messages.
This is great for privacy, right?
But potentially Facebook’s servers don’t even need access to the content of your messages.
The client-side app (i.e. the app you run on your phone to decrypt and read the messages) could locally run your messages through an encoder to extract abstract representations of your ‘thoughts’ [10].
These abstract representations (technically just a sequence aka ‘vector’ of numbers), mean nothing to human minds, and might as well be a bunch of random numbers. But they are exactly what the model needs to be trained on, to build a predictive private model of you.
Conclusion
As a reminder, I have only been playing with GPT-2, a 1.5B parameter model. I’ve seen the difference between the 124M parameter model and the 1.5B parameter model, and it’s huge. The 1.5B parameter model is very noticably more believable, and is arguably a lot closer to and representative of the text that it was trained on. I haven’t had the opportunity to play with GPT-3 yet, and I can’t even imagine the difference jumping to 175B parameters will make.
I don’t know for sure if tech companies are currently building Private Individual Models of us, but I would be very surprised if they weren’t. And if they aren’t already, with the trajectory that we’re on, I’m pretty sure that they will be in the near future. And such models could easily be used to reveal a person’s opinions on various sensitive topics, or queried to find people who hold certain views, all without access to the original content, therefore bypassing existing privacy frameworks and entering into legal grey areas, raising severe privacy and abuse risks.
These new transformer-based massive language models are like a search engine of the future. You don’t search for what has happened, or what has been said. You search for what could happen, what could be said.
Side note
One of the big open problems in AI (in Natural Language Processing — NLP), is Question Answering (QA).
For example, I say to you: “Alice has a book. She gives the book to Bob. Bob goes to the kitchen. Alice goes to the garden”.
And then I ask you “Where is the book?”
You will most likely say “The book is in the kitchen, with Bob”.
To be able to answer this correctly, you need to understand this text. You need to understand the concept of ‘having’. You need to understand that ‘She’ refers to Alice. You need to understand what ‘to give’ means. You need to understand what ‘to go’ means. You need to understand the concept of being somewhere.
And this is a very very simple example. But even this has proven to be a very difficult problem for AI / NLP systems, and it’s only in the last few years that significant progress has been made. And even now, after 3 or 4 sentences the systems still fail, trying to do Multi-Hop Reasoning as it’s called.
Here is another example (known as a Winograd Schema)
Take the sentence: “Bob tried to put the book in his bag, but he couldn’t, because it was too big”.
If I ask you, what does ‘it’ refer to? (i.e. what was too big?)
You can’t find the answer to this grammatically or syntactically, you need to understand the concepts of big and small, and ‘to put inside’.
Because if I had instead said: “Bob tried to put the book in his bag, but he couldn’t, because it was too small”.
This sentence is grammatically, and structurally identical to the previous one. But the answer is different.
In the first case, we understand that ‘it’ refers to the book. The book was too big to fit in his bag.
But in the second case, we understand that ‘it’ refers to the bag. The bag was too small for the book to fit in.
Solving these kinds of problems has been one of the holy grails of AI. This is a very active area of research. But of course not just for one sentence, but for entire paragraphs, for entire essays, books, thousands, millions of emails.
(There’s also a version of Question Answering called Visual Question Answering — VQA. Which involves building systems whch try to answer questions about what’s happening in images or videos. That’s also one of the holy grails of AI. I wonder why).
Because AI — at least as we have it today — is a surveillance technology born of a surveillance economy [12].
[1] “OpenAI’s new language generator GPT-3 is shockingly good — and completely mindless”.
https://www.technologyreview.com/2020/07/20/1005454/openai-machine-learning-language-generator-gpt-3-nlp/
[2] My rant on “Artificial Intelligence as the name of a research field vs a characteristic of being”.
https://memoakten.medium.com/artificial-intelligence-as-the-name-of-a-research-field-vs-a-characteristic-of-being-920cb5f3361e
[3] My tiny introducton to artificial neural networks I wrote to explain Deepdream (2015).
https://memoakten.medium.com/background-info-for-deepdream-is-blowing-my-mind-1983fb7420d9
[4] For a nice description of predictive text models, see Andrei Karpathy’s blog post. But keep in mind, Karpathy’s post is based on a radically different architecture — known as Recurrent Neural Networks. The new architecture underlying GPT models — known as Transformers [5] — are completely different and proving to be way more powerful. https://karpathy.github.io/2015/05/21/rnn-effectiveness/.
[5] The paper that introduced the ‘Transformer’ architecture underlying the GPTs and models like Google’s BERT. “Attention Is All You Need”
https://arxiv.org/abs/1706.03762
[7] https://openai.com/index/better-language-models/
[8] https://openai.com/index/gpt-2-6-month-follow-up/
[9] https://openai.com/index/gpt-2-1-5b-release/
[10] My long (~1 hour read) article explaining high dimensional latent spaces and representations. The last sections ‘10. Word Embeddings’, ‘12. Thought Vectors’ and ‘13. Neural Story Teller’ are most relevant here. Though the sections before lay the ground work.
https://medium.com/artists-and-machine-intelligence/a-journey-through-multiple-dimensions-and-transformations-in-space-the-final-frontier-d8435d81ca51
[11] https://www.amnesty.org/en/latest/news/2019/07/the-great-hack-facebook-cambridge-analytica/
[12] My rant on the real reasons behind the rise of Deep Learning / Artificial Intelligence
https://memoakten.medium.com/retune-2016-part-1-the-dawn-of-deep-learning-672b5490f5a2
