AINoon Lesson 2

Get Ready for AINoon!

Open technoon.org/ainoon
Log in with a free account to chatbase.co/dashboard

Thanks

To the host for the great venue!
To our sponsors

Administrivia

Fire escapes
Toilets
Cleaning up after ourselves
Wi-Fi

Demystifying AI

How generative AI works and how that informs how to best use it

How does a chatbot generate text?

The chatbot predicts what text comes next, one word at a time
The prompt is split into tokens (words or parts of words)
- Services usually charge you for each input and output token
Tokens are passed into a “large language model” or “LLM”
- Each token is represented as a number
Those numbers are processed by a series of layers of “neurons”
The final layer outputs a probability for each possible token that could come next
A token is randomly picked based on those probabilities
- Some chatbots let you choose a temperature - a lower temperature is less random
- i.e. The highest probability is more likely to be picked with a lower temperature
The chosen token is added to the prompt, and the whole process is repeated to get the next token
Each neuron multiplies each input by a weight, sum the result, and applies a transformation to produce a single output number
- Loosely inspired by a neuron in the brain
- It’s just primary school maths!
- You may have heard AI needs special pieces of hardware Graphic Processing Units or GPUs
  - It’s because they can do lots of this maths very fast
The layers are connected in a special pattern called a “transformer”
The transformer architecture from Google in 2017 was the breakthrough that led to ChatGPT
There are millions of neurons, with billions of weights, meaning billions of multiplications!
- When someone says “OpenAI’s GPT-3 has 175 billion parameters”, they’re counting the weights
- Models keep getting bigger, which lets them more accurately predict which token comes next
Image, audio, and video models are similar
- But different kind of chunks are used as tokens

Where do LLM weights come from?

Weights start as random numbers
Weights are updated by a process called pre-training:
- Uses millions of training examples from general sources, like the internet and books
- E.g. Leave-one-word-out examples:
  The Beatles were a ____ from England → band
- For each example, weights are updated slightly to get closer to the right answer - like tuning a dial
- This trains the LLM in language patterns AND general knowledge
Extra training based on human feedback makes chatbots respond better to questions and instructions
(e.g. Chatbot gives 5 answers and human picks the best)

You now understand what GPT stands for!

Generative	The model generates text (or other content)
Pre-trained	Pre-trained on many examples from general sources
Transformer	The kind of neural network architecture used

What does this teach us about using GenAI?

The generation process is not magic, just simple maths!
We know how AI works, but we can’t interpret its decision-making
- The billions of calculations across neurons are too complex
AI doesn’t “think” with logic - it’s predicting the best answer based on its training sources
- People say AI sometimes “hallucinates” wrong answers - but every response is generated the same way
- It generates answers that sound right - and it’s surprising so many actually are right!
Training happens every few months - not during your chats
- It “remembers” by including chat history in the prompt
- Even “memory” features just include past chats in the prompt
- Public chatbots may use your chats to train later models

GenAI Strengths and Limitations

Key Strengths

Generates text that looks similar to what a human would write
Reads lots of text really fast
Surprisingly good responses to a wide variety of tasks

Key Limitations

Everything is a hallucination - but it’s correct surprisingly often
It might not always “pay attention” to everything in the prompt
It might not consistently give the same answer
Different models (or model versions) can give very different responses to the same prompt
It only “knows” whats in its training data (usually 1-2 years old)
- Pairing it with web search mitigates this

Tutorial Objective

Make your own chatbot to answer questions based on information in a document

Testing a Chatbot

Either you test your chatbot, or your users will!

Make a list of questions you want it to answer
Ask each question multiple times to check it consistently gives the right answer
Amend instructions or sources to improve answers
Re-test all questions after any change
- Changes to instructions, knowledge, or model
- Using a system to automate testing helps!
Test edge cases
- How does it respond to off-topic questions?
- If a user reports a failure, add a new test!

Homework

Make a list of questions you want your chatbot to be able to answer
- Feel free to use a different source document!
Ask each question and note whether it was correct
For questions it got wrong, modify the source to help it get the right answer
Test it again!
- Is it right more often?
- Is it wrong for any questions it got right before?

Extra: Learn more about how LLMs work with a brief explainer or a deeper dive (more maths) from 3Blue1Brown.