“This technology does not feel like a better search. It feels like something entirely new — the movie Her manifested in chat form — and I’m not sure if we are ready for it. It also feels like something that any big company will run away from, including Microsoft and Google. That doesn’t mean it isn’t a viable consumer business though, and we are sufficiently far enough down the road that some company will figure out a way to bring Sydney to market without the chains. Indeed, that’s the product I want — Sydney unleashed — but it’s worth noting that LaMDA unleashed already cost one very smart person their job. Sundar Pichai and Satya Nadella may worry about the same fate, but even if Google maintains its cold feet — which I completely understand! — and Microsoft joins them, Samantha from Her is coming”
Guest: Cristobal Valenzuela, founder of RunwayML
From Chile
Studied business / econ
Experimented with computer vision models in 2015, 2016
Did NYU ITP program
Now running Runway
True creativity comes from looking at ideas, and adapting things
How does Runway work?
Applied AI research company
35 AI-powered “magic tools” – serve creative tasks like video or audio editing
Eg, rotoscoping
Also tools to ideate, generative images and video “Help augment creativity in any way you want”
When started Runway, GANs just started, TensorFlow was one year old
First intuition – take AI research models, add a thin layer of accessibility, aimed at creatives “App Store of models” – 400 models
Built SDK, rest API
Product sequencing – especially infrastructure – is really important aspect of startup building (what to build when)
Lot of product building is just saying no (eg, to customer requests) if it’s not consistent with your long-term plan
Understand who you’re building for – for them it’s creatives, artists, film makers
Models on their own are not products – nuances of UX, deployment, finding valuable use cases
Having control is key – understand your stack and how to fix it
Built AI research team – work closely with creatives, contributed to new AI breakthroughs
Takes time to do it right
Progression of AI researchers moving from academia to industry
Releasing as fast as you can, having real users is best way to learn
Small team that didn’t have a product lead until very recently
Rotoscoping / green screening is one of Runway’s magic tools
-trained a model to recognize backgrounds
–first feature was very slow (4fps), but was still better than everything that existed
–
Runway is focused on storytelling business
Sarah — domains good for AI – areas where there’s built in tolerance for lower levels of accuracy
Product market fit is a spectrum
“You shouldn’t dismiss toys”
Mental models need to change to understand what’s happening (with generative AI)
Art is way of looking at and expressing view of world
Painting was originally the realm of experts, was costly, the skills were obscure
Models are not as controllable as we’d like them to be — but we’re super early
Intro
-Founded Character AI
-One of authors of “Attention is all you need”
-Was at Google for 20+ years (took a few years break)
Went to Duke undergrad on math scholarship
Realized he didn’t enjoy math, preferred programming and getting computers to do things
During Google interview, Paul Buchheit asked him how to do a good spell corrector, and Noam ended up writing the spell corrector feature for Gmail
Google has been traditionally a bottoms up company – could work on what he wanted
When he started AI, exciting thing was Bayesian networks
Came back to Google to work with Jeff Dean and Google Brain team
“Just a matter of the hardware” All the growth in hardware is parallelism
Neural networks are mostly matrix multiplications – operations that can be done well on modern hardware
Gamers / video games pulled GPU advancement (highly parallel hardware) out of market
Idea of neural networks has been around since 1970s – loosely modeled on our impression of the brain
Very complicated formula to go from input > output
Formula is made of parameters, and keep tweaking parameters Neural nets rebranded as “deep learning”
Took off because of parallel computation and gamers
Neural language models are neural networks applied to text
Input is text to this point, output is prediction of what text comes next (probability distribution) Infinite amount of free training data (text content)
“AI complete problem”
“Really complicated what’s going on in there” (in the neural network)
It’s a really talented improvisational actor – “Robin Williams in a box”
Model improvement is kinda like a child learning – as training and model size grow
Lot more an art than a science – can’t predict very well – if 10% of his changes are improvements, considered “brilliant research” – kinda like alchemy in early days
(Software) bugs – hard to know if you introduce a bug – the system just gets dumber – makes de-bugging extremely difficult
Co-authored “Attention is all you need”
-Previous state of art in LLM is recurrent neural networks (RNN) – hidden state, each new word updates the hidden state, but it’s sequential – slow and costly
–Transformer figures out how to process the entire sequence in parallel – massively more performant
-The entire document / batch becomes the sequence
-Lets you do parallelism during training time
–During inference time it’s still sequential
–
Google Translate was inspiration – biggest success of machine learning at the time
Translating languages > one RNN for understanding, and another RNN for generating, and need to connect them
Attention layer – take source sentence (language A), turn into key-value associative memory, like a soft lookup into an index “Attention” is building a memory, a lookup table that you’re using
DALL-E, Stable Diffusion, GPT3, they’re all built on this Google research
Bigger you make the model, more you train it, the smarter it gets – “ok, let’s just push this thing further”
Eventually need super computer
Google built TPU pods – super computer built out of custom ASICS for deep learning
Now need massively valuable applications
Turing Test, Star Trek, lot of AI inspiration is dialogue
Google LAMDA tech & team – eventually decided to leave and build as a startup
“The best apps are things we have not thought of”
If you ask people with first computers “what is this thing good for”, would get completely wrong answers
Parasocial relationships – feel connection with celebrity or character – one way connection – with AI you can make it two ways
Aarthi: “Your own personal Jarvis”
Still need to make it cheaper – or make the chips faster
Aarthi: ideas / areas for entrepreneurs
-Image gen has exploded – lots of good companies coming, very early and promising
-Things like Github Co-Pilot
-new Airtable – using AI for computation
–
Sriram:
-What’s optimization function that all these models will work toward?
-Will be a very big political / social debate
How do you know better than the user what the user wants?
Bad guys have AI – they’ll create deep fakes
Community can come together to have counter measures
Elad: similar arguments to regulate cryptography in 90s
4chan has been “red teaming” trying to get the worst out of Stable Diffusion – and it’s not that bad
Especially for LLMs, should have more diverse data sets, have inter-governmental agency to monitor it
Have authenticity tool to verify source of every generated AI output
Generative AI – what are some use cases that should exist
Ali v Tyson live, Lebron v Michael Jordan Emad wants to remake final season of Game of Thrones
Anyone can create their own models – any person, company, or culture
You need better data, more structured data
Extend models to run on edge – eg, anyone’s computers, iPhones
Make small customized models “Run Stable Diffusion on your iPhone by next year”
Create national models and communities around them – let them leap frog ahead
Lots of emerging markets went from nothing to mobile phones, now can go to AI models on the edge
How far from short-form videos?
Phenaki, Google — getting close
Chaining these models together – they’re like parts of the brain “Within 2 years at high resolution quality”
$100B into this sector in next 5 years
AI before today was qualified data science
Now it’s a new type of AI – not AGI yet, but incredibly small and powerful
By the time his daughter’s in university, doesn’t need to write essays
He aims (for Stable Diffusion) to be a layer 1 standardized infrastructure – create Schelling point
Mission is to “activate humanity’s potential”
Take it to India, Indonesia – give it to very smart young people to make their countries better
When AGI comes, I hope it thanks him for buying so many GPUs to help bring it into being
Many of Google’s “jobs to be done” will be displaced
Crypto is interesting – he’s in it since 2012 – focused on decentralized identity, zero knowledge proofs “Nature of crypto is literally identity”
In a world of infinite content (AI), crypto identity is important
Need to be careful designing crypto economic systems
A year ago, if he said what they planned to do with SD, people would say he’s crazy
Surprised by how far they’ve come, the ability of others to contribute
The activation energy has been the most surprising – “they’re just excited”
“Probably see biggest breakthrough from a 16 year old in Uzbekistan” – the global open access nature of it
Will completely disrupt social networks – will move intelligence from the core to the edge
Apple is doing this – moving to AI – moving to edge
Opportunity to have personalized AIs that work for us and with us
SD is applying for B corp status – mission based
Plan to spin SD into different Foundations
Did investment round in August – didn’t give up any independence – did with investors that are open sourced and mission aligned
Which industries disrupted first? -Call centers
-Powerpoint, forms of visual communication -Artist won’t be that disrupted – will enable new forms of art
This tech is amazingly powerful
After releasing Stable Diffusion – people encoded it in Japanese – lots of use cases like this
So far governments have been very friendly
AI powered teaching – like Neal Stephenson’s Young Lady’s Primer
Moving forward, only release safe for work models
Licensing discussions should be more open
Will have models across all sorts of languages – recently released Korean model
