A thought provoking post from Venkatesh Rao (@vgr / Ribbonfarm) on AI:
Yes, there’s still superhuman-ness on display — I can’t paint like Van Gogh as Stable Diffusion can (with or without extra fingers) or command as much information at my finger-tips as the bots — but it’s the humanizing mediocrity and fallibility that seems to be alarming people. We already knew that computers are very good at being better than us in any domain where we can measure better. What’s new is that they’re starting to be good at being ineffectual neurotic sadsacks like us in domains where “better” is not even wrong as a way to assess the nature of a performance.
There are, by definition, only a handful of humans whose identity revolves around being the world’s best Go player. The average human can at best be mildly vicariously threatened by a computer wiping the floor with those few humans. But there are billions whose identity revolves around, for instance, holding some banal views about television shows, sophomoric and shallow opinions about politics and philosophy, the ability to write pedestrian essays, do slow, error-prone arithmetic, write buggy code, and perhaps most importantly, agonize endlessly about relationships with each other, creating our heavens and hells of mutualism.
I don’t think humans are all that special. Yes, each human is special in some limited way, and together as a species we have built some very special things.
But it’s increasingly clear that some of those very special things we have built — such as AI and coming soon, smart robots — will expose our own flaws and imperfections, a kind of inverse magic mirror, and there is and will be a deepening divide between those who use or even love the magic mirror, and those who want to look away or smash it.
This divide is already a driver of the world’s growing income inequality (though I think the generational divide has been a much larger cause of this, at least in developed economies), and I think it will become *the* driver in the coming decades.
He doesn’t read the news
Likes trolling on Twitter…”Twitter’s fun”
“Twitter’s gonna be fine”
HIS INVESTMENTS
400 personal investments, a few thousand including YC
All the companies he’s added value to are those he thinks about in his free time – while hiking, texting the founder an idea
Most successful investment? Stripe on a multiples basis
Helion – fusion energy
Personally invested $375M (!)
Other thing besides OpenAI that he spends a lot of time on
New energy system that works on super low cost Hardest challenge is how to replace all (current) generative capacity on Earth really quickly
“Who can deliver energy the cheapest, and enough of it”
Simple machine, affordable cost, reasonable size
If fusion works…will change dynamics of what’s possible – enables more downstream (eg, more powerful planes)
Hermeus – supersonic jet company
Led $100M round
Was also involved with a competitor Boom – but different tech and approach
Huge market with multiple needs
Worldcoin
He’s a cofounder, on the board, but not day to day involved
Will tell its story soon – believes it will go over well (unlike earlier negative media coverage) We give up more privacy to Facebook than Worldcoin
Phenomenal team
Launch in months
Interested in any tech to experiment with global UBI (versus what one country can do)
Re: crypto — “honestly not super interested” “Love spirit of web3, but don’t intuitively feel why we need it”
Inception Fertility
In-vitro gametogenesis
In shadow of AI Next 5-7 years of biotech will be remarkable
Human life extension – “yeah maybe that’s gonna work”
Investing for 20 years, president of YC for 5-6 years
Garry (new YC president) will do a lot of things differently and be wildly successful
Last few years were really hard for YC YC can remake itself now – tourists are leaving now
—
ARTIFICIAL INTELLIGENCE
OpenAI has pulled together “most talent dense” AI team
“Gonna be tremendously good”
Why did ChatGPT and DALL-E so surprise people?
“Don’t know…reflected on it a lot”
If you make a good UX on top of something – believed users wanted to interact via dialogue
Pieces were there for awhile
Standard belief was AI would take over low skill / truck driving / generic white collar
Going exact opposite direction – taking over creativity where we thought humans might have special sauce
It’s not an intuitive finding
Released GPT-3 three years ago – thought ChatGPT would be incremental, was surprised by public reaction
ChatGPT will cause societal changes – eg, academic integrity “Stakes are still relatively low”
Covid did show us society can update to massive changes faster than he expected
Given expected economic impact – “more gradual is better”
GPT-4 will come out when we’re confident we can do it safely and responsibly
Will release tech much more slowly than people will like
GPT-4 rumor mill is a ridiculous thing
Re: ChatGPT – built a thing, couldn’t figure out how to monetize it, put it out via an API, and users figured out how to use it
Would like to see AI super democratized, have several AGIs in the world Cost of intelligence and energy trends down and down
“Massive surplus…benefits all of us”
Believes in capitalism – best service at lowest price
Society will need to agree on what AGI should never do
Broad absolute rules of the system
Within that, AI can do different things – safe for work one, edgier creative one – different values they enforce A user can write up a spec of what they want, and AI will act according to it – “should be your AI…to serve you”
Microsoft – only tech company he’d be excited to partner with this deeply
Satya, Kevin Scott, Mary McHale
Values-aligned company
“We’re very much here to build AGI”
“We wanna be useful to people”
Re: Google’s AI – hasn’t seen it, assume they’re a competent org
We’re in a new world – generated text is something we all need to adapt to, like we adapted to calculators
“I’d much rather have ChatGPT teach me…than read a textbook”
Anthropic – rival AI, stressing an ethical layer
Very talented team
Multiple AGIs in the world is better than one
Society decided free speech is not quite absolute – in similar ways AI / LLMs will need to have bounds too
Video is coming… no confident prediction about when Legitimate research project – could take awhile
AUDIENCE Q&A
When fusion online?
By 2028, could be plugging fusion generators into grid (pending regulators)
Re: AI worst and best case?
“best case is so unbelievably good that it’s hard to imagine, discovering new knowledge in a year instead of 70K years”
“Bad case is lights out for all of us” More worried about accidental mis-use in short term, less about the AI itself being evil
How far away is AGI? Much blurrier and gradual transition than people think
Re: state of San Francisco
Real shame we treat people like this
How elected leaders don’t fix the problem
Tech has some responsibility for it
But other cities do better than this Super long in-person work and Bay Area
Re: ChatGPT reaction
Expected one order magnitude less hype, users, of everything
Less hype is probably better The tech is impressive, but not robust Use it 100x, see the weaknesses
How Sam uses ChatGPT? Summarize super long documents, emails
For translation
Re: Google code red, threat to search
When people talk about new tech being end of a giant company, they’re usually long
Change is coming (for Google), but not as dramatically as people think
Before Google, memorizing facts was important – and now we’ll change again – and we’ll adapt faster than most people think
Prefers hybrid work, like YC – few days at home, few days in office
Skeptical that fully remote is thing everyone does
Most important companies will still be heavily in-person
Safety engineering for AI is different from standard safety engineering – consequences are greater, deserves more study
Raising capital now is hard, especially later stages – but other things easier – easier to rise above noise, hire, get customers What he’d do now — recommends for founders – “do AI for some vertical”
Advice for AI startups
Differentiate by building deep relationships with users, some moat like network effect
Plan for AI models continually improving
OpenAI is a platform, but also wants to do a killer app (platform + killer app) to show people what’s possible
Derived from things humans said, but doesn’t always know connections between the things – which can lead to it saying whacky things
Transforms everything into an “embedding space”
It’s pastiche – imitating styles, cutting and pasting, template aspect
Both brilliance and errors come from this
Humans have some internal model – of physical world, of a relationship
Understanding is also about indirect meanings
AI doesn’t have these internal representations
Sam Altman – “humans are energy flowing through a neural network”
Human neural networks have much more structure than AI neural nets
Neural nets like ChatGPT have 2 problems:
-not reliable
-not truthful
Making them bigger doesn’t solve those problems – because they don’t have models of “reliability” or “truthfulness” “They’re just auto-complete” “It’s mysticism to think otherwise”
No conception of truth – all fundamentally bullshitting
It’s a very serious problem
Can use GPT3 to make up stories about covid, spread bullshit and misinformation
No existing tech to protect us from this potential tidal wave of misinformation
People used ChatGPT to make up fake coding answers to put on StackOverflow
StackOverflow had to ban ChatGPT
“Talking about having submachine guns of misinformation”
Ezra – ChatGPT is really good at mimicking styles, but core doesn’t have truth value or embedded meaning
Silicon Valley always good at surveillance capitalism
Now you can use AI to write targeted propaganda all day long
Ezra – this AI will be good for people and companies who don’t care about truthfulness. eg, Google doesn’t care about “truth” but about clicks
One of main AI use cases is SEO copy to drive clicks and engagement
In some aspects, as models get bigger they get better (eg, at generating synonyms); but at truthfulness, wasn’t as much progress
Loves AI and have thought about it his whole life – just wants it to work better, be on better path
Biology has enormous complexity – too much for humans – AI could really help us
Also climate change
Could empower individuals like DALL-E does for artists “Reason according to human values”
Right now we have mediocre AI – risk of being net negative – polarization of society, growth of misinformation
Parable of drunk looking for keys at night around a street light The street light in AI is deep learning
Human mind does many things: pattern recognition, use analogies, plan things, interpret language
We’ve only made progress on a tiny part
We need other tools (not just deep learning)
Fight between neural nets and symbols (symbolic systems / symbolic learning)
Find ways to bridge these 2 traditions
Ezra:
Deep learning – give all this data, figure it out itself, but bad at abstraction
Symbolic systems – great at abstraction (eg, teach it multiplication from first principles), abstract procedures that are guaranteed to work
Most symbol manipulation is hard wired
Kids learn to count using generalized rules
AI hasn’t matched humans in capacity to acquire new rules – some paradigm shift needed
We’re worshipping the false god of “more data”
Believes will be genuine new innovation in next decade
A little something we’re working on, you can try it here: http://galerie.ai/
It does two things
1. Returns results from a growing library of the best AI art
2. Lets you create AI art from multiple generative models (including Stable Diffusion 1.5, 2.0, 2.1, and MidJourney)
Let me know what you think!
We auto-suggest some of the trending prompts and popular art results on the home page, try it out!
For example here is ” photorealistic portrait, a young beautiful woman Goddess wearing Echo of Souls Skull Mask armor, skeletal armor bones made from 24k gold and silver metal intricate scroll-work engraving details on armor plating, skeletal armor, gemstones, opals, halo, aura, intricate details, symmetrical,”:
Not yet trillion dollar “take on Googles” startups yet – but will be a serious challenge to Google for first time
eg, Human level chatbot interface that actually works – new medical services, new education services
Idea of language interface where you say in natural language, dialogue, and computer just does it for you
Very powerful models will be one of genuine new tech platforms since mobile
How to create an enduring differentiated business
-small handful of large base models will win – skeptical of startups doing small models
–middle layer will become really important – take large models, tune it, create model for medicine, or model for AI friend – will have data flywheel
Lots of AI experts think these models won’t generate net new knowledge for humanity – thinks they’ll be wrong and surprised
AI in science:
1. Science dedicated products eg Alpha Fold – will see a lot more, bio cos will do amazing things
2. Tools that make us more productive – improve net output of scientists and engineers – eg, CoPilot
3. AI that can be an AI scientist to self improve – automate our own jobs, go off and test new science and research – teaching AI to do that
What is Alignment Problem?
A powerful system that has goals in conflict with ours
How do we build AGI that does things in best interests of humanity
How to avoid accidental or intentional mis-use
AI could eventually help us do alignment research itself Reid: will be able to tell agent “don’t be racist” and let it figure out
AI moonshots?
-language models will go much further than people think – so much algorithmic progress to come, even if we run out of compute or data
–true multi modal models – every modality, fluidly move between them
-continuous learning models
These above 3 things will be huge victory
OpenAI – focus on next thing where we have high confidence, let 10% of company go and explore
Can’t plan for greatness, but sometimes breakthroughs will happen
AI will seep in everywhere Marginal cost of intelligence and energy will rapidly trend towards zero – will touch almost everything
Metaverse will become like iPhone – a new container for software AI will be new technological revolution – more about how metaverse will fit into AI then vice-versa
Low cost + fast cycle times is how you compete as a startup
In bio – simulators are bad, AI could help
What are best utopian sci-fi universes so far
-Star Trek is pretty good
-The Last Question is incredible short story
-Reid: Ian Banks – Culture series
-tried to write his own sci fi story, was a lotta fun
Having a lot of kids is great – wants to do it
Won’t be doing prompt engineering in 5 years
Will be text / voice in natural language to get computer to do what you want
eg, Be my therapist and make my life better; Teach me something I want to know
Reid: great visual thinker can get more out of DALL-E — will be an evolving set of human talents going that extra mile
How to define AGI Equivalent of a median human that you can hire as a coworker – be a doctor, be a coder
Meta-skill of getting good at whatever you need
Super intelligence = smarter than all of humanity put together
Economic impacts will be huge in 20-30 years Society may not tolerate that change – what is the new social contract
How to fairly distribute wealth
How to ensure access to AI systems (“commodity of the realm”)
Not worried about human fulfillment – we’ll always solve it
But concepts of wealth and access and governance will all change
Running largest UBI experiment in world – 5 year project
Tools for creatives — will be the great application for AI in short-term
Mostly not replacing, but enhancing their jobs
How do these LLMs differentiate from each other?
The middle layer is what will differentiate – the startups fine-tuning the base models, about the data flywheel, could include prompt engineering
