If we have $50B to do good, how do we spend it?
Ex: Governance corruption, Sanitation and water, Global warming, Malnutrition, etc
This question was asked at Davos
UN existed for 60 years, but we’ve never made such a list and discussed how to prioritize them
Prioritization is incredibly uncomfortable
But it’s like walking into a pizzeria but not knowing the price of each pizza
3 well-known economists were tasked to come up with such a list
“Bad projects” = invest $1, get <$1 back
Bottom of list was climate change
This offends people
But why is it a bad deal (eg, Kyoto)? It’s very inefficient – can only do very little, at very high cost
Benefits don’t accrue for many decades – and by then most of the affected people will be much richer and more prosperous (even better than 1st world citizens today)
Kyoto agreement estimated to cost $150B/year – 2-3x global development aid to Third World yearly
For half of that amount – $75B/year – we can solve all major basic problems – clean water, sanitation, etc to benefit everyone on the planet
Top priorities – the “best deals”
1. HIV/AIDS – $27B over 8 years, avoid 28M new cases, prevention > treatment
2. Malnutrition – lack of micronutrients, lacking iron zinc vitamin A, $12B
3. Free trade – cut subsidies in US and Europe, enliven global economy, $2.4T improvement in global GDP
4. Malaria – few billion cases each year, invest $13B over 4 years to cut incidence by half
We should do all of them – but we don’t – in fact aid to developing world has been decreasing not increasing
It’s not about making us feel good, about things with the most media attention
Copenhagen Consensus – mapping out right path for world, think about political triage “Let’s do enormous amount of good at very low cost right now”
This podcast made me feel very stupid, and very inspired.
—
Turing test – in 1950s, Turing didn’t mean it to be a rigorous formal test, more of a philosophy experiment, didn’t specify things like parameters of test, how long test should last, etc
More modalities than just language to express intelligence eg physical movement
Played chess at 4, earnings from winning a chess competition let him buy a computer
Bought programming books, started making games, felt they were a magical extension of your mind
“AI is ultimate expression of what a machine can do or learn”
At 12yo he got to chess masters level, the process of learning chess makes him think a lot about thinking and about brains
“Chess computer handbook” by David Levy – explained how chess programs were made
First AI program he built was on his Amiga, programmed Othello
Wrote game called “Theme Park” with a core AI, sandbox game, reacted to players, every game was unique
He designed and wrote AI for games in 90s – at the time, game industry was cutting edge of tech (GPUs for game graphics, AI, John Carmack)
“Black and white” game – train a pet, and depending on how you train it, it would be more or less kind to others, powerful example of reinforcement learning
DeepMind – core part of strategy from start was to use games to test how well AI is doing, if the ideas are working
Eg, Go – clear rules and win conditions, humans have played for thousands of years, easy to test how good is your system vs human players
Part of why their AI has progressed so quickly – by developing against games
“Chess is drosophila of intelligence” – Gary Kasparov
Many AI researchers have all wrote chess AI programs
DeepBlue beating Kasparov was a huge moment – he was in college at time – came away more impressed with Kasparov’s mind than with DeepBlue (because Kasparov could play almost at the AI level, but could also do all these other things as a human, while DeepBlue at that time couldn’t even play tic-tac-toe)
What makes chess compelling as a game? Creative tension between bishop and knight – leads to a lot of dynamism
Chess has evolved to balance those two more or less equally (worth 3 points each)
Balanced by humanity over hundreds of years
Different levels of creativity
1. Lowest level is interpolation – averaging everything you see (eg, “an average looking cat”)
2. Next is extrapolation – AI coming up with a new move in Go that no one’s seen
3. Out of the box innovation – coming up with a new game entirely – AI nowhere close to this yet
Currently AI can do 1 and 2 but not 3
For 3, if you were to instruct an AI to create a game, you’d say “come up with a game that takes 5 minutes to learn, but lifetimes to master, aesthetically beautiful, and can be completed in 3-4 hours”
We can’t abstract high level notions like that to AIs (yet)
AI could be used to make current games better by taking game system, playing millions of times, and then improving the balance of rules and parameters – give it a base set + Monte Carlo tree search – takes humans many years and thousands of testers to do it
His first big game was theme park, amusement park – then whole cities – and Will Wright’s made SimEarth simulating the whole earth
“Simulation theory” – doesn’t believe it, in sense that we’re in a computer simulation / game, but does think best way to understand physics and universe is from computation perspective – information as fundamental unit of reality instead of energy or matter
Understanding physics as information theory could be valuable
Roger Penrose – Emperor’s New Mind – he believes we need quantum, something more, to explain consciousness in the brain
Most neuroscientists / mainstream biologists haven’t found any evidence of this
While continually classic Turing machines keep improving – and DeepMind / Demis work is champion of this
Thinks universal Turing machines can eventually mimic human brain without Penrose need for something more
Something profoundly beautiful and amazing about our brains – incredibly efficient machines, in awe of it
Building AI and comparing to human mind will help us unlock what’s truly unique about our minds – consciousness, dreaming, creativity
Philosophy of mind – there haven’t been the tools, but today we increasingly have them
Lex – Universe built human mind which built computers to help us understand universe and human minds
Protein folding – AlphaFold 2 solved it
Proteins = essential to all life – workhorses of biology, amazing bio-nano machines, specified by genetic sequence, in the body they fold into 3D structure (like string of beads folded into a ball)
The 3D structure determines what it does – and drugs must understand this to interact with it
Structure maps to function, and is specified by amino acid sequence
Unique mapping for every protein – but it’s not obvious – and almost infinite possibilities
Can you by studying the sequence, predict the 3D structure?
Takes 1 PhD student an entire PhD to predict one protein But AlphaFold 2 can do it in seconds now – over Christmas can do it over entire human proteome space (!!)
Biologists can now lookup protein 3D structure in a google search
AlphaFold was most complex and meaningful system they’ve built so far
Started on games (AlphaGo, AlphaZero), to bootstrap general learning systems
His passion is scientific challenges – AlphaFold is first proof point 30 component algorithms needed to crack protein folding
About 150K protein structures had been found – that was their training set
Would put some of AF’s best predictions back into training set to accelerate training
AF2 was truly end to end – from amino acid sequence directly to 3D structure, without needing all the intermediary steps – system is better at learning the constraints on its own instead of guiding it
AlphaGo – learning system but trained only to learn Go
AlphaZero – removed need to learn from human games – just play with itself MuZero – didn’t even need to give rules, just let it learn on its own
Started DeepMind in 2010 – back then no one was talking about AI, people mostly thought it doesn’t work (even at MIT)
If all professors tell you you’re mad, at least you know you’re on a unique track Founding tenets / trends
-Algorithmic advances (reinforcement learning)
-Understanding about human brain (architectures, algos) improving
-Compute and GPUs improving
-Mathematical and theoretical definitions of intelligence
Early days – ideas were most important – deep reinforcement learning, transformers, scaling those up As we get closer to AGI, engineering and data become more important
**For large models – scale is clearly necessary but perhaps not sufficient
DeepMind – purposely built multi-disciplinary organization – neuroscience + machine learning + mathematics + gaming, and now philosophers and ethicists too
“A new type of Bell Labs” DeepMind itself is a learning machine building a learning machine
Top things to apply AI – biology and curing diseases (AlphaFold), but it’s just beginning Eventually simulate a virtual cell (maybe in 10 years) – “that’s my dream”
Drugs take 10 years – target to drug candidate – maybe it can be shortened to 1 year with this, AlphaFold as first proof point
Math is perfect description language for physics AI as perfect description language for biology (!)
Open-sourced AlphaFold (including data) – max benefit to humanity – so many downstream applications, better to accelerate research and discovery, used by 500K researchers (almost every biologist in the world!), amazing fundamental research, almost every pharma company is using it, “gateway drug to biology”
Also open-sourced MuJoCo – purchased it explicitly to open source it
One day an AI system could come up with something like General Relativity (!)
Big new breakthroughs will come at intersection of different subject areas (DeepMind = neuroscience + AI engineering)
We just don’t understand what it’d be like to hold the entire internet in your head (imagine reading all of Wikipedia, but much much greater) – no one knows what will result
Nuclear fusion – believe AI can help
In any new field, talk to domain experts for collaboration
What are all the bottleneck problems? Think from 1st principles
Which AI methods can help
Problem of plasma control is great example – plasma is unstable (mini-sun in a reactor), want to predict what plasma will do next, to best model and control it
They’ve largely solved it with AI, and now looking for other problems in fusion
Simulating properties of electrons – if you do it, you can describe how elements and materials work (fundamental to materials science)
Would like to simulate large materials – approximate Schrodinger’s equation
His ultimate aim for AI – to build a tool to help us understand the universe – to test the limits of physics
A true scientist – the more you find out, the more you realize you don’t know Time, consciousness, gravity, life – fundamental things of nature – we don’t really know what they are
We treat them as fact and box them off – but there’s a lot of uncertainty about what it is
Use of AI is to accelerate science to the maximum – imagine a tree of all knowledge – we’ve barely scratched surface, and AI will turbocharge all of it – understanding and finding patterns, and then building tools
If you’re good at chess, you still can’t come up with a move like Garry Kasparov, but he can explain the move to you – potentially AI systems could understand things we could never by ourselves, and then explain it and make it useful for us
We’re already symbiotic with our phones and computers, Neuralink, and could augment / integrate with these AI
His current feeling is we are alone (no aliens)
We could easily be a million years ahead or behind in our evolution, eg, if meteor that destroyed dinosaurs came earlier or later – and in a few hundred years, imagine where we’ll be, AI, space traveling – we’ll be spreading across the stars; will only take ~1M years for Von Neumann systems to populate across the galaxy with that tech
We should have heard a cacophony of voices – but we haven’t heard anything “We’ve searched enough – it should be everywhere”
If we’re alone, somewhat comforting re: Great Filter (maybe we’ve passed it)
Wouldn’t be surprised if we found single cell alien life – but multi-cellular seems incredibly hard
Another large leap is conscious intelligence
General intelligence is costly to begin with – 20% of body’s energy – a game of professional chess is same as F1 racer
Hard to justify evolutionarily – which is why it’s only been done once (on Earth)
AI systems – easy to craft specific solutions, but hard to do generally – at first general systems are way worse
Do AGI systems need consciousness? Consciousness and intelligence are double dissociable – can have one without the other in both ways
Eg, Dogs have consciousness and self-aware but not very intelligent, most animals are pretty specialized
Eg, some AI are amazingly smart at playing certain games or doing certain tasks, but don’t seem conscious
May be our responsibility to build systems that are not sentient
None of our systems today have one iota of consciousness or sentience – way too premature
Re: Google engineer who believed their language system was sentient – Demis believes it’s more a projection of our own minds, our desire to construct narrative and agency even within inanimate systems
Eliza AI chat bots in 1960s – already fooled some people
Neuroscience – certain pre-reqs may be required for consciousness, like self-awareness, coherent preferences over time
Turing test is important, but there’s second that differ in machines: we’re not running on same substrate (humans are carbon based squishy life forms)
Language models – we don’t understand them well enough yet to deploy them at-scale Should AI be required to announce that it is AI?
re: AI ethics, important to look at theology, philosophy, arts & humanities
Heading into an area of radical abundance and knowledge breakthroughs if we do it right – but also huge risks
Need careful controlled testing instead of just releasing into the wild, the harms could be fast and huge
Better to first build these AI systems as tools – carefully experiment and understand – before we really focus on sentience
How to prevent being corrupted by this AI power:
-Important to remain grounded and humble
-Being multi-disciplinary keeps you humble – because always better experts
-Have good colleagues who are also grounded
AI can learn for itself most of its knowledge, but will have residue of culture / values from who builds it
Globally we can’t seem to cooperate well eg, climate change
Need to remove scarcity to help promote world peace – radical abundance
AI should belong to the world and humanity, everyone should have a say
Advice for young
-what are your true passions? explore as many things as possible; find connections between things
-understand yourself – how do you deal with pressure, hone your uniqueness and skills
Perfect day in Demis’ life, habits
-10 years ago: whole day of research + programming, reading lots of papers, reading sci-fi at night or playing games
-today: very structured, complete night-owl, 11-7pm work (back to back meetings, meet as many people as possible), go home and spend time with family, 10pm-4am do individual work, long stretches of thinking and planning and some email
-quiet hours of morning – love that time (1-3am), inspiring music, think deep thoughts, read philosophy books, do all his creative thinking, get into flow, sometimes will go to 6am next day, and pay for it the next day (but it’s worth it)
Always been a generalist – too many interesting things to spend time on just one
Lex: Why are we here?
Demis: To gain knowledge and understand the universe, understand world around us, humanity, and all these things flow from it: compassion, self-knowledge
Feel like universe is almost structured to let itself be understood and learned – why are computers even possible?
If Demis could ask one question of true AGI: “What’s true nature of reality?”
Answer could be a more fundamental explanation of physics, and how to prove them out
A deeper, simpler explanation of things – leading to consciousness, dreaming, life, gravity
I read this essay every few months and every few months my mind is blown again:
In other words, the difficulty-adjustment is about keeping a constant time, not a constant level of security, difficulty, or energy expenditure. This is ingenious because good money has to be costly in time, not energy. Linking money to energy alone is not sufficient to produce absolute scarcity since every improvement in energy generation would allow us to create more money. Time is the only thing we will never be able to make more of. It is The Ultimate Resource, as Julian Simon points out. This makes Bitcoin the ultimate form of money because its issuance is directly linked to the ultimate resource of our universe: time
Millennial survey – 80+% said major life goal was to get rich, 50% said to become famous
What if we could watch lives unfold as they actually happen?
“Harvard study of adult development” – tracked 724 men for 75 years
60 of those men are still alive and still participating in study – most in 90s Now they have 2000+ children
First group started as Harvard sophomores – most served in WW2
Second group were group of boys from Boston’s most troubled neighborhoods
All entered the study as teenagers
One became president of the US
Every 2 years, research staff does surveys of the participants
Draw blood, scan brains, talk to their children, videotape them talking to their wives
What’s been learned? Clearest message: Good relationships keep us happier and healthier
Relationship lessons
1. Social connections are really good for us – family, friends, community; loneliness is toxic – they’re less happy, their brain functions and health decline sooner
2. It’s not quantity but quality of closest relationships that matter – high conflict marriages are very bad for health, perhaps even worse than divorce
3. Good relationships protect not only our bodies but our brains – secure attachments at age 80, those brains and memories stay sharper for longer
Predictors at age 50 of longevity – not health, but “how satisfied were they in their relationships” – these became the healthiest at age 80
Relationships don’t have to be smooth – but need to be able to count on each other through tough times
Those happiest in retirement – actively replaced workmates with new playmates
People who far best are those who lean into relationships – family, friends, community
Twain quote: “There isn’t time – so brief is life – for bickerings, apologies, heartburnings, callings to account. there is only time for loving – & but an instant, so to speak, for that.”
SBF – founder of Alameda Research, FTX, before that at Jane Street, one of world’s youngest billys
What’s your Truth North?
Day to day – efficient markets, does the risk engine work, does product design make sense (eg, equity markets currently are not 24/7, but it shouldn’t be that way, it’s more a historical artifact)
More generally – philanthropy (effective altruism / utilitarianism), how can I maximize my positive impact on the world
Lots of organizations having lasting impact on long-run future of world, the trillions of people to come after us
What’s become clear is society has no fucking clue what to do about pandemics – not restricted to just one country – even the countries thought to do well initially, economies stalled out, and not clear path forward
What would it have taken for us to be in a much better place?
1. Took a year after covid hit to begin distributing vaccine – could have had it in 2 months
2. Early detection systems, better vaccine production systems, reduced regulatory time for testing / approval
How much is society putting in to prepare better for next pandemic? “I don’t know…zero?”
Doing things to solve this can save tens of trillions of dollars, and have enormous impact
What’s perfect state of markets? Start with latency – how low does it need to be to capture most of economic value – as close as possible to release rate of new economic information
Milliseconds is probably enough
NYSE is milliseconds, but then it’s closed overnight, and on weekends and holidays – “which is sort of insane”
Always open is pretty important
Another easy win: Order books should be free and publicly available
That’s whole purpose of exchanges – so why hide it, except for people paying $50M/year – trading firms are paying it
Crypto exchanges – fees come from transactions
Equity exchanges – service is mostly a commodity, so only proprietary stuff is their data Amount of intermediation is insane – mobile app, clearance custody, prime brokers, equity exchanges – all these intermediaries take fees, slow things down, add tape Innovation slows down too – if exchange wants to move 24/7, needs everyone else to change too, “weakest link component”
Reasonable trading fee is 1 or a few bps (basis points) – larger fees mean less efficient pricing, less liquidity, less overall economic activity – total net fees not just matching engine fees
Crypto represents a migration to better end state – always on, more transparent, globally accessible
State of fairness in crypto markets today
-most important thing is transparency about transparency (everyone agrees what market structure looks like)
-having a level playing field to start with (it’s still a total mess, but less so than 3 years ago)
-eg in 2017 lots of Japanese got excited about bitcoin, but bitcoin price in Japan was a lot higher than rest of world (10% arbitrage opportunity) – exact opposite of inefficient market
-today these arbs still exist – eg, Coinbase trading higher than Gemini for weeks at a time due to flows – driven by lack of liquidity, idiosyncrasies with some assets (eg, Tether), lack of integration with fiat / banking
How to solve a lot of this? “Stablecoins” – useful to move within the crypto system
eg, USDC – can move 24/7, fast and on blockchain, remove reliance on wire transfers
How much fiat inflow has actually gone into space?
Higher bound – Crypto market cap = $2.5 trillion (this was an old podcast)
Lower bound – $100B of stablecoins outstanding Probably $400-500B in actual fiat has invested (20-25% of total market cap)
What will change the ratio – most financial institutions are PLANNING to buy bitcoin at some point – they now have mandates to get involved, but they all say “they’re not ready yet” Should materialize in next few years
Probably the ratio will get closer to one (of actual fiat inflow to total market cap)
Huge demand for good infrastructure in crypto – exchanges still crashing during busy times
When SBF first entering crypto, hardest part of the crypto trade was the wire transfer
Lots of inefficiencies in traditional financial infra – wire to Nigeria is 10%, credit card fees are 3%
Crypto rails can help fix this – eg, all social data posted immediately on-chain, means a tweet can immediately be liked on Facebook, or a TikTok video can be instantly published to Instagram
If you’re in crypto and you’re not thinking extremely hard about regulation and compliance – you’re making a huge mistake
Must remain dynamic / flexible in long-term planning, to adapt to constantly changing environment
Crypto trades almost as much as US equity ($200B in global daily volume) – but US itself is far behind
Crypto is totally new asset class, born 5 years ago, and now is almost as large as the largest asset classes
Reasonable to find strategic parts of ecosystem to put bulk of regulations – eg, any centralized exchanges, or fiat-to-crypto conversion points
“Take steps in right direction” to protect consumers, detect financial crimes – will address most of large points of concerns, and help crypto ecosystem to thrive
There will be stable coins in world – if US bans them, then it will go to EUR or CNY
Why are derivatives so important for markets?
-in every asset class, there’s more derivatives trading volume – if you don’t need physical delivery, derivatives are more economically efficient
-average trade doesn’t require to get the actual thing (eg, a stock, or gold, or oil)
-so it requires less assets on balance sheets, more efficient markets, lowers capital requirements and transaction costs
What competitive differentiation among exchanges
-cross-margining (eg, FTX allows collateral to applies across multiple assets / trades)
Thoughts on paid acquisition
-most FTX users came from Twitter, from user memes and endorsements
-don’t buy FB / Google ads
-for hardcore traders, product is what drives it
-for new / casual traders, name recognition matters “and we’re way behind on that” (compared to Binance or Coinbase)
-want not just recognition of FTX the name, but create a powerful association
-only a few endorsements matter – should be extremely choosy
Thoughts on user generated asset era
-are books UGC? Sort of, but historically the gatekeepers are bookstores and publishing houses – it’s author UGC, but with bottlenecks and gate keeping
-before, 7 asset managers drove equity markets, now it’s social media and asking friends – Tesla is great example, people taking choices into their own hands
-NFTs are UGC, direct to consumer
-tokens / token economies are flourishing around the world, but not in the US due to regulations
-ultimately it’s good, more efficient markets, more dis-intermediation, and more curation will emerge naturally
Right now, there should be many Layer 1 blockchains, competition among them, to see what emerges What’s end game that matters the most? 1 billion users using a chain, trillions of dollars using a chain
To get there, need millions of TPS (transactions per second)
You want to maximize composability, even across shards
If you don’t get there, you won’t be primary player to facilitate all the activity that’s required
5-10 TPS is not enough to be general purpose medium, but it DOES allow you to move bitcoin around – “has potentially a large role in the world” – it’s a different thing from ETH or Solana
Most economically efficient thing is single centralized server
Decentralized blockchains have maybe 10K servers? Will always be less efficient
Blockchains will be connection layers – across more efficient / centralized services
What he thinks he’s good at:
–number of concepts he can hold in his head, and reason about
–make sure not to lose the important threads
-maybe relatively better RAM (flexibility)
-not so good at long-term storage of info and facts
His wealth / fame has changed how people interact with him, but not huge change in his day-to-day life
All the expected value is in the upside tails not in the median outcomes, and you should take that seriously – often the right path is the one that might fail As world speeds up and becomes wackier, this becomes more and more important
Acknowledge things that sound crazy and unlikely may not play out that way
When he started FTX, he was most optimistic on his team about success – thought it was 20%, team even more pessimistic
But even he was way under-estimating the upside
Straightforward EV analysis was the correct one
Most kindest thing anyone’s done for him: lot of people in effective altruism community have been dedicated and selfless, making personal sacrifices to seek the altruistic upside