Podcast notes: Sam Altman (OpenAI) on AI – “One of genuine new tech platforms since mobile”

Interviewer: Reid Hoffman
Guest: Sam Altman

Not yet trillion dollar “take on Googles” startups yet – but will be a serious challenge to Google for first time

eg, Human level chatbot interface that actually works – new medical services, new education services

Idea of language interface where you say in natural language, dialogue, and computer just does it for you

Very powerful models will be one of genuine new tech platforms since mobile

How to create an enduring differentiated business
-small handful of large base models will win – skeptical of startups doing small models
middle layer will become really important – take large models, tune it, create model for medicine, or model for AI friend – will have data flywheel

Lots of AI experts think these models won’t generate net new knowledge for humanity – thinks they’ll be wrong and surprised

AI in science:
1. Science dedicated products eg Alpha Fold – will see a lot more, bio cos will do amazing things
2. Tools that make us more productive – improve net output of scientists and engineers – eg, CoPilot
3. AI that can be an AI scientist to self improve – automate our own jobs, go off and test new science and research – teaching AI to do that

What is Alignment Problem?
A powerful system that has goals in conflict with ours
How do we build AGI that does things in best interests of humanity
How to avoid accidental or intentional mis-use
AI could eventually help us do alignment research itself
Reid: will be able to tell agent “don’t be racist” and let it figure out

AI moonshots?
-language models will go much further than people think – so much algorithmic progress to come, even if we run out of compute or data
true multi modal models – every modality, fluidly move between them
-continuous learning models
These above 3 things will be huge victory

OpenAI – focus on next thing where we have high confidence, let 10% of company go and explore
Can’t plan for greatness, but sometimes breakthroughs will happen

AI will seep in everywhere
Marginal cost of intelligence and energy will rapidly trend towards zero – will touch almost everything

Metaverse will become like iPhone – a new container for software
AI will be new technological revolution – more about how metaverse will fit into AI then vice-versa

Low cost + fast cycle times is how you compete as a startup

In bio – simulators are bad, AI could help

What are best utopian sci-fi universes so far
-Star Trek is pretty good
-The Last Question is incredible short story
-Reid: Ian Banks – Culture series
-tried to write his own sci fi story, was a lotta fun

Having a lot of kids is great – wants to do it

Won’t be doing prompt engineering in 5 years
Will be text / voice in natural language to get computer to do what you want
eg, Be my therapist and make my life better; Teach me something I want to know

Reid: great visual thinker can get more out of DALL-E — will be an evolving set of human talents going that extra mile

How to define AGI
Equivalent of a median human that you can hire as a coworker – be a doctor, be a coder
Meta-skill of getting good at whatever you need

Super intelligence = smarter than all of humanity put together

Economic impacts will be huge in 20-30 years
Society may not tolerate that change – what is the new social contract
How to fairly distribute wealth
How to ensure access to AI systems (“commodity of the realm”)
Not worried about human fulfillment – we’ll always solve it
But concepts of wealth and access and governance will all change

Running largest UBI experiment in world – 5 year project

Tools for creatives — will be the great application for AI in short-term
Mostly not replacing, but enhancing their jobs

How do these LLMs differentiate from each other?
The middle layer is what will differentiate – the startups fine-tuning the base models, about the data flywheel, could include prompt engineering

SBF’s planned congressional testimony was wild: “I am, and for most of my adult life have been, sad”

Just sharing a few memorable excerpts below. Full testimony here.

b) In addition to being false, the claims do not make sense to me. Alameda Research’s own insolvency was triggered by a market crash, which in turn triggered FTX’s insolvency; it would have been absurd to create a market crash in order to take out 3AC, and then in turn bankrupt my own businesses.

7) Various claims that I created a hard-partying culture at FTX
a) Our ‘parties’ were mostly dinner and board games
b) I didn’t have my first drink until I was 21, and to my knowledge have never been drunk

b) I have a prescription for Emsam, and have for roughly a decade. I use it, daily, for its only on-label use as an antidepressant. It is not generally the case that people are expected to talk about their private medical conditions, but enough paparazzi have snapped photos of my belongings and theorized about it online that I guess I have no choice.

On Twitter, CZ claimed that “we decided to pull out as an investor” in a thread chalk full of lies.
a) In fact, I reached out to CZ in 2021 to initiate discussions about buying them out of their stake in FTX.
b) I initiated these discussions because, among other things, it was becoming increasingly difficult for FTX to operate with CZ as a significant equity owner. CZ was not cooperative in sending his KYC information to regulators that we were applying for licenses with.

c) The last few months have been difficult enough for everyone that it feels unremarkable to me, in comparison, that I need to put on the official Congressional Record that I am, and for most of my adult life have been, sad.

Podcast notes – Evolution of NLP – Oren Etzioni – TWIML: “Deep learning is the ultimate prediction engine”

Oren Etzioni – founder of AI2

Late Microsoft cofounder Paul Allen wanted to create Allen Institute for AI – hired Oren to make it happen
Paul had vision of computer revolution, relentless focus on prize of understanding intelligence and the brain

AI2’s mission is “AI for the common good”

AI2’s incubator – 20+ companies in pre-seed stage
Natural part of university lifecycle – ideas that can then grow with right resources

Created Semantic Scholar – free search engine for scientific content
New tool – help make PDFs easier to read, auto-create TLDRs for science papers

Sky Light – computer vision to fight illegal fishing

Deep learning for climate modeling – why use neural network? “Deep learning is ultimate prediction engine”

“Common Sense” project – holy grail for AI – how to endow computers with common sense
Common sense ethics are very important
eg, the paper clip creator that takes over humanity to maximize paper clip production
“Alignment problem” is part of it
Are neural nets enough? Do you need to create symbolic knowledge?
Yujin Choi’s team, Mosaic – common sense repository – a collection of common sense statements about the universe
What about when people disagree? Can relativize answers, eg, “if you’re conservative, you would think X; if liberal, think Y”, etc

“Never trust an AI demo” – need to kick tires and ask right questions
eg, Siri / Alexa – slight changes create very different responses

“You shall know a word by the company it keeps” – underlying principle of NLP

Used to think encoding grammar rules was important
But today’s tech is good at approximating those rules

What is the nature of human level intelligence?
How do we collect and understand human knowledge?

Tech that gets you to space station is different from going to Mars, different from leaving Solar System, etc

Large language models (LLMs) are doing “hallucination”, not very robust (different wording leads to different answers)
Eg, who was US president in 1492? “Columbus”

Is it a game of whack of mole? Or is there some fundamental paradigm of human intelligence?

Some experts believe our current algorithms – back propagation, supervised learning, etc – are foundation for more sophisticated architecture that could get us there
Eg, neural nets are very simple brain models

Disagrees strongly with Elon Musk’s views on AI — doesn’t believe we’re “summoning the demon” — it’s hype, not rooted in data

Neural net tuning – like a billion dials on a stereo

Science is hampered if there are third rails you’re not allowed to study or question

Steadfast in support of open inquiry

Researchers are cautious about releasing language models to public – easy to generate controversial outputs

Surprised by progress of the technology – but again, never trust an AI demo
Think about what’s under hood, implications for society

Podcast notes – Nick Johnson, ENS lead dev – Bell Curve podcast

Guest: Nick Johnson, ENS lead dev
Kiwi; Lived in New Zealand, Ireland, the UK

Goal never was to launch DAO, goal was maximally achievable decentralization — at that time there was only “The DAO” and not a lot of examples or playbook

Launched purely as governance mechanism – what to do with funds raised from registration fees?

Started with generous ETH Foundation grant

ENS is public good, non profit, goal is to leave as much value behind for users as possible

Believes protocols should all be public goods – things like ETH, L2, infrastructure
“If you want people to innovate on your platform then your platform should be open”
Makes exception for apps eg, DeFi, DEXes

Grew up with internet in 90s – open governance at the time, thing that everyone can improve on, left big impression on him

In 90s AOL didn’t succeed because it was walled garden, too much friction, extracting value that wasn’t re-invested in ecosystem

Attitude in web3 that working in nonprofit / charity gets paid badly and need sacrifice — totally reasonable to pay market rate

ENS token
-best tool we had available, but have shortcomings – eg, tendency to plutocracy
-intentionally launched as governance token, not a profit-accruing token
-wrote ENS Constitution explicitly states that revenue will not return to token holders – instead used to re-invest in ecosystem

ENS didn’t launch as a DAO – felt DAO ecosystem, tooling, examples weren’t there
What changed – use of OpenZeppelin contracts; human control of ENS had been reduced

Can be frustrating to run DAO – differing visions

Lots of parallels to corporate governance, but built more like a co-op than a for-profit
Delegation is still necessary – you don’t want token holders voting on every single decision

What if VCs buy a lot of tokens?
“Probably our biggest risk”
Defenses are social (lots of tokens given to long-term contributors, core team) and financial (would need to spend too much relative to value of $ in treasury)
“Constitution as friction mechanism”

ENS Labs is centralized, get budget and is “hired” by DAO
Hope to see other development orgs funded to help ENS (besides ENS Labs)

Quadratic voting is elegant but only works if you can solve Sybil problem (prove it’s a real human, not bots)
Imagine a good ID system, can still offer $50 to stranger on street to use their ID

Voting escrow tokens (lock tokens to get more weight or just participate in voting) – provides way to show long-term commitment to organization

Tough to design token economic / voting systems that can’t be gamed

Cares about financial privacy
KYC doesn’t reduce money laundering but does reduce financial privacy
Tornado Cash is great example – the devs actually built in compliance mechanisms – but OFAC / Dutch prosecutors ignored that and still charged them with money laundering

Optimism is public benefit corp – very encouraging
More skeptical of StarkNet (raised a lot of VC)

Just share the code, even if its messy – so many benefits from open source

Need more DAO tooling for day to day governance, a unified platform
Helpful to have OpenZeppelin’s version of the governor contractor – based on Compound’s – ENS DAO was one of first to use it

Will see more professional DAO delegates

What excites him
see DAO move to long-term vision, setup an endowment to last 50-100 years
-integrating with DNS
-offchain names – names w/o reg fees
-happy to see Ethereum hasn’t fallen into trap that Bitcoin did which resists any change

DAO provided meaningful outlet for community to contribute eg, small grants program
“Giving them permission to be involved”

Often low participation is because people have selected themselves out of it – you don’t have to be prominent to make a contribution

22 random learnings from 2022

1
As you become an adult, you realize that things around you weren’t just always there; people made them happen. But only recently have I started to internalize how much tenacity *everything* requires. That hotel, that park, that railway. The world is a museum of passion projects.
https://twitter.com/collision/status/1529452415346302976

2
…even up against powerful prescription medications like Adderall and Modafinil: sleep and all sport categories are in the top-10 for every metric, weightlifting and low-intensity exercise are ranked 1st and 2nd for “probability of having a positive effect”, and weightlifting is ranked 3rd for “probability of changing your life
https://troof.blog/posts/nootropics/

3
And so we began. At the time it felt like a fun project, but not any sort of life-changing decision. The big moments rarely do, I think, and the danger of retroactive mythologizing is that it makes people want to hold out for something dramatic, rather than throwing themselves into every opportunity.
from Sid Meier’s memoir

4
Today the CO2 exhalation of all machines greatly exceeds the exhalation of all animals and even approaches the volume generated by geological forces.
from Kevin Kelly’s What Technology Wants (I think)

5
When people look at Quentin Tarantino, they see a mad creative with a singular talent for making original movies. But Tarantino’s originality begins with imitation. He’s famous for replicating and building upon scenes from other movies, and he once said: “I steal from every single movie ever made.”
https://perell.com/essay/imitate-then-innovate/

6
This realignment would not be traditional right vs left, but rather land vs cloud, state vs network, centralized vs decentralized, new money vs old money, internationalist/capitalist vs nationalist/socialist, MMT vs BTC, and (perhaps most symbolically) Hamilton vs Satoshi
https://nakamoto.com/bitcoin-becomes-the-flag-of-technology/

7
Lessin’s five steps:
1. The Pre-Internet ‘People Magazine’ Era
2. Content from ‘your friends’ kills People Magazine
3. Kardashians/Professional ‘friends’ kill real friends
4. Algorithmic everyone kills Kardashians
5. Next is pure-AI content which beats ‘algorithmic everyone’
https://stratechery.com/2022/instagram-tiktok-and-the-three-trends/

8
The UN estimates that 5 African countries will be amongst the 10 most populated in the world by 2100:
-Nigeria
-the Democratic Republic of Congo
-Ethiopia
-Tanzania
-Egypt
https://www.pewresearch.org/fact-tank/2019/07/10/for-world-population-day-a-look-at-the-countries-with-the-biggest-projected-gains-and-losses-by-2100/

9
Yes, prices would fall and keep falling as technology and a free market did away with a false construct of needing more growth to pay for prices that were only manipulated higher through money printing in the first place. With prices falling to their natural level, and on a path to free, the entire infrastructure required to support price inflation, which was only caused by ignoring the free market, will fall away.
View at Medium.com

10
AIs can be used to generate “deep fakes” while cryptographic techniques can be used to reliably authenticate things against such fakery. Flipping it around, crypto is a target-rich environment for scammers and hackers, and machine learning can be used to audit crypto code for vulnerabilities. I am convinced there is something deeper going on here. This reeks of real yin-yangery that extends to the roots of computing somehow
https://studio.ribbonfarm.com/p/the-dawn-of-mediocre-computing

11
Bitcoin is here to stay because it is cheap and easy to bring into existence. It is a networked phenomenon that emerges out of equal peers, not unlike electricity and the internet before it. The fear of Bitcoin ceasing to exist arises out of a deep misunderstanding of the nature of these phenomena. It is akin to asking: “What if electricity goes away?”
https://dergigi.com/2022/10/02/bitcoin-is-digital-scarcity/

12
most are just technical ways to reframe the problem: play it faster, play it slower, change the order, change the instruments, add repetition, remove repetition.…They never seem to discuss or argue over these changes, they just play it to see if it works.
https://medium.com/fluxx-studio-notes/10-lessons-in-productivity-and-brainstorming-from-the-beatles-ea14385e27a4

13
In 2020, EVs made up 5.6% of all new car sales in China. Last year, the proportion was over 13%. For 2022, the data shows that EVs will comprise roughly a quarter of all cars sold. This is a case study on how quickly a market can turn

14
In practice, the freakishly specific nature of the stuff ambitious kids have to do in high school is directly proportionate to the hackability of college admissions. The classes you don’t care about that are mostly memorization, the random “extracurricular activities” you have to participate in to show you’re “well-rounded,” the standardized tests as artificial as chess, the “essay” you have to write that’s presumably meant to hit some very specific target, but you’re not told what
http://www.paulgraham.com/lesson.html

15
…opportunity and optionality are often inversely correlated. The challenge is that the greatest rewards generally go to people who are tied down in certain ways. People can only become world-class at things they commit to. Ultimately, the more hesitant people are about making commitments, the higher the rewards are for people who do.
https://perell.com/essay/hugging-the-x-axis/

16
Tezuka himself was a strong admirer of Disney animation, as were many of Japan’s pioneer animators. Even today Japanese animators are strongly aware of American animation. But, virtually from the start, postwar Japanese animation has tended to go in a very different direction, not only in terms of its adult orientation and more complex story lines but also in its overall structure. It is important to emphasize the link between television and Japanese animation in terms of anime’s narrative structure and overall style…This serial quality was also reinforced by animation’s connection with the ubiquitous manga, which emphasized long-running episodic plots as well.

17
One of the best ways of accumulating emotion is to go as rapidly as possible from one take to the next. The actor begins the second take on the emotional level he reached at the end of the first take. Sometimes I don’t even cut the camera. I’ll say quietly, “Don’t cut the camera—everybody back to their opening positions and we’re going again. OK from the top: Action!”
http://www.sheilaomalley.com/?p=36326

18
I spent a morning on a Naha beach working out with Fumiyasu Yamakawa, a one-time banker. Every day at 4:30 a.m., he cycled to the beach, swam a half hour, ran a half hour, did yoga, and then met with a group of other Okinawan seniors who stood in a circle and laughed. “Why is that?” I asked. “It’s vitamin S,” he said. “You smile in the morning and it fortifies you all day long.”
From the book Blue Zones

19
The conservative who resists change is as valuable as the radical who proposes it — perhaps as much more valuable as roots are more vital than grafts. It is good that new ideas should be heard, for the sake of the few that can be used; but it is also good that new ideas should be compelled to go through the mill of objection and opposition; this is the trial heat which innovation must survive before being allowed to enter the human race
From the book Lessons of History

20
Lorecraft is clearly a strikingly millennial school of management thinking. All the thinkers who belong in this tradition are, as far as I can tell, between about 28-35 or so. They are firmly middle-of-the-pack millennials. Founders of startups who seem to practice a sort of management by lorecraft, such as Conor White-Sullivan of Roam Research, are also in this cohort.
https://studio.ribbonfarm.com/p/lands-of-lorecraft

21
That carbon dioxide in every exhale has weight, and we exhale more weight than we inhale. And the way the body loses weight isn’t through profusely sweating or “burning it off.” We lose weight through exhaled breath. For every ten pounds of fat lost in our bodies, eight and a half pounds of it comes out through the lungs; most of it is carbon dioxide mixed with a bit of water vapor. The rest is sweated or urinated out. This is a fact that most doctors, nutritionists, and other medical professionals have historically gotten wrong. The lungs are the weight-regulating system of the body.
From the book Breath (more highlights here)

22
Van Gogh was a prolific painter. For a while, he painted a painting every single day. He still only produced 900.
Gauguin: 516. Cezanne: 1300. Picasso: 1885
https://twitter.com/swombat/status/1488306608908259331