Category: Random Facts

Recent startup, tech, AI, crypto learnings: “some of the most successful consumer products started as what we’d now call tarpit ideas. Think Facebook, Instagram, DoorDash, Yelp”

Yet, some of the most successful consumer products started as what we’d now call tarpit ideas. Think Facebook, Instagram, DoorDash, Yelp. You could argue the problems they were tackling weren’t as broadly appealing at the time, but they weren’t first in their category either. There’s hindsight bias too — they were so wildly successful that we really can’t call them tarpit ideas anymore.

Crypto’s trends from the ICO boom; to NFT summer; to socialFi, to memecoining, show me that people like to do their own research, get some sense of market advantage and then buy in size.

The iPhone’s launch in 2007, was clearly disruptive to the existing “smartphone” market controlled by Blackberry and Palm Pilot. But a few years later, the iPhone went on to disrupt nearly every industry via the creation of the mobile interface. (And we predict that a similar delayed reaction is likely to happen in generative AI as well).

The end of 2023 marked an inflection point for the presence of memes on chain. Averaging about 9,000 new tokens per day from August to November of 2023, Solana now averages more than triple that at 28,000 per day. At its peak, it topped 100,000 new tokens per day using the 30-day moving average (670,000 outright).

One general rule of technical advancement is that it’s not necessarily the most feature rich variant of a new technology that reaches the tipping point and critical mass, or even the cheapest or most available: rather, it tends to be the easiest to use.

And as Sam Altman once said, every year we get closer to AGI everybody will gain +10 crazy points.
—
In any case, people overrate the importance of open-source as we get closer to AGI. Given cluster costs escalating to hundreds of billions, and key algorithmic secrets now being proprietary rather than published as they were a couple years ago, it’ll be 2-3 or so leading players, rather than some happy community of decentralized coders building AGI.

If the last couple waves of startups felt like 10x improvements, AI provides what feels like a 100x better experience than the incumbent substitute (humans!) by compressing what is almost always the significant effort of hiring and managing another person, into a near instant experience that will only get better over time. To do this at a small fraction of the cost of hiring/managing that human dramatically opens up limitless use cases and therefore dramatically expands the market. If people underestimated the size of Uber’s market initially, we’re all underestimating the size of many AI startups’ opportunity.

The analogy here is to Search, another service that requires astronomical investments in both technology and infrastructure; Apple has never built and will never need to build a competitive search engine, because it owns the devices on which search happens, and thus can charge Google for the privilege of making the best search engine the default on Apple devices. This is the advantage of owning the device layer, and it is such an advantageous position that Apple can derive billions of dollars of profit at essentially zero cost.

Coinbase: Last piece of big puzzle is international expansion. 17% of revenue today. We’ve picked 10 markets. “Go deep markets”

Though the team used to run a marketing agency, working with brands like Casper in order to fund MSCHF projects, they stopped taking on clients last year. Now, they pretty much do whatever they want.
—
“Everything is just, ‘How do we kind of make fun of what we’re observing?’” Mr. Whaley said. “Then we have as much fun with it as possible and see what happens.”

Ethereum is a vision with a business while Solana is a business with a vision.

On the one end there are the doomers. They have been obsessing over AGI for many years; I give them a lot of credit for their prescience. But their thinking has become ossified, untethered from the empirical realities of deep learning, their proposals naive and unworkable, and they fail to engage with the very real authoritarian threat.

In the early folds of the paper—for instance, when you’ve folded it seven times and it’s still less than an inch thick—it is hard to see how it is possible that on fold fifty, a thin piece of paper could reach the sun.
—
we get frustrated when our wifi won’t transfer in two seconds what would have taken twenty minutes in the year 2000.

The project that has done the most on the former is perhaps Worldcoin, of which I analyze an earlier version (among other protocols) at length here. Worldcoin uses AI models extensively at protocol level, to (i) convert iris scans into short “iris codes” that are easy to compare for similarity, and (ii) verify that the thing it’s scanning is actually a human being. The main defense that Worldcoin is relying on is the fact that it’s not letting anyone simply call into the AI model: rather, it’s using trusted hardware to ensure that the model only accepts inputs digitally signed by the orb’s camera.

Modularization incurs costs in the design and experience of using products that cannot be overcome, yet cannot be measured. Business buyers — and the analysts who study them — simply ignore them, but consumers don’t. Some consumers inherently know and value quality, look-and-feel, and attention to detail, and are willing to pay a premium that far exceeds the financial costs of being vertically integrated.

When hype hit Snapchat, the product and growth loops had been maturing for months without any distortion from hype.

Rather, a useful way to think about generative AI models is that they are extremely good at telling you what a good answer to a question like that would probably look like. There are some use-cases where ‘looks like a good answer’ is exactly what you want, and there are some where ‘roughly right’ is ‘precisely wrong’.

The number of Chinese websites is shrinking and posts are being removed and censored, stoking fears about what happens when history is erased. China’s internet had 3.9M websites in 2023, down ~27% from 2017; Chinese-language websites were 1.3% of the global total, down 70% from 4.3% in 2013

https://longform.asmartbear.com/extreme-questions/
-If you were forced to increase your prices by 10x, what would you have to do to justify it?
-If you were never allowed to provide tech support, in any form, what would have to change?
-What would be the most fun thing to build? When we work on things that are fun, we work better and harder, yet are happy to do it.
-If our biggest competitor copied every feature we have, how would we still win?
-What if our only goal were to create the most good in the world, personally for our customers?

https://www.workingtheorys.com/p/software-creator
—
They’ll make simple software fast and at high frequency. They’ll have small teams (or no team) and engage directly with their users. Like content creators, there’ll be many kinds of software creators in many mediums. There will be short-form software creators and long-form software creators. There will be educational software creators, entertainment software creators, and lifestyle software creators.
—
We’ll see a lot more software as art, software as a game, software as an experience — not just software as a never-ending utility.
—
More software will be niche, private, personal, and local – and this will be economically rational. And just like we subscribe to our favorite content creators, we’ll subscribe to our favorite software creators too.

Late on the night of June 2nd, the Mianbi Intelligence team confirmed that the Stanford large model project Llama3-V, like MiniCPM, could recognize the “Tsinghua Simple” ancient Warring States script, “not only getting it exactly right, but also making the exact same mistakes.” This ancient script data was manually annotated by the research team after months of scanning word by word from the Tsinghua Simple, and was not publicly available, confirming the fact of plagiarism.

As an investor, the AI opportunity is obviously colossal and on a par with the invention of the internet or railroads in terms of disruption and value creation. But I think it’s likely to be “too successful” in terms of disrupting society. I believe that the effect of AI on the workforce will lead to an empowering of socialist, anti-capital dynamics in the west. So while the move is to allocate aggressively, you have to consider the reprisals to come.

Because the best possible way to find product-market fit is to define your own market.

Soon, some company will make smart glasses that sit in front of our eyes all day. We will go from 50% attention on screens to ~90%+ That’s the moment in time when the metaverse starts. Because at that moment, our virtual life will become more important than our real life.

11/ Crypto users show high present bias (~0.4) and notably low discount factor indicating a tendency toward impatience and immediate gratification

“Without a doubt, memes are becoming a universal language. Memes are shared on every social platform: Facebook, X, Instagram, TikTok, Reddit, family group chats, company Slack channels, marketing messages, celebrity feuds, etc. But if you google the market size of the meme industry, the projected value is expected to be $6.1B in 2025. This makes no sense when one meme coin (like $PEPE) has nearly that value as a market cap. Given how memes drive culture, politics, and entertainment, I’d be so bold to say that the true market size is at least 100x in size.”

one of the few people who recognized, early, the potential dangers of unaligned AGI, Musk, has switched teams, flipping from calling for a pause to going all in on—and wildly overhyping—a technology that remains exactly as incorrigible as it ever was.

Collectively, founders typically own 8–12% of max token supply with individual founders owning between 2.5–7.5% with 4–6% being most common.

Open-source will have a home wherever smaller, less capable, and configurable models are needed – enterprise workloads, for example – but the bulk of the value creation and capture in AI will happen using frontier capabilities. The impulse to release open-source models makes sense as a free marketing strategy and a path to commoditize your complements. But open-source model providers will lose the capital expenditure war as open-source ROI continues to decline.

Cailliau told Motherboard that to make this bot, he first created a large language model framework that was customized to reflect his girlfriend, Sacha’s, personality. Cailliau said he used Google’s chatbot Bard to help him describe her personality. Then, he used ElevenLabs, an AI text-to-speech software, to mimic his girlfriend’s voice. He also added a selfie tool into the code that was connected to the text-to-image model Stable Diffusion that would generate images of her during the conversation. Finally, Cailliau connected it all to Telegram using an app called Steamship, which is also the company he works at.

Software is expensive to create. You have to pay people to create it, maintain it, and distribute it. Because software is expensive to create, it has to make money. And we pay for it–software licenses, SaaS, per seat pricing, etc. Software margins have historically been an architectural envy–90+% margins and zero marginal cost of distribution.

“One of the fundamental rules of marketing is that “a confused mind always says no.”
“People won’t care about any of the success you’ve had, and they won’t follow you or your advice until they know that you’ve been where they are now.”
“If you’re neutral, no one will hate you, but no one will know who you are either.”

I still remember the days when “tokenomics” seemed cheesy af. It felt too academic, like Wall Street bros trying to enter the crypto clown arena in their suits and slacks. Now, a few short years later, tokenomics are widespread and heavily scrutinized. Attentionomics feels like a new appendage in the tokenomic arsenal.
—
Have said before that I believe the biggest memecoins will morph into unrecognizable monsters with chains, dexes, branded apps and more (imagine Pepechain). They will iteratively add utility (just as we saw last cycle w NFTs).

Open-source models have no feedback loop between production usage and model training, so they foot the bill for all incremental training data, whereas closed-source models drive compounding value with data from incremental usage.
If Meta differentiates their model based on their social graph or user feedback, they’ll want to capture that value via their closed products, and not share it with the world.

Cypherpunks participate in core Ethereum research and development, and write privacy software
Regens do Gitcoin grants rounds, retroactive public goods funding, and various other non-financial applications
Degens trade memecoins and NFTs and play games

—

Here was the last post on AI, crypto, startup, tech learnings :)

Recent startup, tech, AI, crypto learnings: Industrial Revolution freed people from using muscles, AI will free people from using brain

“Learning always wins,” said Jones. “The history of AI reflects the reality that it always works better to have a model learn something for itself rather than have a human hand-engineer it. The deep learning revolution itself was an example of this, as we went from building feature detectors by hand to letting neural networks learn their own features. This is going to be a core philosophy for us at Sakana AI, and we will draw on ideas from nature including evolution to explore this space.”

Catastrophic forgetting: a scary name for when a model forgets some of its base knowledge learned in pre-training during fine-tuning. If you run into this, there are a few ways to mitigate it.

Launch memecoin (no roadmap, just for fun) → Raise Capital → Forming a tribalistic community early on → build apps/infrastructure → continually adding utility to the memecoin without making false promises or providing roadmaps

One developer already created a Slack workspace where he and his friend hang out with a group of bots that have different personalities, interests, and skills.

In reality, Navboost has a specific module entirely focused on click signals.
The summary of that module defines it as “click and impression signals for Craps,” one of the ranking systems. As we see below, bad clicks, good clicks, last longest clicks, unsquashed clicks, and unsquashed last longest clicks are all considered as metrics. According to Google’s “Scoring local search results based on location prominence” patent, “Squashing is a function that prevents one large signal from dominating the others.” In other words, the systems are normalizing the click data to ensure there is no runaway manipulation based on the click signal.

Industrial Revolution largely freed people from using brawn
AI will largely free people from using brain

Unfortunately there are absolutely no solid predictions we can do about this stage. At the end of the day the startup just has to be lucky enough to start close enough and navigate optimally enough to hit its first discovery before company disintegrates from lack of funding or team morale. The process can be as fast as few months or as long as a decade.

Six of the eight web companion products bill themselves as “uncensored,” which means users can have conversations or interactions with them that may be restricted on platforms like ChatGPT. Users largely access these products via mobile web, as opposed to desktop — though almost none of them offer apps. On average, 75 percent of traffic to the uncensored companion tools on our web list comes from mobile.

🍰 Only 4 out of 70+ projects I ever did made money and grew📉 >95% of everything I ever did failed📈 My hit rate is only about ~5%🚀 So…ship more — @levelsio (@levelsio)

Vitalik said L3 good for customization (L2 for scaling); L3 good for specific kinds of scaling

It’s inspiring to know at any moment in time there is an infinite number of true statements for new startups to discover and further expand our collective system. Gödel’s theorem is not really about our limits: it’s about possibilities always waiting to be discovered. The process is certainly hard and alien to us.

No nation has ever become the major power without a clear lead in technology, both civilian and military. From the Roman legions, to the naval powers of Portugal, Spain and Great Britain, to Germany in World War I and the US post-World War II, great power status was achieved by those nations that were able to harness their technological advantage for holistic development of their civilian and military capabilities.

This holds at a higher level of conceptual abstraction: looking near a feature related to the concept of “inner conflict”, we find features related to relationship breakups, conflicting allegiances, logical inconsistencies, as well as the phrase “catch-22”. This shows that the internal organization of concepts in the AI model corresponds, at least somewhat, to our human notions of similarity. This might be the origin of Claude’s excellent ability to make analogies and metaphors.

Language differences mean that Chinese firms really are in the hot seat for developing domestic AI products. OpenAi’s most recent version of ChatGPT, GPT-4o, has real issues in China. MIT Technology Review reported that its Chinese token-training data is polluted by spam and porn websites.

Metaplanet becomes Japan’s top-performing stock this week, hitting a +50% daily gain limit for two consecutive days. The company plans to increase its authorized shares by 300% to acquire more BTC for its reserves.

Community is made of people, culture is made up of shared memes, community can be transient, culture is much more persistent, “community” can be formed with a free airdrop, culture can only be formed with a sustained commitment to creating a common story.

Every memecoin is an exquisitely precise ad, a self-measuring barometer of attention: the price jumps if people talk about the memecoin and drops if they don’t

McLuhan believed transformative new technologies, like the stirrup or printing press, extend a man’s abilities to the point where the current social structure must change to accommodate it. Just as the car created the Interstate Highway System, the suburb, and the oil industry, so the stirrup helped create a specialized weapon system (knights) that required land and pasture to support it and provide for training and material.

Wall Street is not going to stand idly by while Tether makes more money than Goldman Sachs.

Terminator: In three years, Cyberdyne will become the largest supplier of military computer systems. All stealth bombers are upgraded with Cyberdyne computers, becoming fully unmanned. Afterwards, they fly with a perfect operational record. The Skynet funding bill is passed. The system goes online on August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.

“Hyperscalers”, which are all looking to create a full stack with an AI model powerhouse at the top and hardware that powers it underneath: OpenAI(models)+Microsoft(compute), Anthropic(models)+AWS(compute), Google (both) and Meta (increasingly both via doubling down on own data center buildout).

Stability AI founder recently stepping down in order to start “decentralizing” his company is one of the first public hints at that. He had previously made no secret of his plans to launch a token in public appearances, but only after the successful completion of the company’s IPO – which sort of gives out the real motives behind the anticipated move.

An additional limitation of transformer models is their inability to learn continuously. Today’s transformer models have static parameters. When a model is trained, its weights (the strength of the connections between its neurons) are set; these weights do not update based on new information that the model encounters as it is deployed in the world.

All this equipment and processes consume large amounts of energy. A large fab might demand 100 megawatts of energy, or 10% of the capacity of a large nuclear reactor. Most of this energy is used by the process tools, the HVAC system and other heating/cooling systems. The demands for power and water are severe enough that some fabs have been canceled or relocated when local utilities can’t guarantee supply.
—
If we considered things in “capital cost per component” terms, and considered transistors as individual components, semiconductor fabs are actually probably among the cheapest manufacturing facilities.

Build for where models will be in 1-2 years, not where they are today. Bake the challenges of inference at scale into your roadmap. And don’t just think in terms of prompting one mega model and getting an answer back. Plan for the iterative systems design, engineering, and monitoring work needed to make your AI product the proverbial “10x better” than existing alternatives.

Ethereum’s ICO returns were 1.5x higher than available on market.Solana’s seed round returns were 10x higher than those available on market. OP’s seed round returns were 30x higher than those available on market.

Across every major ETH NFT project, more than 3/4 of all NFTs haven’t traded once in 2024.
-95% of punks
-93% of world of women
-87% of BAYC
-87% of MFers
are just sitting in wallets through this year’s moves.

We have to start by understanding the really important parts and building that core functionality first, then building additional features around that core. When you’re building consumer products, getting serious leverage in the marketplace (distribution) is the most important first order goal, so you need to accomplish this as quickly as possible and then shift gears to build second generation features.

Every VC fund with a consumer investing team is one foot in, one foot out of consumer. Even when startups hit the desired milestones & metrics, investors are still unclear which to bet on because the past decade of consumer investing hasn’t yielded many big wins, barriers to entry are low, and AI makes the future of human-tech interaction uncertain.
—
Angel investing, especially with small checks, is only good for two things: 1) getting into contractual friendships with founders you respect 2) building a track record for being a full-time venture capitalist (raising a fund or joining one).

Mustafa Suleyman has argued that the real Turing Test that matters is whether a given AI can go off and earn $100,000 for you on the internet. I would argue the test that’s more relevant — and consequential — is whether an AI can empty your inbox.

From dataset Google doc memo:
“Few know how to train efficient models” meant “Few know how to craft informative datasets.”
—
all the consumer graphics cards on the Internet could not compete with a mere thousand GPUs in a supercomputer.
—
Data cleaning, data curation, and data synthesis do not have this problem: dataset creation is a series of (mostly) parallel operations. This makes dataset creation perfectly suited to distributed computation, as one finds on AI blockchains. We can build good datasets together.

Web2 sports betting losing market share to memecoins

Fabs must limit vibrations to several orders of magnitude below the threshold of perception, while simultaneously absorbing 100 times the mechanical energy and 50 times the air flow as a conventional building.

An interesting phenomenon evident blockchain ecosystems is that the networks with the stickiest communities are the ones where a broad base of developers and users had an opportunity to benefit financially from their participation. Think Ethereum and Solana, which have two of the strongest developer communities: the native tokens were publicly available at a much lower price to the current value. In contrast, ecosystems where network tokens launch at a highly efficient market price tend to struggle to retain a passionate community of developers and users, to the long-term detriment of the ecosystem.

Bitcoin surpasses 1 billion confirmed transactions, averaging over 178,000 transactions per day since its launch in 2009.

We are relatively cheaper and don’t bill by the hour. We get more done. We hire and fire firms. CEOs trust _us_.
As a result, in-house lawyers have grown 7.5x times the rate of other kinds of lawyers the last 25 years. The role of “product counsel” boomed, just like the role of product manager in this time.
Today Google employs 828 “product counsel.” That’s more than only the biggest law firms.

Number one predictor of job retention is whether they have a friend at work

We don’t sell saddles essay
-The best — maybe the only? — real, direct measure of “innovation” is change in human behaviour. In fact, it is useful to take this way of thinking as definitional: innovation is the sum of change across the whole system, not a thing which causes a change in how people behave. No small innovation ever caused a large shift in how people spend their time and no large one has ever failed to do so.
-Because the best possible way to find product-market fit is to define your own market.

Transformers’ fundamental innovation, made possible by the attention mechanism, is to make language processing parallelized, meaning that all the words in a given body of text are analyzed at the same time rather than in sequence.

I’ve been making chatbots since the days of AI Dungeon, and have seen the cycle multiple times. A new site appears with low censorship and free content generation. It grows a user base, starts introducing more censorship, raises prices, and before long it becomes unusable and people move on to the next one. Poe has been around for longer that most and I’m only seeing improvements on it. Plus it’s operated by Quora, which I think will give it added sustainability.

Friendtech is uniswap for social tokens

Steve Jobs figured out that “you have to work hard to get your thinking clean to make it simple. – Taleb

I eventually think these open-source LLMs will beat the closed ones, since there are more people training and feeding data to the model for the shared benefit.
Especially because these open source models can be 10 times cheaper than GPT-3 or even 20 times cheaper than GPT-4 when running on Hugging Face or locally even free, just pay electricity and GPU

In a 1985 interview Wozniak posited: “The home computer may be going the way of video games, which are a dying fad” – alluding to the 1983 crash in the video game market. Wozniak continued:
“for most personal tasks, such as balancing a check book, consulting airline schedules, writing a modest number of letters, paper works just as well as a computer, and costs less.”
—
He seemed well aware of the heretical nature of his statements, telling a reporter: “Nobody at Apple is going to like hearing this, but as a general device for everyone, computers have been oversold”and that “Steve Jobs is going to kill me when he hears that.”

Bonus (Reality Check): What Are The Odds You Get Acquired Within 5 Years for a Good Price? Around 1%-1.5% by Jason Lemkin
Data on 3,067 startups founded in 2018. The takeaway: It’s the second 5 years where the real value starts to compound. Startups are a long game

The subset of parameters is chosen according to which parameters have the largest (approximate) Fisher information, which captures how much changing a given parameter will affect the model’s output. We demonstrate that our approach makes it possible to update a small fraction (as few as 0.5%) of the model’s parameters while still attaining similar performance to training all parameters.

Random facts 10 – things I learned recently – “Unspoken expectations are premeditated resentments”

“Bull markets are born on pessimism, grown on skepticism, mature on optimism and die on euphoria. Maximum pessimism is the best time to buy, and maximum optimism the best time to sell.” — John Templeton

Stochastic Gradient Descent (SGD) is employed as an optimization algorithm to update the model’s parameters, aiming to minimize the error. SGD updates parameters for each training example or small batches thereof, moving in the opposite direction of the error gradient

Russia’s Muslim population accounts for roughly 20 percent of the population. More importantly, the fertility rate of Russia’s Muslims is around 2.3 – much higher than the overall national fertility rate of 1.7. Add in immigration from a predominantly Muslim Central Asia, and the demographic picture is that of a rising Muslim population…while Putin doubles down on Russia as the defender of the Orthodox faith

Although ranging companies had been informally used in the 1820s, the Texas Rangers were formally created by the Texas legislature to be highly mobile Indian fighters.
The Rangers deservedly achieved a mythological status in American history. They showed a remarkable frontier adaptability, of necessity adopting many of the same characteristics as their Indian enemies. Speed, stealth, and a capacity for the cold-blooded execution of Indians.

But when we are at our best, we’re not slogging through. Great people are obsessed and they’re not slogging through either. They are driven. They are motivated. They are deeply, deeply engaged

Time is the friend of the consistent and the enemy of the inconsistent.

Lying Is Not the Way: “The best way to achieve AI safety is to grow AI to be really truthful.” – Elon Musk

A report on enterprise buying/usage of generative AI. Some takeaways: Budgets for AI are moving from experimental to recurring, enterprises prefer using more than one model, enterprise’s like open source for control, internal use cases are easier to implement vs external

This brings me to the core idea of degen communism: a political ideology that openly embraces chaos, but tweaks key rules and incentives to create a background pressure where the consequences of chaos are aligned with the common good.

Note that baseball, hockey, the Olympics, golf, etc., are not seeing booming TV ratings. Why? They are too hard to gamble on.
And if you watch closely, all these sports are desperately trying to educate people on how to bet on their sports

0DTE (zero-day to expiration options, or they expire that day on the close) is now over half of all options traded

But the best example of all and one of the greatest jobs of marketing that the universe has ever seen is Nike. Remember, Nike sells the commodity. They sell shoes. And yet when you think of Nike you feel something different than a shoe company. In their ads, as you know, they don’t ever talk about the product, they don’t ever tell you about their air soles and why they’re better than Reebok’s air soles. 
What is Nike doing in their advertising? They honor great athletes and they honor great athletics

All of the best startup leaders I’ve met are highly attuned to their competitors’ moves, and the others get eaten alive. Zuckerberg famously is a total psycho about competition.

Teams that have had good leaders just run better – they make better decisions, they are more resilient, and perhaps most importantly, they have higher standards for others and themselves. Teams that have had poor leaders are predictably the opposite. They take longer to get less done, and more disappointingly they often have a bitter attitude. Leadership quality is durable and highly contagious.

First, people’s minds wandered a lot. About half the time, the average person is thinking about something other than what they are doing.
And second, mind-wandering isn’t that much fun. People tended to be less happy when their minds were elsewhere. This is surprising, especially considering how much mind-wandering is voluntary. Apparently, when we can think about whatever we want, we often go to dark places

They found a surprising result: Large language models (LLMs) often use a very simple linear function to recover and decode stored facts. Moreover, the model uses the same decoding function for similar types of facts. Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables

The chest should neither expand nor collapse during proper inhalation or exhalation.

GPT-3 represents words with an embedding dimension of 12,288. If you were to do a single matrix multiplication of this size on-chain, it would cost about $10 billion at current gas prices — the computation would fill every block for about a month straight.

“We can be retarded longer than you can be solvent”

Plato described ecstasis as an altered state where our normal waking consciousness vanishes completely, replaced by an intense euphoria and a powerful connection to a greater intelligence.

The container made shipping cheap, and by doing so changed the shape of the world economy. The armies of ill-paid, ill-treated workers who once made their livings loading and unloading ships in every port are no more, their tight-knit waterfront communities now just memories.

The particular egoic patterns that you react to most strongly in others and misperceive as their identity tend to be the same patterns that are also in you, but that you are unable or unwilling to detect within yourself.

“Increasing model width (hidden dimension size)” = expanding the capacity of a neural network by enlarging the number of units (neurons) in the hidden layers.
Model Width = size of the hidden layers within a neural network. A “wider” model has more neurons in each hidden layer. The “hidden dimension size” specifically refers to the number of neurons or units in a particular layer.

Some people appeared to be essentially burnout-proof. In a few cases, I observed people working for years with near-permanent on-call and consistent 60+ hour weeks (with spikes to 80+), including through major events like weddings and honeymoons. They appeared no worse for wear than others working half as much.
—
I noticed that many parents of young children, despite having significantly more on their plates, seemed to get burnt out less. I even noticed this in myself, and didn’t have a real way to explain it – my first kid’s birth coincided with the busiest working period of my life (do not recommend), but I found that I had a more positive attitude towards work, for no reason that I could really explain. When I read this post, it all clicked – when you have young children Mission Doubt entirely disappears because you need to feed them.
—
Memory is not an accurate recording, and despite what many of us think, our recollection of the past is highly distorted. False memories are not difficult to implant, and if you think you have an accurate memory of something that happened a year ago, you are almost certainly wrong.
—
Studies of people from dozens of different countries find that, as we pass middle age, we get more agreeable, more conscientious, and less neurotic.

From the previous section, we know that the market size of “solving business problems” is much much bigger than just Software Development. So there’s no reason to believe that Software Development will disappear anytime soon.

Work like a CEO
Train like an athlete
Study like a student
Create like an artist

One of the best nuggets of advice he offered, beyond all the practical advice on how large to draw the cartoons and what kind of Bristol board to draw them on and other such details, was to take pride in rejection letters. He recommended hanging them on the wall and celebrating them. Some people might quit after one or two ideas are rejected, but he knew the important thing was to keep trying. It was brilliant advice.
—
Artistically, the comic achieved everything I believed comic strips were meant to achieve: it was easy to read, it was drawn simply so it popped off the page, and it had recurring characters that were clear archetypes that readers could look forward to seeing every day and eventually fall in love with.
—
Another thing I’d learned about the comic strip business from Ken Muse is that it’s very difficult for a newspaper to stop running a comic strip that’s popular. Newspaper editors are terrified of angry letters, so once a comic strip gains a foothold with an audience, it tends to stick around for a long time. This is why comic strips are the most conservative entertainment medium. This is why there are still comic strips in the newspaper today that are unchanged from the 1920s and 1930s when they first debuted.

Party Over All: The highest-status person at any university in China will not be the top scientist; instead, it will be the person with the strongest connections to the CCP

While we naturally understand that writing is a good way to share ideas with others, we under-appreciate just how much good writing helps us think about an idea ourselves. Writing is not only a means of communication, it enables us to practice reasoning.

Unspoken expectations are premeditated resentments

The single most powerful thing you can do in a relationship, whether it’s personal or professional, is to give someone 100% of your attention

I learned that there is no correlation (or indeed, if there is, it is a negative correlation) between how satisfying a session felt and how interesting the resulting material was. Some of my now favourite music came from the most frustrating sessions

In 1910, Korea was annexed by the Empire of Japan after years of war, intimidation and political machinations; the country would be considered a part of Japan until 1945. In order to establish control over its new protectorate, the Empire of Japan waged an all-out war on Korean culture
—
Today, March 1—the day of the independence protest in 1919—is a national holiday in South Korea, a reminder not just of the resilience of the Korean people, but of the years of occupation they withstood.

Plain Folks Fallacy: People of authority acquiring trust by presenting themselves as Average Joe’s, when in fact their authority proves they are different from everyone else.

What Scale is doing with Remotasks isn’t hard to replicate. Kevin Guo, cofounder of Hive, a startup that once fielded its own Remotasks rival before shuttering it due to tough margins, contends that the sort of data labeling Scale does is a commodity business. “Anyone who puts up a team can compete with you, and it comes down to price really quickly,” he says

Telling the truth to customers and employees, especially when it’s difficult, is how you earn trust and loyalty

Just as no one in the container’s early years imagined a ship that could carry the freight moved by ten thousand trucks or dreamed that the world’s ports would soon be handling the arrival of two million 40-foot containers every week, so, too, did no one conceive that steel shipping containers could be turned into houses and sculptures or that abandoned containers would become a serious nuisance. That simple metal box was what we today label a disruptive technology.

—
Killer apps
One of the most important concepts in software startup strategy is killer apps. Each platform ecosystem is surrounded by a set of applications of varying degrees of importance – the best of those are the killer apps.
In sales the system of record is the CRM, the database in which all customer data lives. Salesforce of course built many important applications around it, and integrates with thousands more. HubSpot began with applications around inbound marketing before backing into the CRM later. In HR you have the HRIS. Lattice started with one important HR application, performance management, before building several more, and then eventually an HRIS. When the iPhone first launched it spawned a huge number of new applications, some of which it built itself and some which were built by third parties. Today we’d consider the killer apps things like messages, maps, musics — in most cases, there are both native and external products offered.
One of the reasons AI is so important for the software landscape is because it shuffles the deck of possible killer apps. It opens up entirely new functionality that was simply impossible to build before; an automated sales rep, a robot bug catcher and fixer, a support agent that can triage many cases automatically.
—

A bitter–and highly popular–internet posting called corrupt officials ‘the new black-collar class’, who concealed their wrongdoing in a shroud of secrecy. ‘They drive top-brand cars. They go to exclusive bars. They sleep on the softest beds in the best hotels. Their furniture is all of the best red wood. Their houses overlook the best landscapes, in the quietest locations. They play golf, travel at the public expense, and enjoy a life of luxury,’ said the anonymous blog, posted in July 2009. ‘They are the newly arrived “black-collar class”. Their cars are black. Their income is hidden. Their life is hidden. Their work is hidden. Everything about them is hidden, like a man wearing black, standing in the black of night.’

“If anything ail a man,” says Thoreau, “so that he does not perform his functions, if he have a pain in his bowels even … he forthwith sets about reforming—the world.”

“Convenience on the internet is basically achieved by two things: speed, and cognitive ease.” In other words, people don’t want to wait, and they don’t want to think — and the internet should respond to that.

Without compassion, you are utterly alone. That is why compassion is the ground of happiness.

“Peranakan” generally refers to a person of mixed Chinese and Malay/Indonesian heritage.
Many Singapore Peranakans trace their origins to 15th-century Malacca, where their ancestors were thought to be Chinese traders who married local women.

In May, China’s space agency announced that it will land astronauts on the moon by 2030, making it the second country with that capability. It’s rare for Beijing to lay out formal timelines unless it’s quite confident that it has the task in hand
—
the party’s strangling of free expression has rendered China into a pitiful underperformer relative to Japan and South Korea in the creation of cultural products. What are the great Chinese creations of the last 20 years, aside from a science fiction trilogy published before Xi took office, a short-video app that doesn’t display Chinese content overseas, and a video game that looks as if it’s thoroughly Japanese

Elon is one half a manufacturing visionary, able to do things with rockets, automobiles, and satellites that no one previously imagined; his other half is a pure gremlin on the public consciousness, who uses his Internet following to drive the rest of society towards madness. It’s not just the Internet that pays attention to his doings: Elon more reliably generates mainstream news headlines than perhaps even the two presidential candidates this year. Who else has become a fixture on every pillar of American imperium: tech in San Francisco, finance in New York, movies in LA, energy in Texas, and government in DC

Most people had not honoured even a half of their dreams and had to die knowing that it was due to choices they had made, or not made
—
However, although people may initially react when you change the way you are by speaking honestly, in the end it raises the relationship to a whole new and healthier level. Either that or it releases the unhealthy relationship from your life. Either way, you win
—
Many did not realise until the end that happiness is a choice. They had stayed stuck in old patterns and habits.

It was also built with a vast outsourced workforce that performs a rudimentary task crucial to AI: labeling the data used to train it. Those people—some 240,000 of them in countries including Kenya, the Philippines and Venezuela—work for Remotasks, a subsidiary Scale doesn’t mention in public marketing materials

False-Consensus Effect: Overestimating how widely held your own beliefs are, caused by the difficulty of imagining the experiences of other people.

TechCrunch found that the GPT Store, OpenAI’s official marketplace for GPTs, is flooded with bizarre, potentially copyright-infringing GPTs that imply a light touch where it concerns OpenAI’s moderation efforts. A cursory search pulls up GPTs that purport to generate art in the style of Disney and Marvel properties, serve as little more than funnels to third-party paid services, advertise themselves as being able to bypass AI content detection tools such as Turnitin and Copyleaks

I think the big markets catalyst will be Robotic McDonalds. The restaurant industry has about 12.5 million employees. Imagine if most of that got automated, and a couple of other low skilled industries. You get to 10-15% unemployment very quickly

Arms for your chairs are chairs for your arms

From the book How to Change by Katy Milkman:
Across data set after data set, we found the same patterns. Undergraduates at a campus fitness center were more likely to visit the gym not only in January, but also earlier in the week, after a school holiday, at the beginning of a new semester, and after their birthdays… Similarly, in January, on Mondays, and after holiday breaks, we documented an uptick in online goal setting…

When you let your mind wander, it wanders to whatever you care about most at that moment. So avoid the kind of distraction that pushes your work out of the top spot, or you’ll waste this valuable type of thinking on the distraction instead. (Exception: Don’t avoid love.)

The iPhone is one of the most successful and important products of the past few decades. But the first version launched with a mountain of issues. It was a terrible phone, ironically. The whole idea was that it was a “smart phone,” yet everyone agreed their cheap-o Nokia flip-phone was ten times better at being a phone. Also, imagine launching an operating system that didn’t include “copy/paste.

Pros:
Devin feels UI/UX first, not Gen AI first. IDK if this makes sense. The AI is a core component but it’s the surrounding infra they built that is the star of the show
The product feels quite finished, This is not a demo. They have things built out such as auto deploy to netlify, api key protection, intelligent way to interrupt without interrupting, a good UI that is *tailored to humans* and bridges LLM and human dev, the slider to move backwards in time

Ludic Fallacy: Falsely associated simulations with real life. Nassim Taleb: “Organized competitive fighting trains the athlete to focus on the game and, in order not to dissipate his concentration, to ignore the possibility of what is not specifically allowed by the rules, such as kicks to the groin, a surprise knife, et cetera. So those who win the gold medal might be precisely those who will be most vulnerable in real life.”

As soon as a startup starts operating in the real world, the vast majority of factors will be outside of a startup direct control or even influence, while having absolutely dominating impact on that startup eventual outcome

There is, however, nothing else remotely like an elephant. (Its closest living relatives are sea cows — dugongs and manatees — and the hyrax, an African shrewmouse about the size of a rabbit.)

They must know how to kindle and fan an extravagant hope. It matters not whether it be hope of a heavenly kingdom, of heaven on earth, of plunder and untold riches, of fabulous achievement or world dominion.

probably nothing is more universal than the experience of stress. Nobody gets through life without experiencing physical pain, illness, disappointment, anger, or loss. The specifics may vary, but the underlying experience is as human as it gets. The challenge is to remember this in your own times of suffering.

(Most westerners seem unaware that MSG is naturally present in Parmesan cheese and other ingredients commonly used in western cuisines.)

The framing of “Attention Economy” is limiting. I believe we live in a Love Economy. People now want to spend their time, money, and attention on things they love, not simply things that grab their attention.

these firms say they want entrepreneurs but they really want franchise operators. there’s some formula and you’re basically operating a tech enabled McDonalds that produces a number of widgets
“politics” or “corporate culture” are basically mechanisms to ensure that you are a script follower, the same way that McDonalds wants to ensure that its franchises are operating according to spec. there are some semi subversive scripts – pioneered by the Paypal mafia, or Solomon brothers – which value creativity. But they’re still ultimately scripts

You gain confidence and eliminate anxiety by doing gradually more difficult tasks, excelling at them, and realizing you are a competent, capable person

The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds…instead we should build in only the meta-methods that can find and capture this arbitrary complexity
—
We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.

It was one of Rikyū’s achievements to take an act which in the West is one of the most routine and unremarkable activities and imbue it with a solemnity and depth of meaning akin to a Catholic Mass. Every aspect of the tea ceremony, from the patient boiling of the water to the measuring out of green tea powder, was coherently related to Zen’s philosophical tenets about the importance of humility, the need to sympathise with and respect nature and the sense of the importance of the transient nature of existence.

The lack of energy is palatable. There is performance upside everywhere. As a leader, your opportunity is to reset in each of these dimensions. You do it in every single conversation, meeting, and encounter

Prior editions:

Learning ergodicity with ChatGPT and an amusement park

I was reading this book, a compilation of essays about bitcoin and capitalism, and I was struggling to understand the concept of ergodicity.

So I asked ChatGPT to explain it with a simple analogy, and I thought the explanation was good enough to share. If it’s wrong in some way, let me know:

Imagine a vast, sprawling amusement park with many different attractions and rides. Each ride represents a different state or condition that a system (like an economy, a molecule in a gas, or even a person’s life) can experience.

Now, imagine two ways to experience this park. One way is as a single visitor: you spend the whole day going from ride to ride, trying each one. Over time, you experience everything the park has to offer. Your single journey through the park represents a time average; that is, you’re sampling all the possible states over a period of time.

The other way to experience the park is by looking at a snapshot of all the visitors at one moment. Instead of one person experiencing all the rides, you see thousands of people, each on a different ride. This snapshot represents a space average, where you see all the possible states (rides) at one instant through the experiences of many different visitors.

A system (or park) is ergodic when the long-term time average (your full day at the park) is the same as the average experience of many people at a single moment (the snapshot of everyone). In other words, what you experience over time is the same as what everyone experiences collectively at one point.

In practical terms, if a system is ergodic, studying a single part of it over time can give you a good understanding of the whole system at any given moment. If it’s not ergodic, then the individual experience might be quite different from the average experience, and more complex methods are needed to understand the whole system.

I like to ask ChatGPT to generate simple analogies and metaphors for complex concepts, because I find them easier to grasp and if needed, explain to others. This method misses some details and nuances, but 75% understanding is better than whatever it was prior…

Unfortunately, I couldn’t get dalle3 to generate an appropriate corresponding image (though that capability *has* to be possible in the near future). Midjourney was slightly better so I’m including here:

Rollercoaster Ergodic Ergodicity

Random facts — things I learned (Dec 7 2023) — “The spirit of the individual is determined by his dominating thought habits”

“When 120 of Cornelius Vanderbilt’s descendants gathered at Vanderbilt University in 1973 for the first family reunion, there was not a millionaire among them.”

Repeat after me: Your rent is the maximum you’ll pay. Your mortgage is the minimum you’ll pay. – Ramit Sethi

Bitcoin: scarcer than gold, faster than fiat – Lyn Alden

Humans are both master and slave to the technium, and our fate is to remain in this uncomfortable dual role. Therefore, we will always be conflicted about technology and find making our choices difficult. – Kevin Kelley

The spirit of the individual is determined by his dominating thought habits

There are 3 Greek gods of dreams (Oneiroi):
Phobetor was thought to be the bringer of nightmares, and had the ability to appear as animals or monsters; Phantasos was believed to bring surreal and strange dreams, and was able to appear as inanimate objects, such as stones or wood. In contrast to his two siblings, Morpheus brought messages and prophesies from the gods to mortals through the medium of dreams.

What, then, is the Singularity? It’s a future period during which the pace of technological change will be so rapid, its impact so deep, that human life will be irreversibly transformed. – Ray Kurzweil

One often reads predictions of the next several decades discussing a variety of demographic, economic, and political trends that largely ignore the revolutionary impact of machines with their own opinions and agendas. – Ray Kurzweil

Discipline is the strongest form of self-love.
*It means choosing what you want later over what you want now.*
Discipline shows how committed you are to your dreams, even on tough days.
Discipline is honoring the covenant you have with yourself. That’s your personal integrity.
Your future self depends on your current self to keep promises made yesterday.
-Ram Ahluwahlia

6/ “highly individualistic, self-obsessed, control-oriented, nonconformist, and analytical.”
👉🏻Culture, independent of the biological evolutionary process, does “alter our brains, hormones, and anatomy, along with our perceptions, motivations, personalities, emotions, and many
-Jim O’Shaughnessy

The researchers, Robert J. Jackson Jr. of the New York University School of law and Prof. Joshua Mitts of Columbia Law School, found a sharp and significant rise in short sales of ETFs that track Israeli companies in the days preceding the attack

The desire to avoid feeling anxious overtakes other goals. In the worst cases, people organize their lives around avoiding anything that provokes anxiety. And while they hope that this will make them feel safe, it tends to have the opposite effect.
I know this well with weed

The principles of operation of software advances such as speech recognition can be written in just a few pages of formulas. Often a key advance is a matter of applying a small change to a single formula. The same observation holds for the “inventions” of biological evolution: consider that the genetic difference between chimpanzees and humans, for example, is only a few hundred thousand bytes of information.

“In January 2012, I beat Nadal in the finals of the Australian Open. The match lasted five hours and fifty-three minutes. Many commentators have called that match the single greatest tennis match of all time. After I won, I sat in the locker room in Melbourne. I wanted one thing: to taste chocolate. I hadn’t tasted it since the summer of 2010. Miljan brought me a candy bar. I broke off one square—one tiny square—and popped it into my mouth, let it melt on my tongue. That was all I would allow myself. That is what it has taken to get to number one.” – Djokovic

Don’t be in a hurry to “fix” things; rather, enrich your understanding in the evergoing process of discovery and finding more the cause of your ignorance – Bruce Lee

In any bond of depth and significance, forgive, forgive, forgive. And then forgive again – Maria Popova

When some part of a doctrine is relatively simple, there is a tendency among the faithful to complicate and obscure it. Simple words are made pregnant with meaning and made to look like symbols in a secret message. There is thus an illiterate air about the most literate true believer. – Eric Hoffer

“Life tastes better when you’re chugging” – Ricky (and Morty)

Jacobs: He gets the value of improvisation. “In jazz, if you hit a wrong note, there’s no such thing as a wrong note. That’s the note, that’s the reality. You radically accept that, and you build on it,” he riffs. “Music is really business. …You have to be using all of your senses at the same time, and you have to be dancing with the circumstances and evolving.”

Here’s a different interpretation: a politically connected billionaire on the verge of losing his shirt called a government official—one who used to be the CEO of the company he now owned a chunk of—and convinced him to use taxpayer funds to prop up the stock market, particularly his stocks. – on Charlie Munger

In the aftermath of the moon landing, approval for that specific mission didn’t meaningfully budge. 47% said it was worth it a decade later, in 1979 and it would take 20 years for amnesia to set it and this number to reach 77% in 1989. Meanwhile opposition to further moon missions remained higher than support for one until at least the mid-1990s

he’d been exposed to the often forgotten history of technological pessimism as an under-graduate at Columbia University. This experience lead to an epiphany which he eloquently recalled:
“I discovered, to my amazement, that all through history there had been resistance…and bitter, exaggerated, last-stitch resistance…to every significant technological change that had taken place on earth.

they found that the more seasoned jumpers weren’t calmer than those about to take their first leap. Instead, the experienced skydivers showed even higher heart rates before and during the jump. The more pumped they were to take the dive, the bigger their excite-and-delight response. When you need to take a leap and want to do well, don’t worry about forcing yourself to relax. Instead, embrace the nerves, tell yourself you’re excited, and know that your heart is in it. – Kelly McGonigal’s amazing book on stress

By emphasizing the importance of reducing stress before an exam, his advice only further confirmed what students feared: Anxiety was a sign that they would do poorly. If you want to help people better cope with anxiety, a more useful strategy might be to simply tell them that you think they can handle it. Studies show that when people are told, “You’re the kind of person whose performance improves under pressure,” their actual performance improves by 33 percent. – more Kelly

How do you expect to manifest what you want if you’re afraid to openly broadcast that you want it?
If you can’t broadcast your desires, it’s fair to say you don’t own them yet. How can your desires become real if you can’t speak up about them? If you’re going to receive them, then let it be known. If you find it necessary to hide what you desire, that suggests you aren’t ready to receive. – Steve Pavlina

Why write in public? “The ability for smart, useful observations to get into the hands of people with fewer ideas but lots of capital has never been better. You can build both a reputation and balance sheet this way.” – Ramit Sethi

A lot of visible problems that you can’t seem to solve are secretly solutions you don’t want to admit to adopting to problems you don’t want to admit to having. – a mensch on Twitter

However, the pain of sitting for weeks at a time, distilling all of that knowledge into a series of 6, concise essays, was a stark reminder that feeling that you know something, and writing it down, are two vastly different things, by many orders of magnitude

The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it. – Mark Weiser

And what I’ve discovered is that from a broad perspective, people are basically the same. Everyone spends most of their time worrying about food, money, their job and their family — even people who are rich and well fed. Everyone wants to look cool and feel important — even people who are already cool and important. Everyone is proud of where they come from. Everyone has insecurities and anxieties that plague them, regardless of how successful they are. Everybody is afraid of failure and looking stupid. Everyone loves their friends and family yet also gets the most irritated by them. – Mark Manson

Factory Meat: “The goal of feed-lot animals is to blow them up fast and quickly with cheap food. That’s what we are doing to our people, too.” – Will Harris

The reaction to Fukushima was completely overblown: it is debated whether zero people or one person died from any sort of radiation at Fukushima

Our model of what’s important in SEO has been wrong for a long time. We thought content was the basis, backlinks the middle layer and user signals sprinkled on top. It turns out user signals were the basis, with a middle layer of backlinks and content understanding sprinkled on top

Jensen Huang


If you want to be successful I would encourage you to grow a tolerance for failure
But the thing about failure is this: if you fail often enough, you actually might become a failure, and that’s different from being successful
So the question is how do you teach someone how to fail, but fail quickly — and to change courses as soon as you know it’s a dead end
And the way to do that is we call it intellectual honesty — we assess on a continuous basis whether something makes sense or not. And if it’s the wrong decision, let’s change our mind

Biggest regrets I have are almost exclusively things I did *not* do. — Sam, 47