In an effort to write more and to think less, I’ll be publishing something every day during November. I’ve a bunch of drafts and partially-written posts which have merit but no polish, I’m hoping these short, low-polish, daily essays will give me an opportunity to say something rather than have the ideas decay slowly.
If any of these ideas make it into a full post, I’ll link to the full post.
These mini-essays will be lower quality, poorly thought out, basically un-edited, and in general a lot more off-the-cuff thoughts. Imagine these a bit closer to Boyd’s stream-of-consciousness or late-night bar-thoughts than the other essays I write. But please let me know if any of them resonate!
AI hasn’t seen widespread adoption because the labs are focussing on automating AI R&D (8 November 2025)
There’s been some questions raised about why AI hasn’t seen more widespread adoption or impact, given how much better ChatGPT et al. are above the previous state of the art for non-human cognition. There’s certainly been lots of progress, but the current era of AI is a few years old at this point and many low-hanging fruit which feels doable with current technology is not in fact, done.
Given this is what we see, I’m fairly confident that the frontier AI companies are intentionally not pursuing the profitable workplace integrations, such as a serious1 integration with Microsoft office suite or the google workplace suite.
The reason for this, is if you had a sufficiently good AI software developer, you could get a small army of them to write all the profitable integrations for you ~overnight. So if you’re looking at where to put your data centre resources or what positions to hire for or how to restructure your teams to take advantage of AI, you emphatically do not tell everyone to spend 6 months integrating a 6-month-out-of-date AI chatbot into all of your existing products. You absolutely do pour resources into automating software engineering, and tell your AI researchers to focus on programming ability in HTML/CSS/JS and in Python. This, not coincidentally I’d argue, is what we see: most of the benchmarks are in Python or some web stack. There is also a significant amount of mathematics/logic in the benchmarks, but these have been shown to improve programming ability.
So what would we predict, if the above is true? I predict that ~none of the labs (Anthropic, Google, Facebook, OpenAI+Microsoft) will launch significant integrations that are designed for immediate business use-cases until either most of the code is written by their internal LLMs or if they see these products as a useful means of collecting data (e.g. Sora).
If this is true, then we can also infer that the leadership and stakeholders of these companies (if it wasn’t already obvious), is very AGI-pilled, and whoever’s pulling the shots absolutely believes they’ll be able to build a synthetic human-level programmer within five years. It doesn’t say anything about ASI or the AI’s ability to perform non-programming tasks, so it’ll be interesting to see if the movements of these big companies indicates that they’re going for ASI or if they just see the profit of automating software development.
While automating AI R&D is an explicit goal of some of these companies, I’m not sure whether this goal will survive the creation of a competent, cheap, human-replacement software developer. Up until this point, the steps towards “automating AI R&D” and “automating software development” are approximately the same: get better reasoning and get better at writing code, using software development tools, etc. But I’d argue that AI R&D is significantly harder than profitable software development. So for now, the companies can use the sexy “we’re automating AI R&D” tag line, but once a company builds a synthetic software developer I’m fairly certain that the profit-maximising forces at be will redirect significant resources towards exploiting this new-found power.
If you’re going to arrive late, don’t also arrive unprepared (7 November 2025)
Being late happens, because life is unpredictable and sometimes you can’t foresee everything or the cost of mitigating some risk is just too high. But often I see someone arrive late and be unprepared, frazzled, or otherwise not ready for what they’re about to do.
I spent some years as a deckhand on yachts in the Mediterranean, and sometimes myself and a crewmate would be tasked with cleaning a particular part of the ship and we’d run late. The guys who’d been working for longer than I had, taught me that if I’m going to be late, I should at least have done a good job. It’s even worth being slightly more late, but to have finished the job properly, than to be late and also have the job be incomplete.
This applies to other parts of life as well: if you’re frantically running to arrive at a meeting, but you get to the door X minutes late, it’s extremely worthwhile to spend literally 1 minute to catch your breath, clean the sweat off your brow, and collect your thoughts a bit.
Whoever is waiting for you almost certainly won’t care about being minutes late vs minutes late: they just care that you’re late. But how you walk through the door will set the tone for the meeting, so it’s worthwhile ensuring that you’ve got everything ready, that your heart has calmed down a bit. Make sure that you go into the meeting actually ready for the meeting, and not recovering from your mad dash to get there in time.
Another example is if you’re running late to a party, don’t be late and arrive without a gift. Rather be an extra 15 minutes late but arrive with a gift for the host.
Of course, this is all a trade-off. Sometimes you can afford to be an hour late, sometimes you really can’t afford to be late at all. And sometimes it’s really worthwhile arriving prepared, but other times you just need to physically be present and how you arrive doesn’t really matter. But I find that people are overwhelmingly biased towards being late and arriving unprepared, as opposed to thinking how to make the best of the situation given that they’re going to arrive late.
The only important ASI timeline (6 November 2025)
There’s lots of talk about timelines and predictions for when someone will build an AI that’s broadly more capable and intelligent than any human. These timelines range from 2 years to 20 years to never, although pretty much everyone2’s timelines shortened after the publication of ChatGPT and the subsequent AI boom.
In the discussion of AI safety and the existential risk that ASI poses to humanity, I think timelines aren’t the right framing. Or at least, they often distract from the critical point: It doesn’t matter if ASI arrives in 5 years time or in 20 years time, it only matters that it arrives during your lifetime3. The risks due to ASI are completely independent of whether they arrive during this hype-cycle of AI, or whether there’s another AI winter, progress stalls for 10 years, but then ASI is built after that winter has passed. If you are convinced that ASI is a catastrophic global risk to humanity, the timelines don’t matter and are somewhat inconsequential, the only thing that matters is 1. we have no idea how we could make something smarter than ourselves without it also being an existential threat, and 2. we can start making progress on this field of research today.
So ultimately, I’m uncertain about whether we’re getting AI in 2 years or 20 or 40. But it seems almost certain that we’ll be able to build ASI within my lifetime. And if that’s the case, nothing else really matters besides making sure that humanity equally realises the benefits of ASI without it also killing us all due to our short-sighted greed.
Detection of AI-generated images is possible, just not widely adopted (5 November 2025)
The project spearheaded by Adobe, but now implemented by OpenAI, LinkedIn, and others, has the incredibly uncatchy name C2PA. It essentially allows the creator of an image to digitally sign their images, and then embeds that signature into the image in a way that 1. follows the image around the internet when it gets shared and 2. doesn’t affect the visual image in any way. Pretty much all image files allow metadata to be attached to the image (like latitude, longitude, shutter speed, etc), and C2PA piggy-backs off of this mechanism to allow a digital signature to follow the image.
This is amazingly useful! It allows good actors to verify that a photo came from a reputable source. So if you’re a journalist and you take a photo that’s politically divisive, you can digitally sign the image as a way of attaching provenance to the image. This does not allow you to identify bad actors, since they can either strip out the C2PA metadata from the image or just take a screenshot of the original image, thus removing any history of the image. But this isn’t disastrous; browsers will flash big security-related warning signs if you try to view an HTTP website, which has lead to an assumption that every non-HTTPS website is dangerous by default. We should apply this same thinking to images: all images should be guilty until proven innocent, and social media websites should display images without C2PA metadata with big warning signs indicating that the image is untrustworthy.
So what’s the problem with C2PA? Just about every social media website strips out all image metadata from the images during upload, as a way of reducing the size and complexity of the image. This means that, while there’s reasonably good support for embedding C2PA metadata into your image, you basically can’t send that image anywhere without the metadata being stripped out behind your back.
I was pleasantly surprised to see that, when I updated my LinkedIn profile picture, it indicated that the photo had C2PA metadata. This is great! I hope this technology gets broad adoption, and that more people come to know that it exists and that it works.
An AI Manhattan project would look like Ilya, Mira, Karpathy, etc all working on secretive projects (4 November 2025)
This prediction is low-confidence, but if there were an AI Manhattan project going on, it would be tricky to pull off without raising suspicion (moreso than the original Manhattan project, due to the internet). One possible way for the US to get enough talent without raising suspicion would be to have the top AI scientists pretend to start their own stealth-mode AI companies, which is indeed what we’ve seen.
I’m pretty sure this isn’t the case, it seems unlikely that an AI Manhattan project would be started given that 1. it would be competing with existing private industry and 2. it would be nearly impossible to hide the project. If the US government did want to achieve the ends of an AI Manhattan project, it seems more likely to me that they’d nationalise the companies at the forefront of the labs, rather than build their own organisation for doing AI research.
I guess only time will tell…
The EU could hold AI capabilities development hostage if they wanted to (3 November 2025)
| Read on LessWrong |
It’s well-known that the process for building AI GPUs has a hilariously fragile supply chain. There are multiple links in the chain that have no redundancy:
- Carl Zeiss (Germany): Supplies optics/lenses for EUV lithography machines
- ASML (Netherlands): Produces the EUV lithography machines that make the chips (using Carl Zeiss’ optics)
- TSMC (Taiwan): Produces the chips (using ASML’s machines)
- Nvidia (USA): Designs the AI chips
Critically, two of these companies are based in the EU, meaning that no matter how much e/acc twitter might laugh at the EU’s GDP or bureaucracy, GPT-6 is not getting built without an implicit sign-off from the EU.
If the EU felt the need, they could halt export of EUV lithography machines out of ASML and also halt export of any EUV-empowering optics from Carl Zeiss. These companies are within the EU, the EU can do it.
This wouldn’t halt AI chip production immediately, I’m sure the existing lithography machines would keep running for a while. I’m unsure of how much regular maintenance or repair parts these machines need from ASML employees, but I’m certain it’s non-zero. So an EU-ban on exporting EUV lithography wouldn’t halt chip production immediately, but it would inevitably bring it to a halt over time.
Banning the export of EUV machines would be a gutsy move, for sure, but it’s entirely possible. And as tensions raise, it only become more likely.
Not many countries have the ability to hold the AI-capabilities world hostage, but through a bizarre twist of fate, the EU is able to do just that. I’m unsure of whether they’re aware of the power they have, given how bloated their bureaucracy appears from the outside. But this is an ace-up-their-sleeves that 1. exists, 2. could be played, and 3. isn’t going away any time soon.
AI Safety has a scaling problem (2 November 2025)
AI safety research is (mostly) done via fellowship funnels that filter out the population of the internet down to people who have shown they can do AI safety research. This process works well, but requires one-on-one mentorship from academics and researchers, and thus cannot scale. Anthropic recently had a fellowship position for which they accepted 32 people, from a total of 2000 applicants (that’s a 1.3% acceptance rate). MATS mentors were recently talking on twitter about the absurdly high qualifications of the applicants. High bars are good, but it’s not good that people who can do AI safety are not in fact doing AI safety. This is a resource constraints problem, and deserves attention to fix it. I’ve got ideas for a AI research bounty, in which anyone can submit cash prizes for completing some well-defined work, and then anyone can submit completion of that work and receive the prize. But a greater description will have to wait for the full essay.
When to be risky and when to be safe (1 November 2025)
There’s a trade-off that’s often made but rarely considered, and best described through the analogy of dating: when dating to marry4, you see some people attempt to make every date go well, and try to ensure everyone they date will like them. This seems like a reasonable goal, but I fear it collapses every potential date into a point, as though they’d respond identically. If everyone reacts the same, then it’s reasonable to play it safe and attempt to please everyone. But that’s not the case, in reality 1. everyone reacts to different things in different (often conflicting) ways, and 2. you don’t care about making everyone think you’re a chill dude (a mediocre response), you care about ensuring one person thinks you’re the best person on the planet (an extreme response). You will not get an extreme response by attempting to please everyone, you must be specific enough that you displease many people in order for you to maximally fit with one person.
This applies more generally: you can either pursue a strategy to optimise your “median” performance (never having any terrible experiences but also never having any amazing experiences), or you can optimise your “maximum” performance (often having bad experiences but occasionally having amazing experiences).
The difference in your optimal strategy comes down to the cost of failure, the number of good outcomes you need, and the bar above which you succeed5. When building a start-up, you cannot afford to play it safe, because if you take “normal” decisions then you’ll end up like “normal” start-ups, which is to say that you’ll go out of business. You must be weird, and take big risks, because the bar for success is above the median performance. You don’t need to figure out the right way to run a business a hundred times, you just need to do it once. So exploration trumps exploitation (in this example). Likewise for finding a job, you just need one company to give you an offer, you don’t care about a hundred companies thinking you’re a nice guy but not quite what they’re looking for. You’d rather 99 companies think you’re not a good fit, but one company think you’re perfect, than have 100 companies think you might be a good fit. “Might be a good fit” does not get you hired.
Footnotes
-
There do exist MS office/google workspace integrations, but they’re about as minimal as you can get away with and still the pointy-haired managers “yes MS office ‘has AI’“. These integrations are not serious, they are missing a lot of very basic functionality that leads them to be little better than copy-pasting the context into your chatbot of choice. ↩
-
everyone referring to everyone who’s watching AI developments and progress ↩
-
or the lifetime of the people you care about, which might include all future humans. ↩
-
Or otherwise with the intention of settling down with someone for more than ten years. ↩
-
I’m pretty sure these are all the same or equivalent, but I’ll explore these ideas in a full essay ↩