Links for October

Reinforcement learning, Charles de Gaulle, Battle Royale

Here are the most interesting things I read, watched, and listened to in the last month. But first, some announcements:

1: Thank you to everyone who came to our latest film screening in Dublin. We are no longer in the season for it, but at some point, I would also like to experiment with running outdoor screenings.

2: You have probably noticed that “Links for September” is missing. That is for a very specific reason, which will be announced shortly.

3: I will be at Kilkenomics on November 7th and 8th, so if anyone wants to join us, I would encourage you to do so. Noah Smith will also be there, and he will be speaking at our regular Dublin reading group on November 9th.

4: I’ve been very encouraged by the response to my efforts to support the Irish Mathematics Olympiad. There are several updates about that project to share, which will be covered in a separate post.

Blogs and short links

Has my favourite restaurant – Mama Chang in Fairfax, Virginia – been involved in a major leak of military intelligence to the Chinese government?

Related: What lessons about American counterintelligence can we learn from the 1943 Tehran conference?

The significance of Frederick Douglass’s time in Ireland. Featured on Marginal Revolution.

One perspective: “Yuval Noah Harari is bringing the moment when AI surpasses human intelligence closer and closer by writing stupider and stupider books.”

Ava Huang on the friendship theory of everything. (I subscribe to this theory.)

Eventually, we will all come to love congestion pricing.

Dwarkesh Patel and Gavin Leech’s The Scaling Era: An Oral History of AI 2019–2025 is now available in hardback. I have bought four copies so far.

Dispatches from China, with Afra Wang, Jasmine Sun, and Bill Bishop. A Sinophile friend was recently planning a group trip to China, but was picking such obscure destinations that, when I suggested that I would be fine with visiting the Great Wall, I was practically laughed out of the room.

Wired magazine on the Man vs. Machine hackathon.¹ Can you guess who won? This was inspired by a paper from Model Evaluation & Threat Research (METR), which found that, while programmers on average think that AI tools make them over 20% faster, they actually make them almost 20% slower. The authors wouldn’t argue that this result will necessarily generalise,² but this does strike me as another example of the phenomenon of how self-reports of learning and efficiency are almost useless.³

Congratulations to the new cohort of Emergent Ventures winners. EV is now funding specifically archaeology-focused grants. Might we be able to recruit someone in Ireland to work on digitising and cataloguing examples of ogham?

Blog post of the month: Sebastian Garren’s whirlwind tour of Chilean economic reforms. Much more on this topic to come soon!

Thank you to Sam Enright and the Fitzwilliam for setting me on this quest.

You don’t have to choose between the environment and economic growth.

Typically, science funding bodies don’t release information about the projects that applied for grants but were unsuccessful. This seriously limits how much one can know about what kinds of research are incentivised by the current system. Here is Stuart Buck on letting unfunded grant applications see the light of day. Jordan Dworkin argues for the compromise position that scientists should be able to opt in to make their grant applications publicly visible. Personally, I like the idea that grant applications should be publicly released by default, where researchers can opt out by filling out a mildly annoying form. The experience with auto-enrollment pensions would suggest this would solve 99% of the problem.

More science policy reforms: Lauren Gilbert on the Randomized Replication Residency.

Gareth Nellis’s syllabi for his classes are absolute gems. Here, for example, is his class about social science replication.

The only countries that tax non-resident citizens on worldwide income are the United States and… Eritrea. Here is a wiki about the other financial and legal restrictions that American citizens face after emigrating, which include not being allowed to invest in the greatest tax instrument in Britain, the ISA. That is from Bogleheads, a website of people who… really like John Bogle.⁴

Speaking of things that are particularly American, something I still need to look into is airline loyalty and credit card points.

I have had bad luck in my various attempts to switch to a flip phone to replace (at least a substantial part of) my smartphone use. I now have two flip phones, both of which stopped working almost immediately. In neither case could I find an electronics shop that would repair it. Here is Karina Bao on the case for the Alcatel SMARTFLIP/GoFlip 3 as the best flip phone. Edit: I now have three flip phones.

Relevant to some of you: the AI Safety, Events and Training newsletter.

Étienne Fortier-Dubois’s historical tech tree.⁵ One of my friends had the idea to make a website almost identical to this, at almost exactly the same time. The agonies of parallel creation!

Personal website of the month: Ker Lee Yap.

From Tom McCarthy: When will quantum computing work? A good overview.

Free market economics is working surprisingly well. As Noah points out in this piece, the benefits that the Argentine economy has seen so far under Milei are probably mostly attributable to orthodox macroeconomic stabilisation policy. It’s too early to say whether the other reforms will be successful. Is an alternative title “We All Owe the IMF an Apology”?

The scientific history of origins of life research, the start of a new series.

Podcasts

Ilya Sutskever on next-token prediction. Has anybody yet applied Quine’s thesis of the radical indeterminacy of translation to the claim that a sufficiently accurate next-token predictor must necessarily understand the underlying causal dynamics that give rise to language production? Am I cooking?

Toby Ord on the graphs that AI companies would prefer you not understand. I was surprised by how confident he is that the shift toward “reasoning” LLMs is permanent (I am in no position to evaluate this claim). Here is Toby’s thread on how inference scaling reshapes AI governance. I would also second the recommendation of the scaling paradox.

Richard Sutton on why he thinks LLMs are hitting a dead end. See above for a hint about how this is largely a philosophy of language debate. When will I learn my own “bitter lesson” that I’m not smart enough to follow these podcasts over audio, and I need to switch to reading the transcripts?

Michael McAndrew’s Norming podcast has a great format: each episode is about the origin of a different norm (social, technical, or historical). Here he is on the origins of the PDF file type, and of the 12-note tuning system.

The Rest is History on Enoch Powell and the ‘Rivers of Blood’ speech. I did not know that the title of that speech comes from Virgil, which fits with Powell’s classicism. That was then, this is now: One of the most widely condemned parts of the Rivers of Blood was the racist hyperbole that, by the year 2000, 10% of the population of Britain would be non-white, a prediction which was broadly accurate. It’s also quite extraordinary that Powell spoke to many of his Indian constituents in Hindi and Urdu. There does seem to be an interesting application of horseshoe theory here, in which the foreigners who have taught themselves Indian languages are either extremely liberal or massive racists. I also listened to Enoch Powell on Desert Island Discs.

Chakravarthi Rangarajan on what’s happened to Indian monetary policy since the 1991 liberalisation. I was unaware of how much of a problem fiscal dominance was in India before the 1990s (or even really what it is).

Alice Evans on why feminism worked best in the West.

Neel Nanda on the fundamentals of mechanistic interpretability and career advice, parts one and two.

It’s always nice to see friends in the wild: Molly Mielke McCarthy on taking bets.

Papers

P.W. Anderson, More is Different: Broken Symmetry and the Nature of the Hierarchical Structure of Science. I’ve heard the title of this paper countless times before, but I never got around to reading it. The author makes an argument for anti-reductionist pluralism, which is (I think?) similar to what Daniel Dennett says in Real Patterns. It’s been a while since I thought about these issues, but from what I recall, I was sympathetic to the claim that “chemistry is just applied physics” is philosophically confused. I also read a 50-year retrospective by Steven Strogatz et al. Sociologically, it is quite fascinating that a non-philosopher managed to write such a widely discussed paper in philosophy in only four pages.

Wolfgang Schwarz, Dynamic Rationality and Disproportionate Belief. This is one of the readings I assigned for a decision theory meetup I will be running next month. I’ll definitely have to read it again; I tried to understand it while in a distractingly beautiful redwood forest in Northern California.⁶ I previously blogged about Wolfgang’s criticism of functional decision theory, which I’ve also been re-reading. His argument is that perfect rationality would not require proportioning one’s belief to the degree of evidence. I gather that one of the conflicts here is between the “rationality is systematised winning” people, and those who think that rationality doesn’t make sense unless anchored by certain axioms (with Wolfgang being in the latter camp). I don’t know enough to say, but I remember one time at lunch a few years ago – when I was introducing him to the concept of hotpot – Wolfgang gave us the following puzzle:

From Monday to Thursday, Smith believes that he has (real, physical) hands. But he’s been building a simulation device, which will create 999 subjectively indistinguishable simulated copies of himself. He will switch it on on Friday. After Friday, should Smith still believe that he has hands?

Wolfgang’s answer was yes.⁷ Sean said that he should think he has hands only until Friday. I recall I defended some intermediate position in which the mere fact that he believed it to be possible to create such a device should be priced in to his background p(simulation).

Kevin Mulligan, Helena Lenihan, Justin Doran, Stephen Roper, Harnessing the Science Base: Results from a National Programme Using Publicly-Funded Research Centres to Reshape Firms’ R&D. This is the first paper I’ve been able to find specifically about science funding in Ireland. I’m pleased that it uses actual causal inference.⁸ Science Foundation Ireland (SFI) was founded in 2000, and was essentially the first substantial scientific funding that Ireland engaged in outside block grants to be spent at the discretion of universities.⁹ SFI was quite specialised and applied, and chose a small number of areas to focus on, decided upon by a task force. It does appear that Mary Harney was quite a serious and effective Minister for Enterprise, not least of which for recruiting the former Assistant Director of the NSF to run SFI. Mulligan & co got access to a new panel dataset from 2007–2017, which found that SFI increased firm R&D, and that that R&D moved in a more applied direction.

In general, I think that this history is quite interesting as a story of institutional change, even if you have no particular interest in the Irish context. If you would be interested in writing ‘The Story of Science Foundation Ireland’ for The Fitzwilliam, I would like to commission you; please email sam [at] thefitzwilliam [dot] com.

Houses of the Oireachtas, Research and Innovation Act 2024. Perhaps the first time I’ve read a substantial law the entire way through. The Oireachtas website actually contains a wealth of information, and is quite helpful. Part 5 is the section that describes the dissolution of Science Foundation Ireland and merger into Research Ireland (see above).

Richard Sutton, The Bitter Lesson. I figured if I’m reading Sutton, I may as well get around to his famous essay, which I learned is only two pages long. Here is the lesson in question:

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin . . . We have to learn the bitter lesson that building in how we think we think does not work in the long run.

One thing I learned from Sutton is that the more general methods of building AI – that scale up compute, and eschew the symbolic representations of GOFAI – used to be called “weak methods”. People were really convinced that scaling wouldn’t work, and honestly, who can blame them?

David Silver, Richard Sutton, Welcome to the Era of Experience.¹⁰ A readable essay from the father of reinforcement learning. I read this as part of a machine learning reading group with the nice folks at Mox. They have a cool group they call the 90/30 Club, in which week-by-week, they are reading through Ilya Sustkever’s list of the 30 AI papers for which “If you really learn all of these, you’ll know 90% of what matters today.” At some point, they seem to have finished that list, and moved on to other papers. I assumed that I wouldn’t be able to follow a conversation with the legendarily “cracked” (am I using this term correctly?) San Francisco engineers, but thankfully, I was also able to listen to Sutton on the Dwarkesh podcast in preparation.

To be honest, I find the intense interestingness of the Bay Area to be overstimulating, and this contributed to low mood and distractibility over the last two weeks. Something I like about Dublin is that it feels like you can know pretty much everyone with a certain set of interests. Small ponds are underrated.

In any case, the basic argument of Silver and Sutton’s paper is that AI is now reaching a limit of what it can learn from human-generated data, and going forward, will be learning mostly from experience, trial and error, and so on. In this view, reaching superintelligence will require the fabled “paradigm shift”, and will rely heavily on reinforcement learning. This is the key graph:

They have a more detailed picture in which the most advanced AI will be steered by human desires and feedback, which I didn’t quite follow. This paper came out in April and will (eventually) be published in a book called Designing an Intelligence, so I will pre-order it once there is a release date.

This is all pretty heavy stuff, and my head hurts, so I will conclude this section with recent wisdom from my mate David:

They should call the opposite of an AI doomer a sloptomist.

Books

Julian Jackson, A Certain Idea of France: The Life of Charles de Gaulle. I don’t have many books to report on this month, because this was, in the words of one friend, “infinite” (236,000 words). It was, in retrospect, extremely ambitious to assign this for a single month’s meetup of my reading group, even if split across two sessions. The points that stood out to me the most I included in a set of discussion prompts.

I read this book because France is a gaping hole in my knowledge, and I haven’t even been there since I was nine. De Gaulle really was one of the most fascinating characters of the 20th century, and I knew almost nothing about him. Before I started the book, I didn’t even know that he essentially took power in a coup d’état.¹¹ Nor did I know about the second time that he vetoed the UK’s entry to the European Economic Community. It’s also important to get context on why on earth France left NATO’s military command structure.

In conclusion, A Certain Idea of France is as good as everybody says it is. Charles de Gaulle would have had an amazing linkspost:

While he was President, de Gaulle read two or three books a week – usually history, novels or poetry – and made a valiant attempt to keep up with contemporary literature.

Edmund Burke, Reflections on the Revolution in France. I will save remarks about this for a forthcoming essay on the Irish Enlightenment.

Music

New albums discovered, or listened to properly, for the first time this month:

Paul McCartney, McCartney III. I had somehow completely missed this home-recorded and produced album from lockdown in 2020. Found via the accompanying playlist to Tyler Cowen’s appearance on Rick Rubin’s podcast. My favourite pieces are the opener Long Tailed Winter Bird, and the closer Winter Bird / When Winter Comes. Has anyone written a list of the most enduring cultural contributions of the pandemic? Has Bo Burnham’s Inside held up?

Dmitri Shostakovich, Symphony No. 8. And the associated Sticky Notes episode. This is darker and more complicated than the triumphal Symphony No. 7, which would have been a better place to start. I think you can hear the cautious optimism about the Red Army’s advance, and in general I find it a lot easier to get into composers with specific historical episodes they are associated with (#8 premiered in 1943, #7 to 1942).

Tabla Beat Science, Tala Matrix. Another one of Zakir Hussain’s bands. If you somehow still haven’t read Shruti’s obituary for Zakir, it is the best thing I’ve found written about Indian music.

I continue to slowly work through the listening guides from The Rest is Noise and The History of Jazz. Other than that, my listening has grown stale, and I’m currently particularly open to music book/documentary/podcast recommendations. I’ve also been feeling recently that I need to pick up my bass again to stem the tide of forgetting any music theory I might once have known. Somehow, having a musician dad for whom these things come so much more naturally is rather demotivating…

Films

Only one new film to report this month. There truly is nothing like employment to crush a cinephile.

Kinji Fukasaku, Battle Royale (2000). Japanese high schoolers dropped on a mysterious island must fight to the death – what’s not to like? This was superb, and is one of my new favourite Japanese films. I had always assumed that The Hunger Games was a pastiche of Battle Royale, but the author says she never heard of it until after the book was done. I went to see this in the Roxie with Noah Smith, and I learned that this film is what originally kicked off his obsession with Japan. It was the first in a series of dominoes that led to him publishing a book about the “weeb economy”, which is only available in Japanese. It was helpful to get the macroeconomic context being satirised, of Japan in the late 1990s as an asset-rich nation with high youth unemployment, crime, and anomie. I previously blogged about the completely insane world of Japanese asset prices in the lead-up to the East Asian financial crisis.

The screening we went to had a featurette with the screenwriter Kenta Fukasaku (who is also the director’s son). They made the terrible decision of showing it before the film, which meant that the entire plot was spoiled for anyone not already familiar. The whole experience reminded me of how much better my life was made by living within a 10-minute walk of an arthouse cinema. And sadly, there doesn’t seem to be a review of Battle Royale from Scott Sumner.

From YouTube, I enjoyed (trying!) to learn about how Brownian motion and other features of statistical physics explain the ‘diffusion’ behind AI image and video generation.

1. Hat-tip Karina Bao for supplying the images.

2. Also, n=16! See Appendix C for robustness checks. Their associated blog post mentions heteroscedasticity and clustering standard errors to achieve a more reliable measure of the statistical reliability of their result, a practice which is not the norm in AI (to say the least).

3. I do often wonder whether a link roundup is an example of the “ridiculously unreliable” self-reports of learning that Justin Skycak warns about. However, I do tend to think that I would perform relatively well on a graded exam about my own corpus of writing.

4. Reading up on this has reminded me of a Marginal Revolution comment from 2023 about how John Bogle should receive the (hypothetical) Nobel Prize for the practice of economics.

5. His announcement post also contains some helpful discussion about vibecoding.

6. It seems to me that the character of Mrs Doyle captured something deep in the Irish psyche; maybe I like the misery.

7. His position can be a bit tricky to understand, but I think the idea is that if you believe you have hands, and you believe that whether you have hands won’t change its truth value – no physical beings are turning into simulations – then you must continue to believe you have hands. This would be an example of taking an axiomatic approach to rationality, in which you bite the bullet that 999 out of 1,000 subjectively indistinguishable copies of Smith will be wrong about whether they have hands.

8. This paper uses a propensity-score matching implementation of differences-in-differences methodology. You would know all about this if you read Beatriz’s excellent but admittedly niche DiD Digest newsletter.

9. Depending on how you classify the Programme for Research in Third Level Institutions.

10. The name David Silver didn’t ring a bell, but I now realise I saw him in that incredible documentary about AlphaGo.

11. I wonder if there is a George Bush joke to be made here about how the trouble with the French is that they don’t have a phrase for coup d’état. My other attempt at humour in this area was workshopping a strategy board game parody of Settlers of Catan about the Battle of France and the establishment of the Vichy regime called Settlers of Pétain.