What are the actual dangers of advanced AI?

That is the focus of my latest Bloomberg column, 2x the normal length.  I cannot cover all the points, but here is one excerpt:

The larger theme is becoming evident: AI will radically disrupt power relations in society.

AI may severely limit, for instance, the status and earnings of the so-called “wordcel” class. It will displace many jobs that deal with words and symbols, or make them less lucrative, or just make those who hold them less influential. Knowing how to write well won’t be as valuable a skill five years from now, because AI can improve the quality of just about any text. Being bilingual (or tri- or quadrilingual, for that matter) will also be less useful, and that too has been a marker of highly educated status. Even if AIs can’t write better books than human authors, readers may prefer to spend their time talking to AIs rather than reading.

It is worth pausing to note how profound and unprecedented this development would be. For centuries, the Western world has awarded higher status to what I will call ideas people — those who are good at developing, expressing and putting into practice new ways of thinking. The Scientific and Industrial revolutions greatly increased the reach and influence of ideas people.

AI may put that trend into reverse.

And on arms races:

If I were to ask AI to sum up my worries about AI — I am confident it would do it well, but to be clear this is all my own work! — it might sound something like this: When dynamic technologies interact with static institutions, conflict is inevitable, and AI makes social disruption for the wordcel class and a higher-stakes arms race are more likely.

That last is the biggest problem, but it is also the unavoidable result of a world order based on nation-states. It is a race that the Western democracies and their allies have to manage and win. That is true regardless of the new technology in question: Today it is AI, but future arms races could concern solar-powered space weapons, faster missiles and nuclear weapons, or some yet-to-be-invented way of wreaking havoc on this planet and beyond. Yes, the US may lose some of these races, which makes it all the more important that it win this one — so it can use AI technologies as a counterweight to its deficiencies elsewhere.

In closing I will note for the nth time that rationalist and EA philosophies — which tend to downgrade the import of travel and cultural learning — are poorly suited for reasoning about foreign policy and foreign affairs.