29 Comments
User's avatar
Claire Hartnell's avatar

If you take a biological perspective, you can view all organisms/entities emerging from recombination & mutation. As new phenotypes (wholes) emerge, they face selection pressures (policy, competitors, technology etc). Stuart Kauffman describes the ‘adaptive landscape’ these organisms must traverse to find higher levels of ‘fitness’. The organism must undertake a ‘random walk’ in its phase space to find the best new fitness ‘hill’. But the phase space is so vast (multiply number of genes + alleles x interacting genes / molecules & you get a vast number of possible outcomes). Kauffman suggests that no gene could traverse that space so natural selection optimises the space to make it easier to ‘see’ nearby fitness ‘hills’ (small, linear improvements). BUT at times of phase change - ie when the system is full of variety & changing from one state to another, you see chaotic behaviour. During these transitions, it is possible (according to Kauffman) to ‘leap’ across linear, incremental (‘metis’ bottom up) landscapes & reach peaks in the far distance. Leaping across spaces creates new phenotypes / species / billionaires without the gradualism (‘Metis’). This is also the punctuated equilibrium theory & has echos in the hopeful monsters theory. The idea that new ‘wholes’ can form alongside gradual, incrementalism. But obviously all this stuff is trial & error, not deterministic, so some of these leaps are maladaptive. Whereas a small, linear optimisation that fails will affect the system as n-1, in chaotic systems, you see power laws so system failure can lead to n^-10 effects. To go back to your social media metaphor - the point here is that meta, the organism, just wants to produce lots of copies of itself. If it makes an adaptation regarding content moderation, it does so because it sees an opportunity for competitive advantage. Within social media platforms there are lots of sourdough starters undergoing rapid state change that occasionally boil over if they keep getting fed. Meta/X will respond to this in a way that optimises their own self interest. But Meta/ X are the sourdoughs within the gov kitchen. And the gov must set rules that keeps the sourdough on the boundary between sub critical & super critical (another Kauffman phrase). This creates the possibility for change without letting it run away. Kauffman uses the analogy of a nuclear reactor that is dampened. I don’t understand nuclear reactors but I can see my sourdough bubbling & know when it’s about to bubble over or when it needs feeding. And to bring this bach to Stafford beer - this is when sensory / local understanding is so valuable. I can’t handle the sourdough if I’m on holiday. I can ask someone to manage it but really I need to smell it, watch it, move it out of the sun. Agile is one way of describing this but really it’s about making small adjustments & learning by being very close to the system under observation & trying to keep up with it by understanding two or 3 rules of the system. Complex, chaotic systems are impossible to predict with accuracy but they have underlying rules (read Wolfram on cellular automata or Parisi’s study of starling murmurations). The role of govt is not to micromanage the sourdough but to keep an eye on it & add flour or refrigerate if eg the weather changes, or add energy (resources, heat) if it gets cold. The deregulation of the financial markets without a gold standard was like pouring food & energy into a vast pot of sourdough & hence we saw an explosion of new life forms & entities running across exponential phase space to create new, fragile ‘wholes’. A total disaster.

Expand full comment
Kaleberg's avatar

This reminds me of what has long bothered me about cybernetics.

Cybernetics seems so horribly general as to be useless. I remember reading some Norbert Weiner in the 1960s and feeling that there had to be more, a lot more. I remember working with Gordon Pask and one of his disciples in the 1970s, and once again, cybernetics was all generalities adorned with mathematics. As soon as you wanted to actually do something or understand how something worked, you wound up having to deal with systems theory and computation and their fellow travelers and their discontents.

You might imagine that cybernetics could illuminate modern biology with all its "omics", but the biologists are stuck having to generalize from the details and, if they ever wish to generalize, to gloss over exceptions and contradictions apparent and otherwise. You can describe what the biologists are learning from a cybernetics point of view, but that doesn't help one do or understand biology.

When dealing with cybernetics fans and acolytes, I often got the impression that they liked the field because of its lack of detail. It could be applied to almost anything. It let one make grandiose statements. Maybe I am too much of a doer or maker, but I got the sense that a big appeal of the field was that it let one gloss over all the sordid detail and leave it to lesser beings.

Expand full comment
DariusClarke's avatar

I’m surprised that Farrell didn’t mention that the words cybernetics and government come from the same Greek root word: kubernesis. Meaning the pilot of a ship.

Practically speaking, the helmsman of a ship does not steer by dead-reckoning, but by many minute adjustments in response to the wind, the waves, the currents, and how the ship’s structure, sails, and crew respond to those external influences, all while steering towards the goal, and usually with the intent for commerce.

Hence, modern governing organizations could then be compared to an armada of ships each with their own Individual, independent pilot responding to the actions, directions, and movement of the other ships in the armada as well. And they do this as well, without communicating directly with all the other pilots, but by watching all the minute data continuously.

And the more a pilot sails, the more experience she has to draw on for the future trips, completing the feedback loop.

Expand full comment
Henry Farrell's avatar

Being a pedant who studied the Attic and the Ionic when I was much younger, the word for pilot or helmsman is kubernetes ...

Expand full comment
Cheez Whiz's avatar

As a bear of little brain, I worry about the boring implementation details. I was with you right up to the phrase "agile software development," triggered PTSD. Agile is a brilliant, simple concept hampered by the boring implementation details that the requirements of its implementation are anathema to a hierarchical system, and requiring end-user guinea pigs to simulate real-world users, introducing their own biases. In other words, it's a lot harder than it looks on paper.

Expand full comment
Alex melville's avatar

I am 11% of the way through the book and can attest to Henry’s praise. It is really high signal book. I am highlighting at least 1 passage every 5 paragraphs.

All this reminds me of computational theories of physics, such as Wolfram’s theory he’s been going on about recently. https://writings.stephenwolfram.com/2021/04/the-wolfram-physics-project-a-one-year-update/. The key idea being “computational irreducibility”. Read that blog post for a more thorough understanding but it’s related to cybernetics by how it assumes that most things are unknowable and that there’s some boundary conditions preventing us from getting the nice simple answers we always fail to find in complex systems.

I’m enjoying reading the book and learning about cybernetics. But I am preparing myself to not find any solutions in it. It’s better at identifying the things to avoid rather than the correct steps to take.

Expand full comment
Henry Farrell's avatar

While Comrade Shalizi had reasonably kind things to say about Wolfram's explanation of LLMs, this remains a classic of the genre ... http://bactra.org/reviews/wolfram/

Expand full comment
ch1n3du's avatar

Reading the article reminded me of the work Abigail Deveraux has done on Complexity Economics, she has also done some work at Wolfram research. Her paper on the TAP economic growth model is really interesting. I'd recommend:

- https://www.cambridge.org/core/elements/abs/explaining-technology/35B60E962727A42ABAB78D33CDE76B7C

- https://scholar.google.com/citations?view_op=view_citation&hl=en&user=us8DRLEAAAAJ&citation_for_view=us8DRLEAAAAJ:_kc_bZDykSQC

Expand full comment
Ned Conner's avatar

I have just discovered your work — Excellent! I hope that you see this note, and take the time to look at the material at the website below through the lens of cybernetics. If you do, I hope that I will hear in some way or other what your thoughts are. allbelong.earth

Expand full comment
Claire Hartnell's avatar

The complement to Stafford Beer’s ideas is Donella Meadows leverage points essay. She has information flows at 5 in her list. Top of the list (where to intervene in a system) is: ‘the mindset or paradigm out of which the system - its goals, power structure, rules, its culture - arises.’ Apply this to Meta & you have your answer: the purpose of the system is to monetise clicks. If the purpose of Meta were to allow people to interact freely, that’s what they’d do. No algorithm, no optimisation of clicks. Just lots of conversations that get a bit boring once they’re into the 100th level of debate. The whole issue of variety attenuation on social media is simple - it’s human feedback! If posts appeared sequentially, there’d be lots of little debates that would peter out as new information hit the system. This is what pre algorithmic control social media looked like. All the mumsnet, friends reunited, early Facebook stuff. It gets pretty boring. You can search a topic & find a conversation but you can’t amplify it. Meadows’ 3rd point is ‘distribution of power over the rules of the system’. Here the content creators are in an unholy alliance with the monopolists. They want their content to get 1,000s of likes. It’s nice to be an academic earning pennies & suddenly find yourself elevated to Mount Olympus. I’ll be more crude - I’m guessing these people are getting laid more so why would they want to go back to little conversations that get lost? So look, in simple terms Beer & Meadows have all the solutions we need to build resilient sensory systems. But the number 1 problem is the one with the biggest leverage: ‘the mindset or paradigm [from which] the system arises.’ And here we must look to the next hierarchy up. What culture/governance system allows extractive, monetised social media to exist? it’s turtles all the way up I’m afraid.

Expand full comment
Alex Tolley's avatar

I attended a talk by Stafford Beer when I was at the Manchester Business School in teh mid 1980s. Much of teh talk was about his work for Allende, in Chile. It was a hierachical control model. He mentioned that when Allende was asked how he intended to provide teh top down control, he responded - "From the people". Today, I would say that was model that was not just recursive, but also recurrent. I have Beer's "The Heart of the Enterprise". This doorstop of a book is very slow going. I also have a cybernetics book by Ross Ashby that is far more like aearly "systems theory" than Beer's recurive design for management that is very top-down, hierarchical control system. I have ordered Davis Book from Amazon UK after this positive review and a similar review by Brad DeLong. What blows me away is that the US version is not yet available and priced at multiples of the UK version. WTF?

Expand full comment
Indy Neogy's avatar

I am loathe to get deep into the Naidu essay, because the issues are in the foundation and it's just too big for a comment. I think the shortest thing to say is that it's really useful to read the "parsimony" argument in economics the way I hope we have all learned to read the "Laffer Curve." Yes, there's a reality here, but the hard question is, what is the shape of the curve relative to the axis on the bottom. And indeed, compared to Laffer, the parsimony curve leaves even more questions hanging about the definitions of the x-axis.

Expand full comment
Gordon Ross's avatar

See also Evgeny Morozov's Santiago Boys podcast/documentary on Beer's work in Chile during the early 70's. https://the-santiago-boys.com/

Expand full comment
Brink Lindsey's avatar

Very striking that you bring up Jen Pahlka, as she's been raving about The Unaccountability Machine recently. Now with your combined recommendations, resistance is futile -- I plan to read it soon.

Expand full comment
Henry Farrell's avatar

There is reason - the two books could have starred in a Hollywood 'separated at birth but extraordinary parallel careers across different spaces' knockabout comedy http://bactra.org/reviews/wolfram/

Expand full comment
John Quiggin's avatar

This discussion of the polycrisis reminded me of the Tinbergen rule (you need as many instruments as targets) and then of being taught control theory 50 years ago (about the time of Eno's visit). One of the things I was told was that control theory was just a more staid name for cybernetics, which was associated with Norbert Wiener (Beer was not mentioned).

Expand full comment
Henry Farrell's avatar

For the cybernetics-control theory connection, Ben Recht (who I just recently met and is fantastic) is serving up the good stuff on a regular basis https://www.argmin.net/

Expand full comment
John Quiggin's avatar

Very interesting, thanks Henry!

Expand full comment
Nemo's avatar

Cybernetics, as Wiener set it out, was/is a unification of control theory and information theory, and a recognition that control/information theoretic concepts applied well beyond the industrial/telecom purposes for which they were devised.

In broad enough strokes, pretty much anything may be described as “something with a state, which we want to drive to some other state, by choosing some action; and we make this choice under uncertainty.” As such, the core cybernetic insight is to understand as best you can how much you can know about the system (observability), and how well you can influence it (controllability). These questions are only generally tractable in the linear case, which leads to the lack of precise insights when applied to complex non-linear systems as described in the article.

——-

Beer is an intellectual descendant of Wiener; once Wiener set out the broad notions of cybernetics, various other practitioners took it to their own domains.

Expand full comment
Michael A Alexander's avatar

I see history as an evolutionary process flowing from the past to the future. As such, theoretically it could be modeled as a system governed by a set of first order differential equations in time like natural systems (e.g. living organisms). Using this metaphor, what this article seems to be talking about is process control theory as Quiggin points out.

Now having good process control is well and good, but all that does is help smooth the flow that the system is on. If that flow is going to hell, what you are doing is better paving the path to hell. Since we cannot know where we are going, is it wise to amplify control using some complex computerized system that, like AI, nobody knows exactly how it works and which we mortals just have to accept. Or is attenuation a better way?

Consider a population of microbes. Each one of these is an enormously complex system carrying out hundreds maybe thousands of operations all at the same time. And this system then interacts with the external environment and with all of the other organisms, both like it and many other kinds that are out there. The situation is one of overwhelming complexity. Yet, plate out a sample from this environment to select a single strain, put it into an artificial environment (chemostat) and you have a system whose behavior can be modeled by a handful of simple equations. The system is still doing thousands of things, but all are governed by a handful of rate-limited steps that you control. This is an extreme example of attenuation.

Now consider a population of humans in Old Kingdom Eqypt. They collectively are doing many millions of things. As with the microbes, most of the activity is focused on energy generation (food production). The rest goes to anabolism, making the materials needed to keep the system going. Much of this goes to production of new Egyptians--plus houses, clothes, tools, etc. A small side stream goes to the production of what we see as the elements of civilization, the state, art, religious institutions and monumental architecture. Most of what these institutions consume is used to maintain them just as the bodies of the inhabitants must be energetically sustained.

What accumulates, and all that is left 4500 years later, is the monumental architecture. So just as we call the ethanol being produced in a fermentor by a population of yeasts the Product of that system, we might say the monumental architecture (e.g. the Pyramids) of Old Kingdom Egypt is the Product of that civilization. The Pyramids are a tiny fraction of what that civilization did, but in time it accumulated to a big noticeable thing that left a mark that can be seen even today.

I would submit that the Product of our civilization under neoliberalism is monumental financial architecture--that is the objective function of economic activity under neoliberalism is to create shareholder value. And over time it has accumulated into a monumental market capitalization equivalent to more than 60 years of global energy use at 2022 levels. All that work is tied up in an entirely ephemeral object.

Is this the path we wish to smooth?

Expand full comment
Vernell Chapman's avatar

I’m very intrigued by whatever this Rubik’s cube metaphor turns out to be. Will be looking for it when I finally get my book copy. But did you have any ideas of your own for a better one?

Expand full comment
Philip Koop's avatar

The application of Auftragstaktik to software engineering, or to bureaucracy, can be thought of as both attenuation and as amplification. On one hand, it reduces the amount of information that must flow up the line of control; on the other, it expands the capacity to control by enlisting those down the line as co-conspirators in the managerial process. The objection, I think, comes down to the question of the degree to which goals are in fact shared. The flaw in the objection is that when the control problem is complex, you can't solve conflicts of interest by centralizing control.

Expand full comment
NickS (WA)'s avatar

This is very helpful, and has a number of ideas that I will need to mull over. Thank you.

Expand full comment
Jeff's avatar

Do you think that Dan’s book would be a suitable entry point into the subject?

Expand full comment
Henry Farrell's avatar

Yes!

Expand full comment