11 Comments

Customised social media have been far less damaging than Fox News and rightwing radio, which serve up the same package to everyone who watches/listens. And old-fashioned word of mouth is still a major vector for conspiracy theories of all kinds.

Recommendation models (aka "algorithms") might help people pick a slightly more appealing hole in the rightwing rabbit warren, but that's all.

Expand full comment

It's strange to see a "thought experiment" on this subject ignore an archive of internal Facebook/Instagram documents at Harvard laying out a great deal of A/B test evidence of how specific design features, not just machine learning-based rec systems, shape the formation of communities and what content they consume.

No serious critic of social media suggests that it turns people into zombies or that self-rationalization and homophily aren't real. The question of wether it is an "important accelerant" is the more serious one, and it's not something that a thought experiment can address.

Expand full comment
author

We're agreed that the thought experiment can't address the question of whether it's an important accelerant (and we are pretty clear that it wasn't designed to address it). And you're right that the piece does not talk about specific design features - again it wasn't meant to. There's a hugely important set of debates about the relationship between design and community (the final bits of https://doi.org/10.1017/S0003055422000715 provide one possible set of starting points for thinking about this; https://www.nature.com/articles/s41562-019-0541-6 another).

But I respectfully disagree with your implied suggestion that it is weird for us to want to study this. A lot of people make sweeping arguments about the evils of engagement optimization - and I think clearing out that brush helps make space for the other, more sophisticated work that you'd presumably like to see!

Expand full comment

No implication that it's weird to want to study this. But I'm not sure what brush is getting cleared out in this instance -- it doesn't seem like it goes beyond making the case that social media can't be shown to be "the primary cause" of political polarization, which is an argument that platforms themselves have happily laid out.

https://www.reuters.com/technology/facebook-continue-research-social-medias-effects-users-exec-tells-cnn-2021-10-03/

Expand full comment
author

So that's not quite the argument - it's a contribution to a narrower debate over the consequences of ML and engagement optimizing algorithms. There are a lot of people who have argued that these algorithms are the crucial factor in creating bubbles of self-isolating crazy. What we are doing is constructing a model of an Internet where they don't exist - and showing that you would still plausibly get similar outcomes. Our findings actually don't let social media in general off the hook - instead it suggests that the _connect everyone and it will be awesome_ philosophy of Facebook etc are a big part of the problem. You could even think of this as two different causal models of how social media can make things worse - one emphasizing ML and optimization, the other emphasizing increased connectivity/search and easier self publishing of the sort we already had with Web 2.0. Our argument suggests that even without the former, the latter is likely to get you to bad places. And it is a _lot_ harder for social media companies to ease off the connectivity without borking their business models than to block the more obvious algorithmic rabbit holes. I think the "Facebook is polarizing everything" argument is overdone for the reasons that people like Yochai Benkler at Harvard have emphasized. But I would also think that our argument modestly contributes to the case that Facebook facilitates polarization (albeit through different channels than many people think), rather than undermining it.

Expand full comment

I work in marketing and can't read "engagement optimization" without thinking about the paid ads component - after all, the internet is built on ads.

And I know it's a deviation from your argument, but I just wanted to add mine in.

In a capitalist society, the consumer is king - their preferences influence and regulate this economic cycle we operate in.

To ensure the balance between supply and demand works, we assume that consumers possess rationality and wield influence over their own desires. That the consumer has sovereignty.

Sovereignty implies a choice. But is there really one?

Is it a choice or were you followed by ads once you've shown any sign of interest?

Is it really consumer sovereignty or just an illusion of agency?

Is consumer desire limitless or did we create it by toying with human's natural need for belonging, social status, egotistic tendencies, etc.?

Expand full comment

"The trick instead is to get competing communities into some form of social and political relationship, where they have to grudgingly take account of each others’ useful criticisms. "

I disagree. The "elite expert consensus" on (for example) climate science isn't perfect, but the answer is to improve its internal processes (peer review and so on), not to engage with the competing community of denialists. And that's true across the board, at least in the US. There is no intellectual value in engaging with the GOP/MAGA/QAnon on any issue. The only point is to peel off enough of their supporters so that we can outnumber them.

Expand full comment

A systematic look can be enlightening and paying attention to our not-so-rational intelligence is key indeed and I really liked the use of mathematical modelling in that article. Fun. You might be interested in this: https://youtu.be/9_Rk-DZCVKE?t=1529 This is a presentation that covers a wide range of effects of the IT-revolution on organisations and societies (e.g. inertia). The link goes to the point which discusses some aspects of social media, mostly the fact that social media directly interact with some fundamental aspects of our intelligence, but the system doesn't have a lot of (stabilising) 'negative feedback', so more an engineering view than a mathematical view. 'Shared convictions' plays a role in that presentation (which lead to your 'shared rationalisations'). Without 'negative feedback' (an engineering term, not a psychological one), our convictions (and thus everything that depends on it, like societies) fragment, so social media is inherently destabilising, even without the positive feedback by attention-driving algorithms. Yes, it works without the algorithms, but that doesn't mean the algorithms have no effect.

How our intelligence is 'convictions-based' (an evolutionary advantage, as these are in fact useful for speed and efficiency, see https://ea.rna.nl/2022/10/24/on-the-psychology-of-architecture-and-the-architecture-of-psychology/) and not so much observations- and rationalisations-based is something that the current phase of the IT revolution is bringing to light.

Expand full comment

Well stated post. Thanks! I agree with you and other commenters that the 'accelerant' part of this issue may be the one that most needs further study. I'm an academic instruction librarian (I teach about information literacy) and also trans and like to think I'm a fairly sophisticated info consumer, but I still had to quit almost all social media years ago. This was due to a combination of consternation at the free and open vacuuming and selling of our habits and data, and horror at the growing uncivility and partisan snark and bullying and conspiracy thinking that has come to so dominate many social media ecosystems (whether caused by the weaponization of that data or not). I fear though, that worse times are to come before it gets better...

Expand full comment

Was there a reason you didn't examine social media platforms that don't use algorithmic timelines at all? Dreamwidth, for example, has been in business since 2009 and has never wavered from its chronological timeline. Such natural experiments would be good tests for the validity of your thought-experiment model.

To be clear, I think your general conclusion is on target, but I think the argument would be stronger with some empirical evidence. We don't have to imagine about non-algorithmic platforms, because they still exist.

Expand full comment

Edward L. Bernays said that people are rarely aware of the real reasons which motivate their actions. Thanks, a very interesting post. Really.

Expand full comment