Back before time began (in the 1990s) I used to maintain an open source software package. I was being paid to do it, so I had a lot of freedom to decide how to approach it. Once its popularity took off a bit, I started to get a lot of requests for help and bug reports on the mailing list. At first I found this hugely problematic—it was really time consuming. But after a while I started to notice patterns in the questions and bug reports, and realized that rather than being a problem, they were a trove of information.
So I started to just figure out what to do next based on the patterns I was seeing on the mailing list. I was doing rapid releases, so the feedback loop was fairly tight, and what I found was that after a while, the traffic started to really decline. I'd still get the occasional thing, but nowhere near as much.
This required a bit of humility—sometimes the user would do something that seemed completely stupid to me, and I needed to make a change because of that, and I _wanted_ to argue with them that they were being silly. But in fact there was usually a way to genuinely improve the software as a result of their question, even if it felt like a dumb question.
The software has since been EOL'd, so this is ancient history, but it honestly surprises me how infrequently I see customer-facing organizations do this sort of thing. Rather than simply addressing the customer report, the organization doubles down and tries to do nothing, because the customer is mistaken.
It's really sad that this reaction is so prevalent. Imagine if instead of trying to find new ways to ignore the customer, they actually fixed the customers' problems!
I fear that humility and/or transparency (2 facets of the same thing, I'd suspect) are individual qualities that humanity just can't seem to bring from the individual level into the broader strokes of civilization for some reason, and the best we've been able to do to date is the enforced transparency of a functioning democracy. It certainly hasn't made its way into the corporate world to any observable degree.
That's an understandable position to take, but it hasn't always been this way. There are knobs to turn. The problem is they were turned the wrong way so long ago that we tend to think of them as givens rather than variables. But even in the last ten years this has obviously changed for the worse, which means better is not impossible.
Right, one of the things that changed IIRC in the 90s is the assumption that corporations' primary responsibility is to increase shareholder value. What you are saying here is a result of that.
However, also, it depends on when. Doing a good job creates a sustainable business. Enshittification increases venture capital value in the short term to allow for an exit, leaving the suckers who buy the stock holding the bag on a company whose customers hate it.
The product of modern venture capital is exits, not viable companies. That's a choice, and it's a choice we should unmake.
Amazon is doomed in the long run. It just might take a while. And who knows, they might fix themselves. But they've been enshittifying more and more as time goes by, and at this point the earlier adopters are all finding ways to get their stuff elsewhere. I haven't ordered anything from Amazon in several years. It's just gotten less and less convenient. I don't use Kindle anymore because they keep pulling books I paid for.
Remember when we thought Borders books was a permanent fixture? K-Mart? Etc. You can get away with screwing your customers and suppliers for a long time, but not forever.
Well, Gosh, Mr. Farrell, I'm remembering a conversation 3 - 4 years ago, with an attorney we used to have drinks with, in which someone mentioned 'ai', and he rather brightened up with the notion that '..it'd be great, to just have the computer generate the case facts for me and the hourlies would still accrue to me with less work." To which I replied, in a piqued tone, "Oh, sure, and it'd be your assistant who'll be buried in the pleas, the petitions, the summonses coming back at you from plaintiff's attorney's 'ai', and you will have to sort through and respond to all of THAT in court. It'll grind justice to a crawl." It just occurred to me that we haven't had drinks since...
This is developing into a situation like the depression-era streets of Chicago, in which whoever had the most firepower held the turf. "Thompson Model 1928? $200? Six hundred rounds per minute? And you can get a 50 round magazine? I'll take two".
I have no interest in engaging in this sort of gunfight, or computer-power struggle for ascendancy and 'winning'. I am more and more intent on simply walking away and not engaging with this 'world of the future', which the Abelard Snazz cartoon portrays so nauseatingly well. It was, as they say, supposed to be a caution, not a blueprint.
Grownups; perhaps, yes: as measured in dog-years' time. Ehm.. I've assumed you were referencing the 'grownups' at The Economist that wrote the article...
Everyone doing this should still be reading the material and correspondence. These agents are irreducibly sloppy and this is serious stuff in a person’s own name.
Brilliant. The whole sequence reminds me of an observation I used regularly in the past: "So, given that the brittleness of (machine) logic has been with us for quite a while, what do we normally do to manage the brittleness problems of using logic?
There is a lot said about AI that is often as true of IT in general. My favourite example: in the EU's AI Act you can replace "AI" with "IT" (s/AI/IT/g) roughly everywhere and have an even better document.
Reminds me of [The Red Queen Hypothesis](https://en.wikipedia.org/wiki/Red_Queen_hypothesis) in evolutionary biology: "species must constantly adapt, evolve, and proliferate in order to survive while pitted against ever-evolving opposing species." Particularly visible in the immune system, which exhibits an Abelard Snazz progression: you have an immune system, which produces – as a side effect – autoimmune diseases, so...
Aren't these arms races the inevitable result of complexity? The Abundance folks think you can turbocharge production by changing the rules to reduce objections, but this will just result in more gaming until the system ends up where it was before. I suspect Britain will find that out with its change in new homes planning and construction. Natural ecosystems become more complex and biodiverse as they proceed through the stages of succession, until they reach their climax state. Introduce a new species, or remove even a keystone species, and the ecosystem will eventually reconfigure itself back towards its maximum complexity state.
Tainter has suggested that civilizations reach their maximum state of complexity...and then collapse, unable to handle more.
AI might be a technology to extend that period for our global civilization, but for how long? Is the rise of authoritarian government the result of complexity that doesn't seem to allow changes, with the population demanding sweeping changes by simplifying things with a "strong leader"?
AI us indeed very good in finding bureaucracy related information. Just the other day, I was able, by chance, to help someone with a donation to a child - tax-free free to the child. Hence, the child's mortgage was reduced significantly.
Shades of the Terminators. By 2100 or sooner Robots will be everywhere. Human interventions is the best remedy now. I am not holding my breath. Thanks for this very illuminating piece!
Similarly the last part of the talk where OpenAI explain the extraordinary coordinated multi agent attack that they accidentally unleashed on hugging-face, is all about how we need more and more ai powered automated cyber defence to protect us from the new paradigm of ai powered automated cyber attack.
Last November, my water bill was 10x its normal amount. I had a leaky appliance (that I had fixed). I used Chat GPT to navigate the appeals process and got my water bill appealed. My hope is that rather then a cycle of escalation, AI reveals all the things that are intentionally opaque, and then we just make better bureaucratic systems. But I am probably naive
Indeed, but would not those prophets praise digital technology in its entirety? Ultimately, it was the leaders of Silicon Valley who claimed that every human issue could be resolved with a technological solution. AI is merely the newest tool in this pursuit.
This is the exact sell for hiring systems. Drowning in the flood of AI generated applications and resumes? You need an AI driven applicant tracking system to automatically handle the sourcing, filtering, and interviewing. AIs all the way down.
"More AI" doesn't inspire confidence as a solution to AI. Remagined processes and "better AI" might be. The example of automatically assessing whether a design complies with building code is compelling. This kind of thing is already reducing bottlenecks.
Policies that implicitly rely on friction will need to be redesigned: either apply AI to move faster or stipulate limits on response time.
Back before time began (in the 1990s) I used to maintain an open source software package. I was being paid to do it, so I had a lot of freedom to decide how to approach it. Once its popularity took off a bit, I started to get a lot of requests for help and bug reports on the mailing list. At first I found this hugely problematic—it was really time consuming. But after a while I started to notice patterns in the questions and bug reports, and realized that rather than being a problem, they were a trove of information.
So I started to just figure out what to do next based on the patterns I was seeing on the mailing list. I was doing rapid releases, so the feedback loop was fairly tight, and what I found was that after a while, the traffic started to really decline. I'd still get the occasional thing, but nowhere near as much.
This required a bit of humility—sometimes the user would do something that seemed completely stupid to me, and I needed to make a change because of that, and I _wanted_ to argue with them that they were being silly. But in fact there was usually a way to genuinely improve the software as a result of their question, even if it felt like a dumb question.
The software has since been EOL'd, so this is ancient history, but it honestly surprises me how infrequently I see customer-facing organizations do this sort of thing. Rather than simply addressing the customer report, the organization doubles down and tries to do nothing, because the customer is mistaken.
It's really sad that this reaction is so prevalent. Imagine if instead of trying to find new ways to ignore the customer, they actually fixed the customers' problems!
One of the main hallmarks of this era, rather sadly, is a distinct lack of humility.
True, but it doesn't hurt to point out the alternative! :)
I fear that humility and/or transparency (2 facets of the same thing, I'd suspect) are individual qualities that humanity just can't seem to bring from the individual level into the broader strokes of civilization for some reason, and the best we've been able to do to date is the enforced transparency of a functioning democracy. It certainly hasn't made its way into the corporate world to any observable degree.
That's an understandable position to take, but it hasn't always been this way. There are knobs to turn. The problem is they were turned the wrong way so long ago that we tend to think of them as givens rather than variables. But even in the last ten years this has obviously changed for the worse, which means better is not impossible.
Ah, but that wouldn't maximize profit! /s
Right, one of the things that changed IIRC in the 90s is the assumption that corporations' primary responsibility is to increase shareholder value. What you are saying here is a result of that.
However, also, it depends on when. Doing a good job creates a sustainable business. Enshittification increases venture capital value in the short term to allow for an exit, leaving the suckers who buy the stock holding the bag on a company whose customers hate it.
The product of modern venture capital is exits, not viable companies. That's a choice, and it's a choice we should unmake.
It either leads to exits or total domination i.e. Amazon.
Amazon is doomed in the long run. It just might take a while. And who knows, they might fix themselves. But they've been enshittifying more and more as time goes by, and at this point the earlier adopters are all finding ways to get their stuff elsewhere. I haven't ordered anything from Amazon in several years. It's just gotten less and less convenient. I don't use Kindle anymore because they keep pulling books I paid for.
Remember when we thought Borders books was a permanent fixture? K-Mart? Etc. You can get away with screwing your customers and suppliers for a long time, but not forever.
Well, Gosh, Mr. Farrell, I'm remembering a conversation 3 - 4 years ago, with an attorney we used to have drinks with, in which someone mentioned 'ai', and he rather brightened up with the notion that '..it'd be great, to just have the computer generate the case facts for me and the hourlies would still accrue to me with less work." To which I replied, in a piqued tone, "Oh, sure, and it'd be your assistant who'll be buried in the pleas, the petitions, the summonses coming back at you from plaintiff's attorney's 'ai', and you will have to sort through and respond to all of THAT in court. It'll grind justice to a crawl." It just occurred to me that we haven't had drinks since...
This is developing into a situation like the depression-era streets of Chicago, in which whoever had the most firepower held the turf. "Thompson Model 1928? $200? Six hundred rounds per minute? And you can get a 50 round magazine? I'll take two".
I have no interest in engaging in this sort of gunfight, or computer-power struggle for ascendancy and 'winning'. I am more and more intent on simply walking away and not engaging with this 'world of the future', which the Abelard Snazz cartoon portrays so nauseatingly well. It was, as they say, supposed to be a caution, not a blueprint.
Thanks for this.
Tim Long, Just Up the Hill from Lock 15
Eleven-year-old me can't believe that Abelard Snazz is being referenced by actual grownups
Grownups; perhaps, yes: as measured in dog-years' time. Ehm.. I've assumed you were referencing the 'grownups' at The Economist that wrote the article...
Tim Long, Just Up the Hill from Lock 15
Excellent callback to a fantastic publication
Everyone doing this should still be reading the material and correspondence. These agents are irreducibly sloppy and this is serious stuff in a person’s own name.
Brilliant. The whole sequence reminds me of an observation I used regularly in the past: "So, given that the brittleness of (machine) logic has been with us for quite a while, what do we normally do to manage the brittleness problems of using logic?
[Drum roll…]
We add more logic!"
https://ea.rna.nl/2020/02/11/a-tipping-point-in-the-information-revolution/
There is a lot said about AI that is often as true of IT in general. My favourite example: in the EU's AI Act you can replace "AI" with "IT" (s/AI/IT/g) roughly everywhere and have an even better document.
Reminds me of [The Red Queen Hypothesis](https://en.wikipedia.org/wiki/Red_Queen_hypothesis) in evolutionary biology: "species must constantly adapt, evolve, and proliferate in order to survive while pitted against ever-evolving opposing species." Particularly visible in the immune system, which exhibits an Abelard Snazz progression: you have an immune system, which produces – as a side effect – autoimmune diseases, so...
Aren't these arms races the inevitable result of complexity? The Abundance folks think you can turbocharge production by changing the rules to reduce objections, but this will just result in more gaming until the system ends up where it was before. I suspect Britain will find that out with its change in new homes planning and construction. Natural ecosystems become more complex and biodiverse as they proceed through the stages of succession, until they reach their climax state. Introduce a new species, or remove even a keystone species, and the ecosystem will eventually reconfigure itself back towards its maximum complexity state.
Tainter has suggested that civilizations reach their maximum state of complexity...and then collapse, unable to handle more.
https://risk.princeton.edu/img/Historical_Collapse_Resources/Tainter_The_Collapse_of_Complex_Societies_ch_1_2_5_6.pdf
AI might be a technology to extend that period for our global civilization, but for how long? Is the rise of authoritarian government the result of complexity that doesn't seem to allow changes, with the population demanding sweeping changes by simplifying things with a "strong leader"?
AI us indeed very good in finding bureaucracy related information. Just the other day, I was able, by chance, to help someone with a donation to a child - tax-free free to the child. Hence, the child's mortgage was reduced significantly.
Brilliant. My new heuristic is, Are we trying to solve a robot problem with more robots?"
The Rise of Ubiquitous Robots is here.
Shades of the Terminators. By 2100 or sooner Robots will be everywhere. Human interventions is the best remedy now. I am not holding my breath. Thanks for this very illuminating piece!
That's assuming global warming doesn't get us first. Or the next pandemic. Or even nuclear annihilation.
Of course. Glad we are both thoughtful Optimists!
Similarly the last part of the talk where OpenAI explain the extraordinary coordinated multi agent attack that they accidentally unleashed on hugging-face, is all about how we need more and more ai powered automated cyber defence to protect us from the new paradigm of ai powered automated cyber attack.
https://youtu.be/87DyyMV0kCY?is=HrOsaXqfAQOvv_gW
Last November, my water bill was 10x its normal amount. I had a leaky appliance (that I had fixed). I used Chat GPT to navigate the appeals process and got my water bill appealed. My hope is that rather then a cycle of escalation, AI reveals all the things that are intentionally opaque, and then we just make better bureaucratic systems. But I am probably naive
Yeah, that does sound a bit too optimistic.
We can more easily imagine Robots destroying our society then we can rational bureaucratic processes
The key word there being "rational". Bureaucratic processes aren't always rational.
Thank you. What I found interesting is how ‘solutionism’ has now become only an AI problem... I wonder why?
Just follow the prophets of profit.
Indeed, but would not those prophets praise digital technology in its entirety? Ultimately, it was the leaders of Silicon Valley who claimed that every human issue could be resolved with a technological solution. AI is merely the newest tool in this pursuit.
Exactly!
This is the exact sell for hiring systems. Drowning in the flood of AI generated applications and resumes? You need an AI driven applicant tracking system to automatically handle the sourcing, filtering, and interviewing. AIs all the way down.
"More AI" doesn't inspire confidence as a solution to AI. Remagined processes and "better AI" might be. The example of automatically assessing whether a design complies with building code is compelling. This kind of thing is already reducing bottlenecks.
Policies that implicitly rely on friction will need to be redesigned: either apply AI to move faster or stipulate limits on response time.