Why did software eat the world but not increase the size of the pie?
Software is eating the world. But isn’t the point of technology to increase the size of the pie not cannibalize it? In the era of the public cloud Productivity Growth is well below long-term average, Life Expectancy plateaued (before Covid) and broke it’s growth trend, and Real Wages have stagnated. We are creating more software than ever and fewer real world outcomes. The explosive, software fueled, growth has been largely confined to the consumer internet and the sellers of software themselves, but somehow not their end customers. What has gone wrong?
Software companies are shipping more, institutions are consuming more and the result of all of that software consumption is productivity growth below the long term trend. Every day for the past 15 years, I have witnessed one cause of the data points fueling these charts: users are fighting their software instead of being empowered by it. I believe this is caused by an error in decomposition as stark as that of the infamous Food Pyramid. In 1992, the USDA introduced the American food pyramid as a guide for healthy eating, emphasizing 6-11 servings of carbohydrates per day. As a result, an entire generation consumed a Cambrian explosion of refined grains and processed foods, encouraged by extensive marketing efforts from food companies.
A similar scale of structural error exists today in enterprise software. SaaS made it easier and cheaper to develop and distribute software, leading to an explosion of new players in existing and ever more narrowly defined categories, much like the snack aisle at the grocery store. These categories are largely the same categories you would have used to describe enterprise software thirty years ago, now subdivided.
This market structure has been reinforced by a software industrial complex. Buyers comfort with faster horses, funding and valuation norms, “analysts and experts” comfort with existing categories and business models, a powerful oligopoly of hyperscalers, and the nature of the services ecosystem has created an industrial complex that disproportionately rewards an overweight focus on the illusion of infinite scalability of minimum common denominator offerings within existing categories.
A generation of software companies was incentivized to figure out what the most easily consumed calorie is, synthesize it on mass, and sell to business as something they think they need. They have not worked backwards from a harder question of what do these companies need from technology in order to achieve their mission. The singular focus on zero marginal cost for software companies has created a parasitic relationship between enterprise software and the customers it serves. Zero marginal cost has resulted in zero marginal utility for end customers.
I have seen this up close and personal, in the category I understand most deeply, while building Palantir commercial to a billion dollar revenue business: Data. Conventional wisdom in Silicon Valley defined innovation by the improvement of incremental components in a data stack that treats data with the same frame established in the 1990’s: data is exhaust to be analyzed not fuel to drive operations. As Steve Jobs put it disparagingly long ago, management productivity not operational productivity.
It is obvious if you spend the time to deeply understand how big companies operate that this approach very quickly hits an asymptote of utility. Systems of action that actually affect real world action (transactional systems, SCADA systems, ERP’s etc..) and systems of insight (everything data and analytics) are entirely distinct from one another. The result is companies constantly try to get smarter but struggle to actually get better. They are left wondering why the potential value of their data investments have not turned into kinetic value on their balance sheets. And the human beings trying to actually create and deliver products are left to constantly be the wizard behind the curtain, working around their software every day to keep the machine running. Data and insight alone provide the nutritional value of a candy bar. Data, Humans, and Workflows. Carbs, fats, and proteins.
While SaaS innovations focus on incremental improvements to data processing - new flavor profiles, amount of sugar — the real value lies in redrawing the boundaries of what a software product does.
Why AI Can be Different
With AI it has to be different, and it will only be different if we redefine the structure of our software diet. Adding AI to the standard enterprise architecture diagram without redrawing boundaries will not matter. It’s like adding a new flavor of Mountain Dew. Delicious but not transformative. We have to reimagine what enterprise software needs to do. Not by layering chatbots on to the existing technology stack but by connecting the tech to the original mission of the organization.
Perhaps the most exciting thing about AI, aside from the power of the technology itself, is that there is no established playbook on how to deploy it. We are free from the planogram. The fact that everyone has such anxiety about how to monetize it is the most bullish thing I have heard in a long time for the potential impact of the technology. As the hype clears, AI will earn its keep as protein not a parasite.
And the potential is truly enormous. If you want to understand the geometric potential of AI on businesses you need only look at the place I purposefully ignored above where software HAS driven insane growth: the consumer internet. Consumer internet companies and their trillion dollar valuations have one critical thing in common: their core value proposition is software (and AI)-defined.
For most of the world, companies are software-facilitated. Their value creation happens outside of, sometimes enabled by, but often in spite of their software. Of course it is much easier to have a software defined business when the business is digitally native, but the underlying reason the internet companies are able to create compounding value for their end customers is not only because they operate primarily on the internet. It is because they have integrated their applications, data, models and value proposition to the user in one thoughtfully designed software system that constantly improves itself. The companies have leveraged the relative simplicity of digital operating models, well-organized data, the scale of the consumer internet, and simple interfaces to create software defined value propositions -- using data to compound in value over time. The Google search bar, the Facebook timeline, the Netflix home screen. The result is the most valuable products in the world.
The internet has been a catalyst for software-defined value propositions, but it is not the ultimate destination. The same extreme level of efficiency, compounding value propositions, and winner take all outcomes is possible in every industry and mission under the sun. To date, this has been virtually impossible for most companies because the technical difficulty of achieving software-defined value propositions for complex systems that have to rely on human beings and atoms is far greater than those of the consumer internet. Ambitious projects tend to fail; as seen in government software programs or the pervasive pain of commercial systems like ERPs, that must be so rigid to work at all that most actual work happens around them and in spite of systems of record.
Software and AI are now getting good enough to create software defined value propositions far afield from the pristine data gardens of the consumer internet. One fundamental step change is the ability to incorporate humans and human thought as a core component of an integrated technology system. In the same way that your iPhone gets better and better for you the more you use it, every single enterprise workflow can now turn from a user fighting technology and creating value outside of it to creating value with and through it. The relationship between software and humans will no longer be defined by the nurse wrestling with an EHR, the supply chain manager fixing what the ERP got wrong, the sales rep logging what they already did in a CRM. While there has been substantial panic about the anti-human risk of AI, my lived experience with all of our AI use cases is that this is the most pro-human enterprise technology we have ever created. Finally tech expert users can improve not fight.
One easy to understand example of this phenomenon is with Tesla. It is the first software defined car company. And because of that Tesla has and will continue to build a value proposition that moves further and further away from what you expect from a car company. By driving the car, you are creating the value of the product in the future. Tesla is playing a different game. AI powered software is now capable of enabling every single industry in the world to generate similar integrated feedback loops. Each industry will have its Tesla; FSD for every workflow and every company. It is just a matter of time.
Wrap
The shortcomings of the food pyramid had significant negative consequences for public health. The shortcomings of the conventional “wisdom” in enterprise technology are already apparent. The stakes for fixing this are not trivial.
We describe the consumer internet companies as monopolies. The domination of economic returns in the automotive industry is stark. Imagine the same technical domination of FAANG being achieved by an adversarial military or terrorist organization, or in less visceral but just as stark terms a rival industrial base. What will our military advantage look like if China is the Tesla of defense? What will your company look like if your competitor harnesses technology as effectively as Google has Search? What if we keep eating only Cheetos? I don’t intend to find out.
Insightful article! I'm definitely looking forward that that kind of an AI enabled future. I wrestled way too much with endless menus looking for things in Google Admin, uncustomizable platforms, and website scripting. I can't wait for a natural, intuitive way to build and refine digital systems - and see where that future takes us.
But I'm shocked that life expectancy in the US is declining.
Just wondering, because much of the modern world adopted the carb-heavy food pyramid teachings too, and our increasing reliance on heavily processed foods - and we've seen a significant rise in obesity in Australia. So technically we we should be seeing the same trend in other countries too.
Is that because of the US health system perhaps placing more priority on private healthcare for working individuals and less emphasis on a public health system we see in the UK, Northern Europe and Australia - could that be a contributor the sharper decline?
What are the essential shifts that need to happen in the mindset of software developers and corporate leaders to move from creating software that facilitates business operations to software that fundamentally enhances and defines business strategy and execution? This in itself seems more like a larger structural and mindset shift that needs to happen, but once the culture is set like concrete it may be painful to even get people to recognize this.