The prices are too damn high. On that, basically everyone can agree. But what if your prices are higher than mine? For the same thing, at the same time? As the LPE Blog recently covered, it turns out that hidden personalized pricing is far more common than many people think. From big-ticket items like flights and hotel rooms to everyday purchases like food, when we see a price, we can no longer be sure it’s “the” price. It might instead be our price. And the differences can be substantial: when a recent investigation found Instacart charging individualized prices for shoppers at several popular grocery chains, the disparities were enough to “translate into a cost swing of about $1,200 per year” for a family of four.
Prices aren’t the only aspect of the consumer experience that businesses secretly individualize based on the person they’re dealing with. Sellers can use highly sophisticated behavioral modeling to make a wide range of automated decisions tailored to each individual consumer, taking into account such variables as the relative availability of competitors, the individual’s likelihood of switching, and the projected lifetime value of that individual to the business. As a result, the amount of time you wait on hold may not in fact be random. The decision to grant (or deny) you a refund may have more to do with who you are than with the “merits” of your request. It is likely that many of the businesses you patronize have already labeled you a high- or low-priority customer in ways that materially affect what you are offered and how you are treated.
In a new paper, Invisible Allies: Algorithmic Consumer Profiling and the Rise of New Group Harms, I argue that this hidden consumer profiling inflicts harms on our entire polity beyond dollars and cents. In a society already failing to confront visible inequality, consumer profiling tools threaten to create new, more under-the-radar patterns of advantage and disadvantage. When such patterns spread and become entrenched in the marketplace, democracy suffers in two interrelated ways.
First, democracy requires that discriminatory market practices be legible to public deliberation, so that they can be evaluated against our collective norms of fairness and equality. In many jurisdictions, offering a senior discount is uncontroversial and lawful, while giving discounts to native English speakers or physically attractive patrons could get the seller in legal trouble. This reflects a decision our polity has made about the sorts of market ordering we are and are not comfortable with. And this is also the sort of decision we are being prevented from making about the new winners and losers in the consumer economy.
Second, harmed individuals themselves can no longer forge the solidarities necessary for collective action in a democracy. Our understanding of permissible and impermissible marketplace practices didn’t fall from the sky. It developed from an iterative process of public conversation and political activism, which required that harmed individuals (a) know they were harmed, (b) know why they were harmed, and (c) work together to argue for change. Hidden consumer profiling kneecaps this process at every stage, keeping harmed individuals atomized and in the dark. Thus, those on the losing end of marketplace practices can’t act in concert (as was once possible) to make the argument that such practices are unjust.
To make the stakes of the problem more concrete, it is helpful to consider who the new harmed groups might be. While the invisibility of the tools in question makes it hard to build a detailed, empirically grounded picture, some stylized sketches are possible.
Take the specific practice I opened with: individualized pricing. On a basic level, businesses want to charge different consumers different prices so they can get as close as possible to each consumer’s “reservation price,” or the maximum amount they’d be willing to pay. There are many reasons why a consumer might pay more for a given good or service than someone else. Certainly, someone might just have more money, making them less price-sensitive overall.
The richer-consumers-pay-more scenario, however, is hardly the only possible outcome. Certain consumers might also pay a higher price because of a structural feature of their lives that makes a good or service more valuable, be it a disability (some people need painkillers more than others); a market failure or shortcoming in public services (some people need rideshares more than others); a fact about their family or friend group (some people have more caretaking responsibilities than others); or a deeply held belief (some people’s diets might depend more on plant-based proteins than others’). A transient situation could also make a good or service more valuable to a given consumer at a certain point in time. If a construction project started unexpectedly across the street, I would probably pay more for earplugs or a white noise machine than I would have last week. Importantly, some consumers might pay higher prices for reasons that have nothing to do with perceived value. They might, for instance, be especially susceptible to falling for hidden fees or other kinds of misleading overcharges, or constrained in their ability to comparison shop effectively.
Who stands to lose from these practices? One category is consumers assessed to be “distracted” or “sad.” Thanks to the streams of data that follow us around and the increasingly sophisticated models that make personalized judgments based on that data, businesses can now measure and exploit a wide range of cognitive shortfalls, from inattentiveness to impulsivity. Relatedly, businesses can and do measure the emotional states of their customers in real time. Reporting has revealed that Facebook, for instance, has used analyses of photos and other posts to assess when users are feeling “stressed, defeated, overwhelmed, anxious, nervous, stupid, silly, useless, and a failure.” Other digital profiling tools have categorized consumers based on difficult life events such as “bereavement.” Even some brick-and-mortar businesses use facial recognition technology and other data to make similar assessments about the emotional and psychological characteristics of their customers.
If every consumer who was charged more for an item because they were perceived (correctly or not) to be sad or distractible ultimately found out, they might have something to say about it. While some such consumers might suffer from diagnosed or diagnosable disabilities, others might fall into the above categories for other reasons, ranging from structural features of their lives (e.g., a chaotic home environment) to more temporary situations (e.g., the recent death of a pet). These invisible groups of harmed consumers represent latent coalitions that, at least currently, cannot counter-mobilize against the market logics that created them. For reasons I detail in the piece, I do not argue that all such groups should necessarily succeed in trying to ban or restrict the forms of social sorting they oppose. But democracy suffers when they don’t even get a chance to try.
While a whole host of reforms would likely be necessary to make these invisible allies visible to each other, some of the raw materials needed to build these new solidarities may already exist. Our system of aggregate litigation, including both class actions and multidistrict litigation, provide us with essential mechanisms for revealing widely dispersed patterns of harm and making affected individuals aware of them.
Take, for example, the many state laws banning “unfair and deceptive acts and practices,” often referred to as UDAP statutes. Many of these statutes use purposefully broad language, leaving it up to courts to refine and update what constitutes unfairness or deception using a common-law style of reasoning. These legal regimes in turn depend on litigants to bring cases that test the boundaries of the relevant statutes. By allowing courts to adjudicate a large number of (alleged) small-dollar UDAP harms in one fell swoop, the class action system can help shed light on new, arguably unjust marketplace practices that might otherwise remain hidden—both from the courts and from the victims. And to the extent one or more court rulings reveal a gap between the scope of current law and the public’s evolving sense of justice, that feeling of disjuncture can feed back into activism, public argumentation, and ultimately the legislative process. In this way, the aggregate litigation system can function as both a spur to democratic action and a crucial source of information for public deliberation.
At least, that’s how the system is supposed to work. As I explain in the piece, our aggregate litigation system in its current form may not be fully up to the task. While I urge advocates to use the litigation tools we already have to kickstart our much-delayed reckoning with these consumer profiling practices, we must also develop and advance a reform agenda that restores aggregate litigation to its full potential as an adjunct to the democratic process.
Additionally, new legislation, such as the new New York statute requiring the disclosure of individualized pricing, could also help bring the hidden categories of winners and losers to light. Most important, however, is that the various actors in our political and legal ecosystems train their attention on these profiling tools. From academics and reporters to public enforcers, private litigants, and legislators, it will take an all-hands-on-deck effort to ensure that democratic oversight can continue to play its proper role in the consumer marketplace.
