Skip to content

The dark side of the ‘data-driven’


Frank Pasquale (@FrankPasquale) is Professor of Law at Brooklyn Law School, and author of New Laws of Robotics (2020) and The Black Box Society (2015).

In her fascinating new book Automating Inequality, Virginia Eubanks recounts that the first “big data” set in the United States “was the Eugenics Records Office in Cold Spring Harbor. It was the public arm of the eugenics movement.” While the systematic collection of data has underpinned many important initiatives, it also has a dark side. Expect to see that dark side re-emerge with a vengeance in the next few years, as many American states intensify their surveillance of already disadvantaged groups.

Are there forms of knowledge that the state—or even university researchers—should not aspire to attain? Privacy law is meant to empower us with zones of thought and experience that no one can access without permission. Another branch of law, governing human subjects research, ensures that experimenters obtain consent before gathering data about individuals. As a member of the Council on Big Data, Ethics, and Society, I have thought and written about the types of data corporations and states should be able to gather about individuals, and the power relationships that data gathering entailed.benthams-panopticon-copy.jpg

Like disputes over free expression, the politics of data gathering for social science research is becoming a fraught area for progressives. For some, knowledge is an intrinsic good. Research of all stripes is a way of better understanding ourselves and our world. But there is another, more Foucauldian perspective: Where does the burden of scrutiny fall? What complicity does a social scientist have with the regime that provides data? The construction of what counts as “success” or “failure” in a given study is a highly political decision. A particular focus on some data or metrics comes at the cost of an exclusion or devaluation of others (akin to the “jurispathic” judgments Robert Cover recognized). All these questions will be critical as America’s laboratories (or meth labs) of democracy concoct innovative ways of denying health care to the poor, and ask social scientists to study “what works” in health policy.

Evaluating the Costs of Program Evaluation

The Trump Administration recently announced an intent to grant states permission to condition Medicaid benefits on work requirements (via Section 1115 of the Social Security Act). Former CMS Administrator Andy Slavitt immediately condemned the move. Activists were even more outraged. Journalists chronicled the many ways the work requirements were likely to worsen health outcomes, while burdening the vulnerable with paperwork and bureaucratic hurdles. New state “flexibility” will translate into cruel cutbacks for the disabled (who now may be denied transportation benefits).  

Critics characterized the move as a sharp break from a more data-driven Obama Administration. However, there is another way to tell the story.  The Trump CMS letter “Opportunities to Promote Work and Community Engagement Among Medicaid Beneficiaries” is carefully written to suggest a deep commitment to experimentalism and health services research. For example, Director Brian Neale is demanding “rigorous evaluation design” to assess work- (or volunteering-) requirement programs:

States will be required to evaluate health and other outcomes of individuals that have been enrolled in and subject to the provisions of the demonstration, and will be required to conduct robust, independent program evaluations. . . . The hypothesis testing should include, where possible, assessment of both process and outcome measures, and proposed measures should be selected from nationally-recognized sources and national measures sets, where possible.

The evaluation design should use both quantitative and qualitative methods, and will need to identify comparison groups and appropriate statistical analyses to evaluate the impact of the demonstration.  Evaluation designs should also include descriptions of multiple data sources to be used, including but not limited to multiple stakeholder perspectives, surveys of beneficiaries (both enrolled and those no longer enrolled as a result of the implementation of program requirements), claims data, and survey data (such as Consumer Assessment of Healthcare Providers and Systems (CAHPS)).

After speaking on a health law panel at the recent AALS conference, I overheard someone proudly mentioning that their friend had just received a grant to study the rollout of work requirements in Kentucky. The reality-based community rolls on, rigorously quantifying whatever punitive new “innovation” that Red America delivers.

On one level, I applaud the social scientists who will soon be chronicling predictable tales of Kafkaesque frustrations. However, I must also raise some questions about the import of such research. If, by some miracle, work requirements manage to scramble some percentage of those who could work, into employment, we shall never stop hearing about the success of this latest version of “welfare reform”—however irrelevant it is likely to become in an era of accelerating automation and persistent export of jobs. Cajoling persons into low-paid work that is likely to disappear soon is a recipe for disenfranchisement.

On the other hand, if work requirements fail, will the push for more labor end? I am skeptical. Advocates of punitive programs are rarely, if ever, put off by failure. They simply pivot to say that while the particular program design studied did not work, others can be developed. “Fail fast, fail early,” as Silicon Valley gurus counsel. Indeed, we should expect future partnerships between Medicaid authorities and platform capitalists. Former Obama advisor David Plouffe has already touted the quasi-criminal conspiracy known as Uber as an effort-forcing safety net/work-house.

For the neoliberal proponents of work requirements, whatever results emerge from any given study of current programs, there will always be new forms of discipline on the horizon—new ways to shape the body and soul of the Medicaid subject to render her or him more functional for an economic system premised on ever-cheapening labor costs. As long as it’s other people’s bodies on the line, endless rounds of trial and error are acceptable as a mode of policymaking.

Social scientists should also consider the possibility that the proponents of work requirements or wellness programs have no interest at all in their effectiveness. Consider, for instance, the evergreen initiative to drug test welfare beneficiaries. Has it ever proven cost-effective? And what about reports that the IRS recently spent $20 million to have private debt collectors recover just $6 million in taxes, and that “45% of the collections by private contractors were from taxpayers whose incomes fell below the minimum threshold, including those who received Social Security disability payments”? In each of these cases, neither helping beneficiaries, nor protecting government resources, seems adequate explanation (or justification) for agency practices.

Indeed, no one should be surprised if the new Medicaid initiatives actually cost the government more money than they bring in. Work requirements will disrupt preventive care for many recipients (and many of their extant caregiving duties), resulting in rapid intensification of medical problems, and exceptionally expensive visits to the emergency room. Through the cost-shift hydraulic, that money will eventually be recovered via taxation or private health insurance premiums. Like Trump’s infamous decision to abandon cost-sharing reduction (CSR) payments, the work requirements will likely amount to a trifecta of waste: higher costs for government, worse health outcomes for individuals, and even more intense bureaucratic rigmarole for all involved.

Calling Out Punitive Neoliberalism

In health quality research, a particularly bad outcome (like a wrong-leg amputation, or a large sponge left in a body after surgery) is known as a “never event:” Any competent health care system should have processes in place to stop them from happening. Health services researchers have not yet christened a category of health policy “never events.” But the abandonment of CSRs would likely count, Medicaid work requirements are likely to be a disaster, and “association health plans” (the Trump Administration’s other big idée fixe) have a disgraceful record. Thus TrumpCare is shaping up as “punitive neoliberalism,” unmoored from even the patina of policy rationality. As Will Davies explains:

The Schmittian worldview of the neoliberal pioneers, which pitted free market capitalism against all forms of non-capitalist systems, has mutated into something equally paranoid and simplistic, but now hatefully self-destructive. In contrast to the offensive upon socialism, the ‘enemies’ to be targeted are now largely disempowered and internal to the neoliberal system. In some instances, such as those crippled by poverty, debt and collapsing social safety nets, they have already been largely destroyed as an autonomous political force. Yet somehow this increases the urge to punish them further. . . .

What has emerged, I want to suggest, is not simply another ‘post’ (post-Fordism, posthegemony etc) but a new phase of neoliberalism, which is organised around an ethos of punishment. Moreover, this is not the form of punishment conceived by Bentham and historicised by Foucault, namely a measured science of displeasure. Rather, it is a constant, nonsensical and repetitive form of punishment which acts in place of reasoned discourse, replacing the need for hegemonic consensus formation.

Davies gives many concrete examples of how punitive neoliberalism now functions in Britain:

Social policies aimed at disciplining vulnerable populations have become … non-sensical. Britain’s ‘benefit sanctions’ regime sees welfare payments to job-seekers being suddenly suspended for up to a month, on account of breaches of intensely pedantic rules. Most often the breach involves missing a daily appointment at a job centre, but without any sense of procedural reason in how the rules are applied. One man had a heart attack on the way to the job centre, but was nevertheless sanctioned; another was sanctioned for going to his brother’s funeral, having tried to phone the job centre to inform them, but been unable to get through. Over a million people in the UK have received sanctions for one reason or another.

Workfare programmes in Britain now seem better designed to wreak humiliation and suffering than to do ‘what works’. Nearly 90 people a month have been dying after being declared ‘fit for work’ by workfare contractors and having their disability benefits cut. ‘Work capability assessments’ have been linked to an ‘additional’ 590 suicides. Active labour market policies now incorporate a wide range of often dubious behavioural activation techniques, from neurolinguistic programming to self-marketing slogans, which are used to coerce people into a desire for work and physical activity.

Like Brexit, a monumental case of nationally self-inflicted harm, this austerity is short-sighted and mean-spirited.

In an interview a few years ago with Russ Roberts, economist Daron Acemoglu cast doubt on the trend toward empirical studies in development economics. Leaders already know the difference between good and bad policies, Acemoglu said. Nations fail to grow and equitably distribute resources by design, not by mistake. Extractive institutions, such as corrupt oil companies or mercenary banks, continually stifle growth. But a research-industrial complex continues searching for the “right policy” when the real task is to diminish the power of those who sap resources and rig economies.

A similar lesson might enlighten health policy experts in the US and UK. Establishment journalists worry that university researchers have a “liberal bias,” and often wrap calls for more conservative approaches to health provision in the mantle of “ideological diversity.” But we should also be concerned that continuing research on programs that fail, over and over again, confers an unearned legitimacy on policy approaches that have no real hope of succeeding—or which define “success” in a cruel and biased way. The problem of bias may be less a question of lefty researchers stealthily advancing socialist ideas, than one of somnambulist techocrats giving undue credibility to programs that endanger the populations they ostensibly help. At some point, when the potential benefits of an experiment like work requirements are low enough (and the risks are high enough), it is unethical to expose human subjects to such policy research.

In short: If state policymakers want to see more Medicaid recipients working, they should try offering a job guarantee. They should not pretend that forcing already vulnerable and marginalized populations to scramble for low-paid work, volunteer “opportunities,” or worse, is doing Medicaid recipients any favors. And they should hold themselves to much higher ethical standards by merely piloting the work requirements in a few communities first, rather than imposing them statewide. As Nicholas Bagley has observed, “Congress thought that states would use section 1115 to launch small-scale tests: indeed, the Senate report accompanying the law assumed that ‘a demonstration project usually cannot be statewide in operation.’” It’s time to return to that Congressional intent, rather than subjecting some of the most vulnerable persons in society to unnecessary disruption.