The Death of the Legal Subject

tags: Predictive Analytics Digisprudence

Notes

I. INTRODUCTION

NOTER_PAGE: (3 0.7708674304418985 . 0.4332627118644068)

legal decision making relies increasingly on predictive algorithms to determine individual rights and interests.

NOTER_PAGE: (3 0.809328968903437 . 0.5360169491525424)

epistemic effect of algorithmic knowledge on the construction of legal subjectivity—the capacity to be recognized by law

NOTER_PAGE: (4 0.20703764320785598 . 0.3400423728813559)

NOTER_PAGE: (4 0.251227495908347 . 0.586864406779661)

diminishes the participation of the legal subject in the epistemic processes

NOTER_PAGE: (4 0.31751227495908346 . 0.2065677966101695)

exclusion of legal subjects from the production of knowledge about themselves has participatory, dignitary, and expressive effects, as power over self-articulation is transferred from the legal subject to the data capitalist.

NOTER_PAGE: (4 0.3821603927986907 . 0.4586864406779661)

The purpose of algorithmic subjectivity is not to faithfully portray the underlying flesh-and-blood individual using a one-to-one correspondence, but to facilitate their classification for “stochastic governance” through the identification of high-level behavioral patterns.

NOTER_PAGE: (5 0.2872340425531915 . 0.4459745762711864)

the algorithmic subject deliberately avoids the underlying individual;

NOTER_PAGE: (5 0.4378068739770868 . 0.4014830508474576)

individuals are relatively good at predicting their own behavior.

NOTER_PAGE: (6 0.5204582651391162 . 0.2733050847457627)

How is the basic unit of liberal society—the legal subject—transformed by the elevation of algorithmic knowledge? How are they differently represented? Is the actuarial project of algorithmic governance compatible with law’s normative commitment to individualized justice?

NOTER_PAGE: (7 0.2937806873977087 . 0.7108050847457628)

the rituals of law, including legal subjecthood, matter not only as devices for achieving certain legal outcomes, but as affirmations of respect for the individual.

NOTER_PAGE: (7 0.4328968903436989 . 0.3156779661016949)

NOTER_PAGE: (7 0.5900163666121113 . 0.326271186440678)

Biopolitics

predictive algorithms reflect persistent optimism that individual-level interventions can overcome the structural forces that sustain patterns of criminality.

NOTER_PAGE: (8 0.3625204582651391 . 0.2044491525423729)

NOTER_PAGE: (8 0.4328968903436989 . 0.2531779661016949)

II. THE TRADITIONAL LEGAL SUBJECT

NOTER_PAGE: (9 0.18003273322422259 . 0.3463983050847458)

A. Mental Autonomy

NOTER_PAGE: (11 0.28314238952536824 . 0.4163135593220339)

assumes that individuals possess the mental autonomy required to interpret and apply such instructions to their particular circumstances.

NOTER_PAGE: (11 0.3363338788870704 . 0.3919491525423729)

reflects society’s normative commitment to individual autonomy.

NOTER_PAGE: (12 0.3657937806873977 . 0.6949152542372882)

Although this approach bears more risk to public safety (not interfering until harm has occurred), that risk is “the price we pay for general recognition that a man’s fate should depend upon his choice.”

NOTER_PAGE: (13 0.24959083469721768 . 0.6038135593220338)

would expose every individual to unlimited legal sanction, for the test would not be our intentions but “sheer accident; and accident, by definition, may befall us all.”

NOTER_PAGE: (13 0.3657937806873977 . 0.2055084745762712)

B. Physical Autonomy

NOTER_PAGE: (13 0.43453355155482815 . 0.4163135593220339)

bus companies would have less incentive to improve the safety of their services because liability would bear no relationship to their individual conduct. In the absence of any other identifying evidence, the largest operator in any given area would always be held liable for any unexplained accidents.

NOTER_PAGE: (15 0.3018991486574984 . 0.3042372881355932)

hear this argument a lot and it never really makes sense to me. the largest operator is certainly still incentivized to reduce accidents, and even the smaller operators are only off the hook in the absence of evidence, which they can't rely on. it's not awesome but it's better than rando pedestrians bearing the risk. also disagree with the premise that a bus company ought to be treated as the same kind of legal subject as a person - using statistical evidence against a company doesn't have the previously discussed effect of eroding normative commitment to individual autonomy. a company is not an individual! and the individuals involved in the company are probably shielded from liability. what's the problem?

The promotion of law-abiding behavior, then, is an instrumental reason to adjudicate liability on the basis of individualized rather than statistical evidence.

NOTER_PAGE: (16 0.19842829076620827 . 0.2635593220338983)

NOTER_PAGE: (16 0.4328749181401441 . 0.5330508474576271)

III. THE ALGORITHMIC SUBJECT

NOTER_PAGE: (17 0.22789783889980356 . 0.3635593220338983)

Rogers encouraged actuaries to classify rather than to aggregate, to “personalize” risk ratings, and to construct risk classes.

NOTER_PAGE: (18 0.297969875573019 . 0.376271186440678)

Insurers framed economic security as an individual responsibility rather than a right of citizenship, justifying a reduced role for the state and securing the indispensability of their own services.

NOTER_PAGE: (19 0.29862475442043224 . 0.2271186440677966)

NOTER_PAGE: (19 0.3857236411263916 . 0.6516949152542373)

The actuarial subject of the twentieth century has been reborn as the algorithmic subject of the twenty-first.

NOTER_PAGE: (20 0.3510150622134905 . 0.2771186440677966)

A. Legal Legitimation

NOTER_PAGE: (20 0.41453831041257366 . 0.42033898305084744)

data itself is neither patentable nor copyrightable.152 Accordingly, data capitalists have engineered the de facto propertization of data “where the map of formal legal entitlements ends,”153 using a combination of contractual provisions, technical protocols, and trade secrecy protection.154 This “intellectual property entrepreneurship” has allowed data capitalists to reconfigure intangible property rights around new kinds of extractive practices.

NOTER_PAGE: (21 0.48919449901768175 . 0.7110169491525423)

B. Biopower

NOTER_PAGE: (22 0.2180746561886051 . 0.4533898305084746)

frames the behavioral patterns, predictions, and forecasts derived from personal data as “new forms of datafied and depoliticized truth” previously invisible

NOTER_PAGE: (22 0.5153896529142109 . 0.2957627118644068)

shields their practices from regulatory scrutiny on the basis that they are delivering statistical “truth.”

NOTER_PAGE: (23 0.13621480026195154 . 0.4271186440677966)

claims “personalized” knowledge about individual subjects, despite the fact that the exclusion of the individual from the knowledge production process forms the very basis of “Big Data’s” claim to objectivity.

NOTER_PAGE: (23 0.18860510805500982 . 0.4322033898305085)

the model will not reflect differences between individuals “along dimensions that are not captured by covariates.”

NOTER_PAGE: (24 0.29862475442043224 . 0.31525423728813556)

low confidence that an individual’s personal probability of failing to appear was similar to the probability ascribed to them by their risk group.

NOTER_PAGE: (24 0.3366077275703995 . 0.4923728813559322)

C. The Performance of Personalization

NOTER_PAGE: (24 0.435494433529797 . 0.3423728813559322)

Data science recalibrates the physical body as a site of information processing so that users are motivated by biometric data obtained through self-surveillance, rather than bodily signs of hunger, pain, and stress.

NOTER_PAGE: (24 0.5795677799607073 . 0.4576271186440678)

where algorithmic intermediation is a condition of access to essential services, individuals have no choice but to submit to algorithmic measurement.

NOTER_PAGE: (25 0.3005893909626719 . 0.31101694915254235)

debtors are compelled to participate in the credit score game in order to counter its marginalizing effects.

NOTER_PAGE: (25 0.3713163064833006 . 0.2906779661016949)

These behaviors do not alter the “riskiness” of the underlying financial subject, but are designed to make debtors appear more “trustworthy” to financial institutions.

NOTER_PAGE: (25 0.46889325474787164 . 0.29915254237288136)

giving individuals the “means of conforming to the gaze of the ‘surveillant assemblage’ . . . should not be confused with emancipation from the subjectivizing apparatuses”

NOTER_PAGE: (26 0.38768827766863134 . 0.33644067796610166)

fraud monitoring systems establish an invisible boundary between permissible and impermissible consumption.

NOTER_PAGE: (27 0.183366077275704 . 0.20677966101694914)

IV. THE DEATH OF THE LEGAL SUBJECT

NOTER_PAGE: (27 0.26260641781270466 . 0.3389830508474576)

A. Mental Autonomy

NOTER_PAGE: (28 0.26653569089718404 . 0.42203389830508475)

The algorithmic subject is not required to formulate or express individual desires or preferences because they are statistically preempted.219 This is the governance of statistical relations, not individuals.

NOTER_PAGE: (29 0.15062213490504256 . 0.40932203389830507)

B. Physical Autonomy

NOTER_PAGE: (29 0.2265880812049771 . 0.4177966101694915)

algorithmic subjects have no physical autonomy.221 Their actions are predetermined by the average historical behavior of their statistical predecessors.

NOTER_PAGE: (29 0.2652259332023576 . 0.5855932203389831)

when the algorithmic subject displaces the analog legal subject as the target of decision making, the opportunities available for the legal subject to exercise their full autonomy are reduced.

NOTER_PAGE: (29 0.40209561231172236 . 0.4949152542372881)

The prediction itself affects the outcome it claims to predict.

NOTER_PAGE: (30 0.12311722331368698 . 0.5728813559322034)

Judicial reliance on predictive algorithms exacerbates the autonomy-eroding effects of incapacitation

NOTER_PAGE: (31 0.21938441388343158 . 0.5398305084745763)

ignores a defendant’s capacity to diverge both from their own past and from their statistical peers—that is, their capacity to be an outlier.

NOTER_PAGE: (31 0.2999345121152587 . 0.6542372881355932)

an algorithmic score cannot be “controlled” by the individual it claims to represent

NOTER_PAGE: (31 0.4675834970530452 . 0.35677966101694913)

NOTER_PAGE: (32 0.1669941060903733 . 0.46779661016949153)

a defendant may be unable to meaningfully counter an algorithmic prediction with qualitative information about their personal circumstances and intentions due to the prejudicial effect of automation bias.

NOTER_PAGE: (33 0.18467583497053047 . 0.33559322033898303)

substantial empirical evidence that human decision makers tend to accept, rather than challenge, quantitative assessments and to assign greater weight, amongst a set of variables, to the variable that has been quantified.

NOTER_PAGE: (33 0.4178127046496398 . 0.34491525423728814)

C. Future Potentiality

NOTER_PAGE: (34 0.31237721021611004 . 0.41271186440677965)

require the persistence of existing disparities in order to be effective. Applying a fairness constraint to account for the effects of structural inequalities would reduce the accuracy of the predictive model and its ability to use historical data to predict future outcomes.

NOTER_PAGE: (35 0.21676489849377867 . 0.5576271186440678)

Pattern-based discrimination produces a “seemingly permanent economic underclass,” bound on all sides by historical data and the self-reinforcing loop of predictive profiling.

NOTER_PAGE: (39 0.2796332678454486 . 0.3423728813559322)

a White time that is “futurally open” (indeterminate), and a non-White time that is “futurally closed” (predetermined).300 Philosopher Charles Mills describes this as the “racialization of time”—the transfer of time from one set of lives to another.

NOTER_PAGE: (40 0.3339882121807466 . 0.26864406779661015)

The algorithmic administration of populations, or “stochastic governance,”302 secures the data freedom of a minority of elites while categorizing and disciplining the “risky” majority,

NOTER_PAGE: (40 0.3994760969220694 . 0.20677966101694914)

D. The Epistemological Inferiority of the Algorithmic Subject

NOTER_PAGE: (41 0.13228552717747216 . 0.25169491525423726)

the algorithm assigns greater epistemic weight to public institutional data than the unrecorded experience of parenting.

NOTER_PAGE: (41 0.5173542894564506 . 0.22372881355932203)

E. The Redistribution of Expressive Power

NOTER_PAGE: (42 0.46168958742632615 . 0.32627118644067793)

NOTER_PAGE: (45 0.3857236411263916 . 0.6627118644067796)

Courts, however, have been unwilling to recognize algorithmic tools as meeting the Brady standard,358 thereby instantiating the power of private capital over the conditions of human freedom.

NOTER_PAGE: (47 0.33136869679109365 . 0.423728813559322)

V. IS THE LEGAL SUBJECT WORTH SAVING?

NOTER_PAGE: (47 0.5121152586771448 . 0.3288135593220339)

NOTER_PAGE: (51 0.3005893909626719 . 0.3016949152542373)

VI. CONCLUSION

NOTER_PAGE: (51 0.4237066142763589 . 0.4398305084745763)

Notes

I. INTRODUCTION

legal decision making relies increasingly on predictive algorithms to determine individual rights and interests.

epistemic effect of algorithmic knowledge on the construction of legal subjectivity—the capacity to be recognized by law

personal narratives of defendants become less important than the statistical features they share with historical recidivists.

diminishes the participation of the legal subject in the epistemic processes

exclusion of legal subjects from the production of knowledge about themselves has participatory, dignitary, and expressive effects, as power over self-articulation is transferred from the legal subject to the data capitalist.

The purpose of algorithmic subjectivity is not to faithfully portray the underlying flesh-and-blood individual using a one-to-one correspondence, but to facilitate their classification for “stochastic governance” through the identification of high-level behavioral patterns.

the algorithmic subject deliberately avoids the underlying individual;

individuals are relatively good at predicting their own behavior.

How is the basic unit of liberal society—the legal subject—transformed by the elevation of algorithmic knowledge? How are they differently represented? Is the actuarial project of algorithmic governance compatible with law’s normative commitment to individualized justice?

the rituals of law, including legal subjecthood, matter not only as devices for achieving certain legal outcomes, but as affirmations of respect for the individual.

predictive algorithms reflect persistent optimism that individual-level interventions can overcome the structural forces that sustain patterns of criminality.

society neglects investments in social infrastructure in favor of predicting individual behavior using models that require the persistence of existing disparities in order to be effective.

II. THE TRADITIONAL LEGAL SUBJECT

A. Mental Autonomy

assumes that individuals possess the mental autonomy required to interpret and apply such instructions to their particular circumstances.

reflects society’s normative commitment to individual autonomy.

Although this approach bears more risk to public safety (not interfering until harm has occurred), that risk is “the price we pay for general recognition that a man’s fate should depend upon his choice.”

would expose every individual to unlimited legal sanction, for the test would not be our intentions but “sheer accident; and accident, by definition, may befall us all.”

B. Physical Autonomy

The promotion of law-abiding behavior, then, is an instrumental reason to adjudicate liability on the basis of individualized rather than statistical evidence.

Although both forms of evidence are probabilistically equivalent, courts are likely to view the individual (eyewitness) evidence as a more legitimate basis for liability than the statistical (market share) evidence.

III. THE ALGORITHMIC SUBJECT

Rogers encouraged actuaries to classify rather than to aggregate, to “personalize” risk ratings, and to construct risk classes.

Insurers framed economic security as an individual responsibility rather than a right of citizenship, justifying a reduced role for the state and securing the indispensability of their own services.

The actuarial subject of the twentieth century has been reborn as the algorithmic subject of the twenty-first.

A. Legal Legitimation

B. Biopower

frames the behavioral patterns, predictions, and forecasts derived from personal data as “new forms of datafied and depoliticized truth” previously invisible

shields their practices from regulatory scrutiny on the basis that they are delivering statistical “truth.”

claims “personalized” knowledge about individual subjects, despite the fact that the exclusion of the individual from the knowledge production process forms the very basis of “Big Data’s” claim to objectivity.

the model will not reflect differences between individuals “along dimensions that are not captured by covariates.”

low confidence that an individual’s personal probability of failing to appear was similar to the probability ascribed to them by their risk group.

C. The Performance of Personalization

Data science recalibrates the physical body as a site of information processing so that users are motivated by biometric data obtained through self-surveillance, rather than bodily signs of hunger, pain, and stress.

where algorithmic intermediation is a condition of access to essential services, individuals have no choice but to submit to algorithmic measurement.

debtors are compelled to participate in the credit score game in order to counter its marginalizing effects.

These behaviors do not alter the “riskiness” of the underlying financial subject, but are designed to make debtors appear more “trustworthy” to financial institutions.

giving individuals the “means of conforming to the gaze of the ‘surveillant assemblage’ . . . should not be confused with emancipation from the subjectivizing apparatuses”

fraud monitoring systems establish an invisible boundary between permissible and impermissible consumption.

IV. THE DEATH OF THE LEGAL SUBJECT

A. Mental Autonomy

The algorithmic subject is not required to formulate or express individual desires or preferences because they are statistically preempted.219 This is the governance of statistical relations, not individuals.

B. Physical Autonomy

algorithmic subjects have no physical autonomy.221 Their actions are predetermined by the average historical behavior of their statistical predecessors.

when the algorithmic subject displaces the analog legal subject as the target of decision making, the opportunities available for the legal subject to exercise their full autonomy are reduced.

The prediction itself affects the outcome it claims to predict.

Judicial reliance on predictive algorithms exacerbates the autonomy-eroding effects of incapacitation

ignores a defendant’s capacity to diverge both from their own past and from their statistical peers—that is, their capacity to be an outlier.

an algorithmic score cannot be “controlled” by the individual it claims to represent

reliance on an algorithmic prediction effectively punishes the underlying individual for membership of a statistical group, where membership is neither voluntary nor causally related to the outcome being predicted.

a defendant may be unable to meaningfully counter an algorithmic prediction with qualitative information about their personal circumstances and intentions due to the prejudicial effect of automation bias.

substantial empirical evidence that human decision makers tend to accept, rather than challenge, quantitative assessments and to assign greater weight, amongst a set of variables, to the variable that has been quantified.

C. Future Potentiality

require the persistence of existing disparities in order to be effective. Applying a fairness constraint to account for the effects of structural inequalities would reduce the accuracy of the predictive model and its ability to use historical data to predict future outcomes.

Pattern-based discrimination produces a “seemingly permanent economic underclass,” bound on all sides by historical data and the self-reinforcing loop of predictive profiling.

a White time that is “futurally open” (indeterminate), and a non-White time that is “futurally closed” (predetermined).300 Philosopher Charles Mills describes this as the “racialization of time”—the transfer of time from one set of lives to another.

The algorithmic administration of populations, or “stochastic governance,”302 secures the data freedom of a minority of elites while categorizing and disciplining the “risky” majority,

D. The Epistemological Inferiority of the Algorithmic Subject

the algorithm assigns greater epistemic weight to public institutional data than the unrecorded experience of parenting.

E. The Redistribution of Expressive Power

When criminal defendants have few meaningful opportunities to share their personal stories, the institution suffers the loss of their perspective.

Courts, however, have been unwilling to recognize algorithmic tools as meeting the Brady standard,358 thereby instantiating the power of private capital over the conditions of human freedom.

V. IS THE LEGAL SUBJECT WORTH SAVING?

Contemporary legal systems need a new conception of self, a new legal subject, that retains and expresses our fundamental commitment to equality and autonomy but pays greater attention to the social relations that constitute the self and, therefore, the legal subject.

VI. CONCLUSION