Algorithmic realism: expanding the boundaries of algorithmic thought

tags

Critical Technical Practice Legal Realism

Notes

many well-intentioned applications of algorithms have led to harm.

NOTER_PAGE: (1 0.31748071979434445 . 0.5474209650582362)

The challenge for the field is how to account for social and political concerns in order to more reliably achieve its aims.

NOTER_PAGE: (1 0.39845758354755784 . 0.757071547420965)

when aspects of the world fall outside these boundaries, a method “has no hope of discovering these truths, since it has no means of representing them”

NOTER_PAGE: (1 0.730719794344473 . 0.7520798668885191)

many of the harms of algorithmic inter- ventions derive from the dominant mode of thinking within com- puter science, which we characterize as “algorithmic formalism.” Algorithmic formalism involves three key orientations: objectiv- ity/neutrality, internalism, and universalism.

NOTER_PAGE: (2 0.1979434447300771 . 0.23044925124792015)

law and computer science are typically seen as in tension, or subject to opposing logics: technology moves “fast” while law is “slow,” technology is about “innovation” while law is about “regula- tion,” and so on.

NOTER_PAGE: (2 0.2095115681233933 . 0.5707154742096506)

reforms involve incorporating new processes or metrics into the formal method, and thus do not allow prac- titioners to transcend formalism itself. Additions of form—most notably, algorithmic fairness—fail to provide the epistemic and methodological tools necessary

NOTER_PAGE: (2 0.4222365038560411 . 0.21797004991680533)

the evolution toward realism is an expansion of computer science to em- brace realist orientations alongside formalist ones, not a wholesale rejection of formalism.

NOTER_PAGE: (2 0.6137532133676092 . 0.6422628951747088)

explore a parallel to formalism in another field—law—and to how legal formalism was addressed with a methodological evolu- tion toward legal realism.

NOTER_PAGE: (2 0.6311053984575835 . 0.15141430948419302)

computer scientists are “diverse and ambivalent charac- ters”

NOTER_PAGE: (2 0.7776349614395887 . 0.5299500831946755)

drawing on the lessons of legal realism, we propose a new mode of computer science thinking—“algorithmic realism”— that responds to the concerns raised by STS and related disciplines.

NOTER_PAGE: (2 0.8374035989717223 . 0.15224625623960067)

algorithmic realism provides three alternative orientations: a reflexive political consciousness, a porousness that recognizes the complexity and fluidity of the social world, and contextualism.

NOTER_PAGE: (2 0.8746786632390745 . 0.31114808652246256)

Because formal knowledge “requires a narrowing of vision,” writes James Scott, “the formal order encoded in social-engineering de- signs inevitably leaves out elements that are essential to their actual functioning”

NOTER_PAGE: (3 0.12660668380462725 . 0.5108153078202995)

political de- cisions about what is and is not important.

NOTER_PAGE: (3 0.19601542416452442 . 0.7504159733777038)

Algorithmic realism may do little directly to remedy the harms of algorithms deployed through discriminatory public policies, by authoritarian regimes, or under exploitative business models.

NOTER_PAGE: (3 0.2230077120822622 . 0.2820299500831947)

trace a middle path between technological determinism and social determinism,

NOTER_PAGE: (3 0.4177377892030848 . 0.15307820299500832)

2 ALGORITHMIC FORMALISM

NOTER_PAGE: (3 0.6079691516709511 . 0.08652246256239601)

2.1 Objective and Neutral

NOTER_PAGE: (3 0.62146529562982 . 0.5207986688851913)

Formalism implies an adherence to prescribed form and rules.

NOTER_PAGE: (3 0.6233933161953727 . 0.09068219633943428)

algorithms are perceived as neutral tools and are often argued for on the grounds that they are capa- ble of making “objective” and “neutral” decisions

NOTER_PAGE: (3 0.6548843187660668 . 0.7071547420965059)

neutral actors following the scientific principles of algorithm design from positions of objectivity

NOTER_PAGE: (3 0.7152956298200513 . 0.5207986688851913)

objectivity—“the suppres- sion of some aspect of the self, the countering of subjectivity”—has become a widespread set of ethical and normative practices

NOTER_PAGE: (3 0.7390745501285346 . 0.7579034941763727)

“I’m just an engineer”

NOTER_PAGE: (4 0.1780205655526992 . 0.34775374376039936)

“Our job isn’t to take political stances”

NOTER_PAGE: (4 0.198586118251928 . 0.1447587354409318)

emphasis on objectivity and neutrality leads to algorithmic interventions that reproduce existing social conditions and policies.

NOTER_PAGE: (4 0.211439588688946 . 0.13560732113144758)

Because significant aspects of the social and political world are illegible within algorithmic reasoning, these features are held as fixed constants “outside” of the algorithmic system.

NOTER_PAGE: (4 0.3181233933161953 . 0.64891846921797)

existing social and political conditions treated as static.

NOTER_PAGE: (4 0.37146529562982 . 0.6738768718801997)

recidivism prevalence as a “constraint—one that we have no direct control over”

NOTER_PAGE: (4 0.4209511568123393 . 0.6613976705490848)

optimize the status quo rather than challenge it”

NOTER_PAGE: (4 0.5841902313624678 . 0.6089850249584027)

goal becomes to predict (static) distributions of social outcomes in order to make more informed decisions rather than to shift (fluid) distributions in order to enable better outcomes.

NOTER_PAGE: (4 0.5983290488431876 . 0.5449251247920133)

algorithms that uphold common definitions of crime and how to address it are not (indeed, cannot be) removed from politics—they merely seem removed from politics.

NOTER_PAGE: (4 0.6047557840616966 . 0.08153078202995008)

2.2 Internalist

NOTER_PAGE: (4 0.8187660668380462 . 0.09234608985024959)

unable to account for the particular ways that people, institutions, and society will actually interact with algorithms.

NOTER_PAGE: (4 0.8232647814910025 . 0.5183028286189684)

internalism: only con- siderations that are legible within the language of algorithms—e.g., efficiency and accuracy—are recognized as important

NOTER_PAGE: (4 0.8438303341902313 . 0.3527454242928453)

In “smart cities,” for instance, algorithms are being deployed to make many aspects of municipal governance more efficient [53]. Yet efficiency is just one of many values that city governments must promote,

NOTER_PAGE: (5 0.12596401028277635 . 0.6722129783693844)

because they rarely account for the second-order effects of their own introduction, these algo- rithms drastically overestimate the benefits of increasing roadway capacity: in response to more efficient automobile travel, motorists change their behavior to take advantage of the new road capacity, ultimately leading to more driving and congestion

NOTER_PAGE: (5 0.17866323907455012 . 0.32029950083194675)

2.3 Universalism

NOTER_PAGE: (5 0.422879177377892 . 0.07903494176372712)

universalism: a sense that algorithms can be applied to all situations and problems.

NOTER_PAGE: (5 0.44987146529562977 . 0.40016638935108156)

3 FORMALIST INCORPORATION

NOTER_PAGE: (5 0.5109254498714653 . 0.5158069883527454)

tend to respond to critiques of formalizations with calls for alternative formalizations

NOTER_PAGE: (5 0.5841902313624678 . 0.5524126455906823)

taking for granted with scant justification that algorithms are an effective tool for addressing social problems

NOTER_PAGE: (5 0.7924164524421593 . 0.11980033277870217)

formalist in- corporation cannot address the failures of formalism itself.

NOTER_PAGE: (5 0.8367609254498714 . 0.8394342762063228)

4 METHODOLOGICAL REFORM: FROM FORMALISM TO REALISM IN THE LAW

NOTER_PAGE: (6 0.10668380462724934 . 0.5224625623960066)

algorithmic fairness is ill-equipped to address these concerns because it is itself a manifes- tation of algorithmic formalism

NOTER_PAGE: (6 0.15809768637532132 . 0.2720465890183028)

4.1 Legal Formalism

NOTER_PAGE: (6 0.20179948586118251 . 0.5183028286189684)

What may appear “fair” within a narrow computational scope can reinforce historical discrimination

NOTER_PAGE: (6 0.30719794344473006 . 0.25207986688851913)

law was seen “as a science [that] con- sists of certain principles or doctrines”

NOTER_PAGE: (6 0.31748071979434445 . 0.6938435940099834)

mathematically define aspects of the fundamentally vague notions of fairness

NOTER_PAGE: (6 0.35089974293059123 . 0.39434276206322794)

Each entity ex- ercised absolute power within its sphere of authority but was not supposed to consider what lay beyond its internalist bounds.

NOTER_PAGE: (6 0.4003856041131105 . 0.8261231281198004)

legal formalism aspired to deter- minism. It was assumed that a small number of universal principles, derived from natural rights, could be applied to reliably deduce the correct application of law in specific instances

NOTER_PAGE: (6 0.487146529562982 . 0.7254575707154742)

overlook the ways in which these “fair” algorithms can lead to unfair social impacts, whether through biased uses by practitioners [2, 29, 55, 56], entrenching unjust policies [54], distorting deliberative processes [51], or shifting control of governance toward unaccountable private actors [17, 77, 146].

NOTER_PAGE: (6 0.5019280205655526 . 0.24958402662229617)

legal formalism coincided with the period of laissez- faire policies and provided reasoning well-suited to defend these policies from progressive challenge.

NOTER_PAGE: (6 0.5417737789203084 . 0.6181364392678869)

Attempts to define and operationalize fairness treat the concept universally, with little attention to the normative meaning behind these definitions or to the social and political context

NOTER_PAGE: (6 0.5687660668380462 . 0.26622296173044924)

prioritizes portability of definitions and methods across contexts

NOTER_PAGE: (6 0.6285347043701799 . 0.24126455906821964)

enabling the law to account for the realities of social life necessitated, as a first step, methodological critiques of the formal reasoning that judges used to uphold the status quo.

NOTER_PAGE: (6 0.7802056555269923 . 0.740432612312812)

4.2 Legal Realism

NOTER_PAGE: (7 0.10668380462724934 . 0.07820299500831947)

Motivated by what they saw as the failure of legal reasoning to account for its real-world impacts,

NOTER_PAGE: (7 0.14717223650385602 . 0.12728785357737105)

inability of supposedly well-reasoned legal analysis to address social chal- lenges

NOTER_PAGE: (7 0.1863753213367609 . 0.1156405990016639)

Unlike philosophy, argued Holmes, the law was not a project of the ideal, but an instrumental means of administering justice in the messy and complex world [70].

NOTER_PAGE: (7 0.18766066838046272 . 0.7129783693843594)

4.2.2 From Objective Decisions to Political Assessments.

NOTER_PAGE: (7 0.2326478149100257 . 0.519134775374376)

it is impossible to engage in legal decision making without exercising some degree of subjective judgment.

NOTER_PAGE: (7 0.2602827763496144 . 0.5965058236272879)

The upshot for realists was not that such expressions of politics are inappropriate, but that they are inevitable.

NOTER_PAGE: (7 0.31362467866323906 . 0.5848585690515807)

pragmatic approach to reform

NOTER_PAGE: (7 0.3380462724935732 . 0.18136439267886856)

“real rules” that actually described the behavior of courts

NOTER_PAGE: (7 0.3798200514138817 . 0.13394342762063227)

judicial restraint was as much of a political choice as judicial intervention

NOTER_PAGE: (7 0.39010282776349614 . 0.8777038269550749)

realists displaced the dominance of bright-line rules3 with a shift towards standards

NOTER_PAGE: (7 0.4158097686375321 . 0.6364392678868552)

realist critiques put forward new modes of practical reasoning that overcame the epistemic limitations of formalism and that expanded the commonsense modes of “thinking like a lawyer.”

NOTER_PAGE: (7 0.4331619537275064 . 0.1572379367720466)

4.2.3 From Internalist Boundaries to Porous Analysis.

NOTER_PAGE: (7 0.5012853470437018 . 0.5224625623960066)

4.2.1 From Universal Principles to Contextual Grounding.

NOTER_PAGE: (7 0.5102827763496144 . 0.08236272878535773)

legal outcomes were not—and could not be—the result of a scientific process.

NOTER_PAGE: (7 0.5282776349614395 . 0.3003327787021631)

developed limits to legal reasoning within legal authority,

NOTER_PAGE: (7 0.5861182519280205 . 0.7737104825291181)

efforts to deduce rights and duties from universal principles of liberty or autonomy overlooked how the law was indeterminate (e.g., it could protect “liberty” in multiple competing yet equally plausible ways), confronting decision mak- ers with “a choice which could not be solved by deduction”

NOTER_PAGE: (7 0.6401028277634961 . 0.2545757071547421)

resolve ambiguities in legal materials by look- ing to social realities.

NOTER_PAGE: (7 0.6760925449871464 . 0.6306156405990017)

multiple legiti- mate outcomes due to gaps, conflicts, ambiguities, and circularities

NOTER_PAGE: (7 0.7390745501285346 . 0.40016638935108156)

workers faced extreme coercive pressure from private employers. Hale showed how legal decisions necessarily distribute freedom and coercion among parties, thus necessitating that decisions be made in reference to a broader social objective.

NOTER_PAGE: (7 0.7512853470437018 . 0.6381031613976705)

forced legal actors to make judgments,

NOTER_PAGE: (7 0.7776349614395887 . 0.09317803660565724)

legal inquiry should concern itself with the messy ad- ministration of justice among real-world actors.

NOTER_PAGE: (8 0.11375321336760924 . 0.16555740432612312)

anticipate the behavior of actors looking to take advantage of the law.

NOTER_PAGE: (8 0.19601542416452442 . 0.2296173044925125)

computer scientists can interrogate the ways in which their assumptions and values influence algorithm design.

NOTER_PAGE: (8 0.2140102827763496 . 0.569883527454243)

4.2.4 The Realist Evolution of Legal Common Sense.

NOTER_PAGE: (8 0.23650385604113108 . 0.06821963394342762)

shifts the primary focus of algorithmic interventions from the quality of an algorithmic system (in an inter- nalist sense) to the social outcomes that the intervention produces in practice.

NOTER_PAGE: (8 0.30912596401028275 . 0.5790349417637272)

whether that system actually leads to the desired social changes.

NOTER_PAGE: (8 0.3727506426735218 . 0.540765391014975)

non-reformist reform “is conceived not in terms of what is possible within the framework of a given system and administration, but in view of what should be made possible in terms of human needs and demands.”

NOTER_PAGE: (8 0.48778920308483287 . 0.6813643926788686)

critical design, which “rejects how things are now as being the only possibility”

NOTER_PAGE: (8 0.5578406169665809 . 0.8344425956738769)

5 ALGORITHMIC REALISM

NOTER_PAGE: (8 0.5809768637532133 . 0.08818635607321132)

While no mode of reasoning can avoid imposing its logic on the world, self-conscious modes can expand their internal logic to explicitly reason about their effects on the world.

NOTER_PAGE: (8 0.6992287917737788 . 0.15224625623960067)

5.1 Political

NOTER_PAGE: (8 0.7538560411311054 . 0.08402662229617305)

interrogating the types of subjectivity that typically fly under the radar of “objective” practice: choices such as formu- lating research questions, selecting methodologies and evaluation metrics, and interpreting results.

NOTER_PAGE: (8 0.8508997429305912 . 0.17803660565723795)

new technologies contain numerous potential interpretations and purposes; how a technol- ogy stabilizes (in “closure”) depends on the social groups involved in defining that technology and the relative resources each has to promote its particular vision

NOTER_PAGE: (9 0.18508997429305912 . 0.6655574043261231)

5.2 Porous

NOTER_PAGE: (9 0.2589974293059126 . 0.08818635607321132)

formalist considerations (e.g., accuracy, efficiency, and fairness) are recognized as necessary but no longer sufficient

NOTER_PAGE: (9 0.36246786632390743 . 0.14143094841930118)

Who are the relevant social actors? What are their interests and relative amounts of power? Which people need to approve this algorithm? What are their goals? On whose use of the algorithmic system does success depend? What are their interests and capabili- ties? How might this algorithm affect existing scientific, social, and political discourses or introduce new discourses?

NOTER_PAGE: (9 0.3740359897172236 . 0.56738768718802)

Rather than optimizing existing systems under the assumption of a static society, computer scientists can develop interventions under the recogni- tion of a fluid society.

NOTER_PAGE: (9 0.5976863753213367 . 0.19633943427620631)

5.3 Contextual

NOTER_PAGE: (9 0.6368894601542416 . 0.5249584026622296)

developed algorithms to reduce the risk of crime and violence through targeted and non-punitive social services

NOTER_PAGE: (9 0.6812339331619537 . 0.16555740432612312)

Agnosticism entails approaching algorithms instru- mentally, recognizing them as just one type of intervention,

NOTER_PAGE: (9 0.8605398457583547 . 0.6064891846921797)

algorithmic realism casts practices such as failing to consider how users interact with an algorithm as no less negligent than failing to test a model’s accuracy.

NOTER_PAGE: (9 0.8611825192802056 . 0.27787021630615644)

6 DISCUSSION

NOTER_PAGE: (10 0.10732647814910025 . 0.5199667221297837)

agnostic as to the means, but not the ends.

NOTER_PAGE: (10 0.13303341902313623 . 0.1788685524126456)

the most impactful algorithmic interventions occur when algorithms are deployed in conjunction with policy and governance reforms

NOTER_PAGE: (10 0.1928020565552699 . 0.370216306156406)

Remedying these failings requires not incorporating new variables or metrics (such as fairness) into the formal method but instead introducing new epistemic and methodological tools that expand the bounds of what it means to “do” algorithms.

NOTER_PAGE: (10 0.2152956298200514 . 0.7204658901830283)

such a “solution” would not fit into EMS’s operations nor would it address the underlying issues generating long response times.

NOTER_PAGE: (10 0.5546272493573264 . 0.26206322795341097)

ambulance response efficiency was a limited frame for understanding (and thus reforming) EMS’s opera- tions: the efficiency of ambulance responses said nothing about the broader goal of providing services that address people’s needs.

NOTER_PAGE: (10 0.6651670951156812 . 0.26455906821963393)

instead worked with EMS to create a new unit of EMTs who would respond to these incidents via bicycle or car and be specially trained to connect people to local social services; the parameters of when and where this unit would operate were determined by analyzing EMS incident data.

NOTER_PAGE: (10 0.7480719794344473 . 0.10399334442595674)