Hermeneutical injustice and the computational turn in law

tags
Algorithmic Discrimination

Notes

NOTER_PAGE: (2 0.15161725067385445 . 0.6148713060057197)

‘[T]he injustice of having some significant area of one’s social ex- perience obscured from collective understanding owing to hermeneutical marginalization’.

NOTER_PAGE: (3 0.2506738544474394 . 0.8732125834127741)

deprives disadvantaged parties of the conceptual and interpretive tools needed to make sense of their experiences.

NOTER_PAGE: (3 0.3200808625336927 . 0.7540514775977121)

impossible for them to communicate these expe- riences to others, to contribute equally to society’s shared hermeneutical resources or to argue against wrongs com- mitted against them.

NOTER_PAGE: (3 0.3699460916442049 . 0.5748331744518589)

membership of a group which does not have equal access to participation in the generation and utilisation of social meanings.

NOTER_PAGE: (3 0.511455525606469 . 0.7940896091515729)

Miranda Fricker’s work considers how individuals can be ‘wronged specifically in their capacity as a subject of knowledge’.

NOTER_PAGE: (3 0.555256064690027 . 0.0905624404194471)

hermeneutical injustice affects the individual’s capacity for social understanding.

NOTER_PAGE: (3 0.6549865229110512 . 0.14871306005719734)

Both Wood and her harasser experienced the same ‘hermeneutical lacuna’ — as Fricker says, ‘neither have a proper under- standing of how he is treating her’ — this lack of proper understanding of sexual harassment was at that time more or less shared by everyone, even perpetrators. Key to un- derstanding why this was an injustice to Wood, however, is that the lacuna only caused serious disadvantage to her, and not her harasser.

NOTER_PAGE: (4 0.1987870619946092 . 0.8398474737845567)

When Wood quit her job, she was faced with her own inability to properly make sense of what had transpired: she did not have the concepts or words to explain, to her- self or to others, her experience. She only knew that she felt terrible, stressed and ashamed,

NOTER_PAGE: (4 0.2951482479784367 . 0.1344137273593899)

Having been unable to provide a clear explanation of the circumstances,16 Wood’s subsequent claim for unemploy- ment insurance was denied

NOTER_PAGE: (4 0.46832884097035044 . 0.08674928503336511)

for a long time it was not recognised that the failure to leave an abusive partner is not neces- sarily an indication of the severity of the abuse. Legally speaking, Fricker argues, the inadequate collective under- standing of the victim’s experience helped delay the legal move to construe cases of pre-meditated counter-violence on the victim’s part as the result of ‘long-term provoca- tion’.

NOTER_PAGE: (5 0.20080862533692723 . 0.6539561487130601)

what is most important to safeguard against domination is the notion of contestation.

NOTER_PAGE: (5 0.3942048517520216 . 0.16396568160152525)

epistemic justice must be served in the process of contestation:

NOTER_PAGE: (5 0.5121293800539084 . 0.25071496663489035)

Challen’s son, who campaigned for the release of his mother: ‘We talk a lot more now be- cause we’re able to. Understanding how coercive control works has given us the words – and I know, if I’d had those words earlier, I would have got the police. I could have stopped it’.

NOTER_PAGE: (6 0.3692722371967655 . 0.2716873212583413)

The rule of law and giving an account of oneself

NOTER_PAGE: (7 0.47102425876010784 . 0.09246901811248807)

Dignity is the status of a person predicated on the fact that she is recognised as having the ability to control and regulate her actions in accordance with her own apprehension of norms and reasons that apply to her; it assumes she is capable of giving and entitled to give an account of herself

NOTER_PAGE: (7 0.5923180592991915 . 0.5624404194470924)

The idea of ‘giving an account of oneself’ in the legal con-

NOTER_PAGE: (8 0.20080862533692723 . 0.09532888465204957)

text thus looks to people’s capacity to understand the law

NOTER_PAGE: (8 0.21495956873315364 . 0.0886558627264061)

and to apply it to their own circumstances.

NOTER_PAGE: (8 0.23517520215633425 . 0.0877025738798856)

whether an algorithm can ever be said to respect the dignity, in this rich sense, of those who are subject to it.

NOTER_PAGE: (8 0.5653638814016173 . 0.7702573879885605)

Self-application and contestation thus rely on the subject’s capacity to understand and interpret how the law relates to their own life,

NOTER_PAGE: (8 0.6765498652291105 . 0.5185891325071497)

machine learning methods are productive of the kind of epistemic injustices under discussion here

NOTER_PAGE: (9 0.24528301886792453 . 0.5166825548141086)

‘the set of social groups whose life patterns will be authoritatively interpreted by distant machines is growing ever larger’,1

NOTER_PAGE: (9 0.31873315363881405 . 0.6997140133460439)

Without knowledge of the workings of the algorithm and with- out a fully developed conceptual and terminological ‘tool- kit’ that includes notions like ‘algorithmic discrimination’ (or ‘coded gaze’ as in Buolamwini’s work), it is much more difficult for those subjected to this type of discrimination to make sense of their treatment by legal institutions and consequently to contest the legal decisions taken against them.

NOTER_PAGE: (9 0.5390835579514826 . 0.880838894184938)

That SyRI operates in relative silence and is de facto invisible to the naked eye may actually add to the unease and prejudice suffered by those living in those areas.

NOTER_PAGE: (10 0.46630727762803237 . 0.8150619637750238)

ba- sic information is kept secret about the risk models used in SyRI so that the inner workings of the system are un- known, inner workings of some of these systems might be unknowable due to their technical opac- ity or black-box nature.

NOTER_PAGE: (10 0.5539083557951483 . 0.897998093422307)

At best, individuals would only be able to form an ‘extremely general idea at best about the functioning of SyRI and how it might affect them individually’.

NOTER_PAGE: (10 0.6212938005390836 . 0.08293612964728313)

if a citizen does not have the hermeneu- tical tools to make sense of what has happened to them, because the computational system does not or cannot pro- vide them, it will be effectively impossible to contest the decisions taken against them, despite the fact that they produce significant legal effects.

NOTER_PAGE: (11 0.18800539083557952 . 0.20114394661582458)

on a more individual level, by depriving the individual in question of (legally) significant knowledge, like the algorithmic la- bel that designates them as a risk, it becomes significantly more difficult for them to understand their life and them- selves in light of the law as it applies to them. Let us call this the individual hermeneutical challenge to contesta- tion.

NOTER_PAGE: (11 0.4845013477088949 . 0.4146806482364156)

NOTER_PAGE: (11 0.6536388140161725 . 0.5214489990467112)

on a more structural or systemic level, an increase in the use of an algorithmic systems in law, like SyRI and COM- PAS, can create an institutional legal environment that affords less space for interpretation and argumentation and for the procedures that have traditionally offered legal protection.

NOTER_PAGE: (11 0.7884097035040432 . 0.08198284080076262)

what happens to our ability to contest if the com- putational turn in law limits our hermeneutical abilities and resources? What does that mean for legal protection under the rule of law and the notion of human dignity that it rests upon?

NOTER_PAGE: (12 0.29582210242587603 . 0.1363203050524309)

Footnotes:

1

Dan McQuillan, “People’s Councils for Ethical Machine Learning,” Social Media + Society 4, no. 2 (April 1, 2018): 4, https://doi.org/10.1177/2056305118768303.