Legitimacy, Authority, and the Political Value of Explanations

Notes

Increasingly secret, complex and inscrutable computational systems are being used to intensify existing power relations, and to create new ones

NOTER_PAGE: (1 . 0.2708208508088676)

To explain X is to communicate information about X that enables some presumed audience to reach a justified understanding of X.

NOTER_PAGE: (1 . 0.5261575466025256)

One can explain acts causally

NOTER_PAGE: (1 . 0.5904990980156344)

Or one can give a normative explanation

NOTER_PAGE: (1 . 0.6205652435357787)

Our aim is a 'justified' understanding of X

NOTER_PAGE: (2 . 0.13890559230306676)

Justified understanding is telic: it depends on the audience's goals.

NOTER_PAGE: (2 . 0.17378232110643416)

So: to know what counts as an adequate explanation, we need to know why explanations matter, and to whom they are owed.

NOTER_PAGE: (2 . 0.2832230907997595)

proprietary tools, kept secret from those affected by them.

NOTER_PAGE: (2 . 0.4149128081779916)

too complex to be fully understood by any particular actor

NOTER_PAGE: (2 . 0.561034275405893)

ML models are often inscrutable to human analysts

NOTER_PAGE: (2 . 0.6879134095009021)

Power is one-way control

NOTER_PAGE: (3 . 0.5742633794347565)

As well as substantive justification, standards of procedural legitimacy and proper authority must be met.

NOTER_PAGE: (5 . 0.2988574864702345)

a central task of our political institutions—captured by the ideal of procedural legitimacy—is to limit power by subjecting it to rules.

NOTER_PAGE: (5 . 0.4113048707155743)

legitimately exercised power is limited in both range and degree, to the minimum needed

NOTER_PAGE: (5 . 0.5532170775706555)

the powerful must follow exacting procedural standards.

NOTER_PAGE: (5 . 0.6656644618159953)

mechanisms of contestability and accountability

NOTER_PAGE: (6 . 0.22309079975947085)

We can explain many important aspects of decision systems that use computational tools

NOTER_PAGE: (11 . 0.6392062537582682)

Procedural legitimacy requires that significant decisions be made according to clear, defensible, publicly accessible rules.

NOTER_PAGE: (11 . 0.7654840649428744)

normative explanations of precisely which rules were being applied

NOTER_PAGE: (11 . 0.8027660853878533)

Complex computational systems often bury the rules that they purport to apply, or else apply rules that they have no business applying, simply because they can easily be implemented.

NOTER_PAGE: (12 . 0.14311485267588697)

In the subsequent class action suit against the Australian federal government, it was revealed that the algorithm applied an 'income-averaging' rule that was explicitly deemed unconstitutional in the 1990s.

NOTER_PAGE: (12 . 0.2393265183403488)

Explanations of decisions made using computational systems should reveal the data on which the model was trained

NOTER_PAGE: (12 . 0.46662657847263983)

Procedural legitimacy demands that we treat (relevantly) like cases alike. To know if this standard is being met, we can use counterfactual explanations

NOTER_PAGE: (12 . 0.5742633794347565)

More generally, procedural legitimacy should protect us against risk of harm—by minimising both unjustified decisions, and accidentally justified decisions.

NOTER_PAGE: (13 . 0.1864101022248948)

are not only treated fairly, but are secure in that status.

NOTER_PAGE: (13 . 0.22309079975947085)

would a minor perturbation in the input data have completely changed the outcome?

NOTER_PAGE: (13 . 0.34876728803367407)

multiple roughly equally well-performing models to choose from

NOTER_PAGE: (13 . 0.3668069753457607)

Procedural legitimacy also requires accountability. Complex computational systems make it easy to obfuscate human responsibility.

NOTER_PAGE: (14 . 0.17077570655441973)

subcontractors who clearly lack authority to adapt our laws in implementing them

NOTER_PAGE: (14 . 0.5117257967528563)

outsourced enforcement of digital copyright to digital intermediaries

NOTER_PAGE: (14 . 0.6704750450992183)

minimise their costs and exposure to liability. So they predictably over-enforce. They are rewriting copyright law without any authorisation to do so

NOTER_PAGE: (15 . 0.14251352976548406)

These three features of empirical verification, trust, and aligned interests are often absent from the exercise of power by means of computational systems.

NOTER_PAGE: (17 . 0.5706554419723392)

no ground truth against which they can be measured

NOTER_PAGE: (17 . 0.6428141912206855)