Technological Due Process

tags

Automation Danielle Citron Technological Opacity and Due Process

(Citron, D. K.: Technological Due Process 66)

Notes

This century’s automated decision making systems combine individual adjudications with rulemaking while adhering to the procedural safeguards of neither

NOTER_PAGE: (2 . 0.4236168455821635)

those who view themselves simply as data processors will lose their motivation to learn the rules applied by computers

NOTER_PAGE: (25 . 0.3715937241948803)

Agencies also fail to test automated public benefits systems in a way that would ensure that the embedded rules accurately reflect policy

NOTER_PAGE: (25 . 0.5507844756399669)

Agencies routinely delay the implementation of new federal and state law into automated systems

NOTER_PAGE: (26 . 0.27580511973575556)

Automated systems misidentify individuals

NOTER_PAGE: (26 . 0.37241948802642444)

erroneously labeled individuals must undergo lengthy proceedings to prove their innocence

NOTER_PAGE: (26 . 0.4236168455821635)

Automated systems routinely send faulty notices

NOTER_PAGE: (28 . 0.4227910817506193)

Automated public benefits systems have changed or terminated benefits without providing any warning at all

NOTER_PAGE: (29 . 0.29232039636663915)

Automated systems often fail to maintain audit trails

NOTER_PAGE: (29 . 0.45169281585466553)

procedural due process safeguards the important interests of individuals when an agency acts against a person or a small group of people on the basis of their particular circumstances

NOTER_PAGE: (31 . 0.43352601156069365)

When agencies adopt policies that affect a large number of people, by contrast, no individualized due process protections apply

NOTER_PAGE: (31 . 0.546655656482246)

computers both render decisions about important individual rights and engage in rulemaking

NOTER_PAGE: (32 . 0.22956234516928156)

Automation jeopardizes the due process safeguards owed individuals and destroys the twentieth-century assumption that policymaking will be channeled through participatory procedures

NOTER_PAGE: (34 . 0.2741535920726672)

Automated decision systems endanger the basic right to be given notice of an agency’s intended actions

NOTER_PAGE: (34 . 0.6292320396366639)

provide individuals with an opportunity to be heard “at a meaningful time and in a meaningful manner.”

NOTER_PAGE: (36 . 0.37489677952105693)

Computer programmers inevitably engage in rulemaking when they construct an automated system’s code.

NOTER_PAGE: (41 . 0.5937241948802642)

Automated systems establish new policy when they embed distorted or simplified rules

NOTER_PAGE: (41 . 0.7440132122213047)

policy changes caused by programming mistakes constitute new rules, regardless of the agency’s specific motive

NOTER_PAGE: (42 . 0.3270024772914946)

Notice-and-comment rulemaking requires agencies to publish the rule, solicit the public’s comments, and issue a final rule that responds to those comments and articulates the basis for the rule

NOTER_PAGE: (43 . 0.24525185796862095)

Because vendors typically build these systems, the source code is proprietary and closed. The public has no opportunity to review new rules embedded in closed source code. Individuals lack notice of the new rules that will bind them.

NOTER_PAGE: (43 . 0.3261767134599504)

Embedding new rules in code undermines the democratic process.

NOTER_PAGE: (43 . 0.48554913294797686)

The hidden nature of encoded rules violates open-government laws and regulations that are intended to provide the public access to basic information about the conduct of agencies.

NOTER_PAGE: (44 . 0.5615194054500413)

Automated systems can be conceptualized as de facto delegations of rulemaking power.

NOTER_PAGE: (47 . 0.24525185796862095)

Automation encourages agencies to adopt overly simplified policy, which can more easily be translated into code.

NOTER_PAGE: (50 . 0.49793559042113955)

Meaningful Judicial Review in Jeopardy

NOTER_PAGE: (51 . 0.3220478943022295)

draws upon the rules-versus-standards literature to provide a systematic approach to deciding between automation and human discretion

NOTER_PAGE: (54 . 0.27663088356729976)

rules a huge, and often decisive, advantage on the basis of cost and convenience rather than the desirability of the substantive results

NOTER_PAGE: (56 . 0.2617671345995045)

Decisions best addressed with standards should not be automated

NOTER_PAGE: (57 . 0.21635012386457472)

At a minimum, automated systems should generate audit trails

NOTER_PAGE: (58 . 0.4112303881090008)

agencies should require hearing officers to explain, in detail, their reliance on an automated system’s decision

NOTER_PAGE: (60 . 0.21552436003303055)