Seeking the Source: Criminal Defendants’ Constitutional Right to Source Code

tags: Technological Opacity and Due Process

Constitutional right to challenge witnesses implies right to inspect source code of evidence-producing software

Notes

Combination of 3 factors: Software itself produces evidence, software errors are ubiquitous, software cannot be cross-examined (due to proprietary concerns and bureaucratic opacity)

NOTER_PAGE: 2

Breathalyzer example

NOTER_PAGE: 6

Incorrect averaging code leads to low second measurement being ignored. Undiscovered until outside examiners hired by counsel were allowed to look. Software later updated but state failed to apply updates.

Timeline was a crucial issue in the case, and so Zdziarski needed to look at timestamps for device erasures, backup restores, and file accesses. The commercial forensic tools had reported these inaccurately

NOTER_PAGE: (9 . 0.16696750902527077)

The essence of our argument is this: the nature of software, and hence of computer programming, is such that certain errors are more likely to be found by adversarial testing

NOTER_PAGE: (11 . 0.39981949458483756)

adversarial audits—examination and testing of software by defendants—is necessary for a fair trial

NOTER_PAGE: (20 . 0.4268953068592058)

for software based on machine learning, not only should the source code be available to defendants but also the training data

NOTER_PAGE: (26 . 0.5514440433212997)

Overall, the problem is that the courts have applied the wrong standard

NOTER_PAGE: (45 . 0.3203971119133574)

USA courts have presumed (foolishly) that software is reliable