Early AI: ELIZA, SHRDLU
Socrates thought the written word was a poor AI substitute for dialogue
Can call things "intelligent" based on their ability to take the place of a human in a network of social relations.
Dreyfus: Rules always include a ceteris paribus condition, "everything else being equal". Defining "everything else" and "equal" is impossible for a finite machine (cf Frame Problem)
Must understand the "form-of-life" (Wittgenstein) to understand rules
Dreyfus: Computers struggle with open domains. Must limit ourselves to domains where articulation can be complete. cf Capability vs Tractability
Critiques Dreyfus and Wittgenstein: Distinction between open and closed domains is spurious
Calculator works like mathematician and ELIZA works like psychiatrist because we, the humans, can interactively repair their deficiencies (so easily we might not even notice doing it)
"Action" vs "behaviour": Acts are intentional, behaviours are not
An act can be executed by many behaviours (move a piece of wood on a chessboard vs mail a letter describing the move); a behaviour can execute many different acts (signing a cheque to pay vs signing to commit fraud)
A "machine-like" act merges action and behaviour: the intention is simply to perform the behaviour, e.g. assembly line work, military drill
Action is reasonably consistent, but past behaviour is not related in a simple way to future behaviour
Connection with digitization: Machine-like acts can be mapped to symbols without loss
Suchman: Any action considered in hindsight can be looked at as following a plan, but this isn't necessarily the case
Major question of the book: How completely can we describe action by breaking it down into machine-like acts?
Formalization gives up some individual interpretive capacity, some appreciation of fine detail, in exchange for mutual comprehensibility
Mirror writing: Intentionally undo a mental digitization to attend better to detail
Calculator games, e.g. 34 + 5771 = 5805 (he + ills = sobs): Is the calculator doing arithmetic? "It does not make sense to ask what the calculator is doing […] It only makes sense to ask what the kids are doing."
"Now I knew the numbers were 69 and 2.54, and using some other mathematical skill, I knew they had to be multiplied together."
"I considered the display. I performed another unconscious miracle of inductive generalization and saw the stylized liquid crystal display as digital symbols"
"the computer cannot do what we can do quite easily. In each case the blame is shifted from the computer to the user; the computer's deficiencies are ignored and all the stress is put on the user's failure to repair the deficiency in what has become, for most of us, the natural and expected fashion."
"Of course, now that they have been mentioned, it would not be hard to write little "hacks" that would cope with these particular deficiencies"
'"May I have this dance?" "Yes." "My machine will see to it."'
"We cannot delegate acts, we can only delegate the behavioural coordinates of the act"
"Perhaps because of our educational systems, we tend to think of the paradigm case of knowledge as a mixture of information and logico-mathematical ability that can be recapitulated as in an examination. It is as though we think of all our knowledge as being fully exhausted by what we could say about it"
Wittgenstein via Dreyfus: Rules of action can be fully explicated only "on pain of regress"
"What we can say decreases with the incorporation of what was once strange and new into inexpressible common sense"
"What were once explicit rules can become part of a society's unexpressed taken-for-granted-reality"
4 kinds of knowledge: Facts and formal rules, heuristics, manual and perceptual skills, cultural skills
"The difference between the machine-like performance of central features of arithmetic, and being able to do arithmetic is the addition of cultural skills."
Think about explainability "in terms of the relationship between what the user knows and what the system contributes"
"people exhibit a remarkable propensity to produce justifications of a solution even when the solution is wrong"
"it is a positively bad thing that intelligent computers are attaining natural conversational competence because this enables them to give the impression that they are more expert and less fallible than they really are"
"[…] stories of scientific research are nearly always told retrospectively. It is only because I have told the story of the TEA-laser retrospectively that the top leads have salience. At the time everything seemed equally salient."
"Given clean data that already conform to the expected result there is no reason why a program should not uncover the underlying relationship. But suppose some of the readings contained errors - as all real science does - how would the program cope?"
"The data can be taken to demonstrate the Pythagorean theorem only after the universe of possibilities has been narrowed to the binary choice between Pythagorean and non-Pythagorean"
On induction: What comes after 2, 4, 6, 8? 10, 12, 14, 16, or "who do we appreciate?"
"He pointed out to me that he was careful not to direct the compressed air into the ampule because, having passed through a pump, the air was marginally contaminated with oil. Should this be a rule in CRYSTAL?"
"Resting one's finger on the ampule as you vibrate one hand with the other is the crucial technique! Why is it nowhere mentioned? After all, the fact is that if you cannot do it, you cannot grow crystals."
"Once more it is the tolerances that an unskilled person like myself cannot grasp. I don't know what a proper zone melter ought to look like."
"Again, the rules of crystal growing as we had understood them up to that point were broken immediately. CRYSTAL was wrong or unhelpful on every occasion where I compared it with laboratory practice."
"Even in the small world of an ampule containing three elementary chemicals there are so many variables that the behavior of the system cannot be predicted."
Simply calculating masses from atomic weights "is probably the most useful bit of the whole program. It does something that even an expert finds irritating and error prone. Everrything else that CRYSTAL does is either wrong or too trivial to be worth bothering with from the point of view of an expert, or wrong and unusable from the point of view of a novice."
"we tested CRYSTAL out on some fairly skilled users. They all loved to play with it […] They felt that there was a class of user (not themselves) who could learn something of crystal growing from such an exercise."
"the machine is to mimic a man who is imitating a woman - not a man being himself"
"Like the spy in Semipalatinsk, a man pretending to be something he is not will have a far smaller repertoire than normal."
"in the mid-1980s we found that the frequency with which the students could tell the sex of respondents in the imitation game was almost chance. Because of men's and women's common experience the game is no longer exciting."
"it is impossible to know whether one possesses the ability to do an experiment without actually trying it out […] Discovering (or discovering the nonexistence of) thargs is a process exactly coextensive with finding out how to build a good tharg detector."
Interesting endorsement of parapsychology: "Each design improvement of the parapsychologists is matched by the critics' invention of a possible new ordinary-sense channel of communication."
"If one is speaking to an intelligent person over a two-way radio, the radio will normally emit sensible replies to whatever one says."
"Invention of new languages may be the prerogative of individual geniuses but legitimation of new languages is the prerogative of the cultural collectivity."
"It is a mistake to try to counterfeit skill with hyper-accuracy."
"Often, the mistake happens when well-ordered scientific descriptions of the world are taken to be the world itself […] in textbooks everything has already been parceled and tidied. Try to describe day-to-day practice and it bursts from the wrapping" (cf The Map Is Not the Territory)