next up previous index
Next: Repeated questions across evaluators Up: Baseline model performance Previous: Results

Discussion

 The evaluation results demonstrate that ISAAC is reading and comprehending these three stories at a level indistinguishable from humans, based on independent judges. This was the overall result I expected--ISAAC is a capable reader.[*] Three points are worth mentioning. First, ISAAC lacked the crucial concepts it needed to comprehend the stories before reading them. Second, the initial evaluation performed indicates that   ISAAC is reading and comprehending the three stories at the same level on these comprehension questions as the human participants, as judged by the independent evaluators. Third, I am able to examine ISAAC's memory after the reading episodes and examine the concepts which were created during the course of reading--in each case, the model successfully creates concepts which allow it to comprehend the stories without creating unnecessary concepts or bizarre concepts.



Kenneth Moorman
11/4/1997