From the Perspective of Experimental Psychology
By SUEN KWAN
The first step in the development of an expert system is the extraction and characterization of the knowledge and skills of an expert. This step is widely regarded as the major bottleneck in the system development process.
In this paper, the author ( see reference ) gave us:
(1) a working classification of methods for extracting an expert’s knowledge,
(2) some ideas about the types of data that methods yield ,
(3) a set of criteria by which the methods can be compared relative to the needs of the system developer.
In generating expert systems, one must begin by characterizing the knowledge of the expert. But the expert’s knowledge is extremely detailed and interconnected, and the understanding of the expert’s perceptual and conceptual processes is limited. Little or no systematic research has been conducted on the question of how to elicit an expert’s knowledge and inference strategies.
Methods for extracting expert knowledge :
That method means studying the expert while the expert is engaged in the kinds of tasks that are usually or typically engaged in. One should see commonalities in terms of goals, the information the experts like to have available, and the data or records that are produced. Although the result might not actually be very informative about the expert’s reasoning, the analysis of familiar tasks can still be very beneficial as it can gives the knowledge engineer a feel for the kinds of knowledge and skills involved in the domain. Besides, the method can be used to generate a “first pass” at a database. The expert’s knowledge can be represented as a categorized list of statements cast in some sort of formal language using terms and categories that are meaningful and related to the domain at hand.
In an unstructured interview, the knowledge engineer asks more or less spontaneous questions of the expert while the expert is performing or talking about a familiar task. A prior analysis of the familiar tasks should have given the knowledge engineer sufficient basic information and trained the engineer in what to look for during the interview. Knowledge engineers sometimes make an audiotape of the expert’s ruminations. Audiotapes are necessarily a partial representation of the key information. An expert’s facial expressions and gestures can also reveal inference-making processes. Therefore, the interviewer should always take copious notes and not rely passively on audiotapes.
In order to add structure to an interview, the knowledge engineer initially makes a first pass at a database by analyzing the available texts and technical manuals, or by conducting an unstructured interview. The expert then goes over the first pass database one entry at a time, making comments on each one. A structured interview in one way or another forces the expert to systematically go back over the knowledge. Any given comment can lead to (1) the addition or deletion of entries, (2) the qualification of entries, (3) the reorganization of the hierarchical or categorical structure of the database (4) the addition or deletion of categories. The result is a second pass at the database.
Limited-information tasks are similar to the familiar tasks, but the amount or kind of information that is available to the expert is somehow restricted, forcing the expert to rely heavily upon (and hence provide additional evidence about) their knowledge and reasoning skills. This method is especially useful for revealing an expert’s strategies, like the formulation of hypotheses, strategic thinking and the use of heuristics. It can also be used to provide information about subdomains of the expert’s knowledge, to fill in any gaps in the database or a set of inference rules.
This method involves deliberate attempts to constrain or alter the reasoning strategies that the expert uses. One simple way is to limit the amount of time that the expert has in which to absorb information or make judgements. Another way is to ask the expert a specific question rather than to require the full analysis that is conducted during the familiar task.
To get evidence about the subtle or refined aspects of the expert’s reasoning when some of the expert’s knowledge and methods of reasoning have been described, tough cases can help to manifest the subtle or refined aspects of an expert’s reasoning. As tough cases are rare, the knowledge engineer must adopt special methods of study. The expert can be equipped with a small tape recorder and instructed how to make a verbal protocol of personal ruminations when encountering a tough case.
Can use some quantifiable criteria:
Brevity of the instructions that are necessary to specify exactly what the expert is expected to do, and in what order. And the number of stimuli or other materials that are needed, and their complexity, relative to the familiar task. The structured interview stands out under this criterion because it requires a first-pass database, which can itself consist of a relatively large and complex set of materials. For all other methods, the instructions involved are about one page long, and the materials will be one or few pages.
Total time taken by the task, including the reading of the instructions and the analysis of the data.
Familiar tasks can take a few minutes to an hour or entire day. Interviews can take days or weeks.
Is the task adaptable to different materials, to variations in the instructions, or to different experts?
Does the task depart much from the familiar task? The further the task departs from the usual problem-solving situation, the less it tells the knowledge engineer about the usual consequences of mental operations and judgements of the expert.
Are the data in a format ready for inputting into a database? Only the structured interview has this characteristic. Other methods’ result needed to be analyzed and transcribed.
Do the data records provide correct evidence about important knowledge and reasoning strategies? Do they disclose truths about the expert’s perceptions and observations, and the methods of testing hypotheses? Do they cover all the relevant subdomains of knowledge?
How many informative propositions are produced per task minute? Time is expressed as total task time, i.e. the time it takes the knowledge engineer to prepare to run the expert at the task plus the time it takes to run the task plus the time it takes to analyze the data.
Preliminary unstructured interviews can be critical for obtaining information on three key questions: (1) Who are the experts? (2) Is this problem well suited to the expert system approach?
(3) What are the needs of the users of the system?
If the knowledge engineer needs to become familiar with the domain, then an analysis of familiar tasks should be conducted and a first-pass database gleaned from this analysis. Then unstructured interviews can be used to determine users needs. If the engineer wants to build a refined or second-pass database, structured interviews are more efficient.
As the databases become very large, it becomes necessary to use special tasks. Limited-information or constrained-processing tasks or the method of tough cases can be used in order to focus on specific subdomains of knowledge or to disclose aspects of reasoning that the databases might lack.
All experts differ, and all domains of expertise differ; so, too, will all expert system development projects differ. Some methods for extracting an expert’s knowledge will fit some projects but not others.