Information Research, Vol. 6 No. 2, January 2001
Information Research, Vol. 6 No. 2, January 2001 | ||||
|
|
|
|
|
This paper considers how designers of information systems and end-user perspectives, communication models and linguistic behaviour differ. A critique of these differences is made by applying Habermas's communicative action principles. An empirical study of human-human information seeking, based on those principles, indicates what behaviour is a predictor of successful interactions and so is a candidate behaviour for integration into computerized information systems.
This paper considers how the perspectives of the designers of computerized information systems are in conflict with the needs and communicative behaviour of information system users. The hypothesis I present is that system designers are members of a discourse community whose internal self-orientation justifies the decisions taken by the group in developing systems. Furthermore, the paper holds that a critique of how information system designers' assumptions of communications hinder the end-user's ability to construct meaning that is appropriate for his/her own lifeworld (Agre & Horswill,1997; Habermas, 1985) needs. Critical theory, a philosophical perspective of the Frankfurt School, was chosen as the foundation for an empirical study of language use between experts and non-experts in information seeking. The results of the study suggest what modes of communication favour meaning construction and how they may be incorporated into computerized information systems.
It is necessary to introduce to the reader theories from the different research streams that explain the assumptions of the project and frame the investigation. The next section discusses communications models, linguistic perspectives from critical theory, and the design of systems. Following that are the results of an empirical study of information seeking behaviour. The paper concludes with suggestions for a different model of information systems design and possible consequences for users and designers.
We often assume that a sentence acquires its meaning as a result of its internal lexical components and syntactic relations. But it is also true that the meaning of a sentence can depend on assumed knowledge, and the sequence of sentences as a whole (Ellis, 1992). Typically there is an underlying semantic structure to a sequence of sentences. "It is important for a speaker or writer to discover a way to ensure that a listener or reader can establish relationships such as events, times, space, causality, and the like, to ensure correct interpretation" (Ellis, 1992). Discourse is not, then, merely a suite of elements from analytic sentences but, to be meaningful must satisfy, what may be termed "conditions" of coherency based on the life-world of the speaker and hearer.
Cohesion is an organizational scheme on a large-scale structural level. Following van Dijk (1983, 1985) meaning in discourse is influenced to a large degree by more general conversation organizing schemes that are not immediately apparent. "Sentences might be connected appropriately according to the given local coherence criteria, but the sequence would simply go astray without some constraint on what it should be about globally." Van Dijk's approach is that discourse is locally bound by semantic/syntactic; the collection of "utterances" is linked to some overarching proposition. Communication within these propositions includes the general background knowledge of speakers and hearers.
How we represent our background knowledge and understand typical sequences of events for a common situation is a described by some authors as a script (Schank & Abelson, 1997). Scripts have objects, roles and activities that help one handle typical interactions in efficient ways. Similarly, people develop efficient behaviour to interpret language in order to construct meaning (Rumelhart, 1983). These propositions may exist as large-scale cultural phenomena and exercise degrees of influence upon an individual. Some phenomena are socially identifiably "work labels" - teachers, police, programmers, etc. - whose individual practitioners master the scripts and schemata of their respective fields. Upon proof of mastery these individuals are admitted to the group and represent the group through their interactions with non-members (Lave & Wenger, 1991). The individual practitioner, then, is a potentially responsible for group performance. On the one hand, these work labels are identified with what we usually call "experts" or "authorities" and typically are trusted uncritically. People interacting with these experts do not know the scripts and schemata and therein lay a significant opportunity for influencing interpretation of events and communication. On the other hand, most peer-to-peer interaction is founded on shared knowledge, schemata, and scripts to help interpret speech. This is the case especially when the individual word units ("semantic primitives") themselves do not match the hearer's interpretation. For example, a speaker may say "I'm cold" when the speaker's intention is to cause the hearer to perform an action, in this case, perhaps to close a window. The "appropriateness" of the interpretation is guided by the interpeer schema.
Furthermore, speech entails obligations and responsibility. Speakers and hearers hold one another to promises implied in speech and accept responsibility for promises implied. In our daily interactions, we develop scripts for multiple schemata based on our relationship with the other party that includes actions for handing our misunderstanding.
Information seeking is one type of communicative interaction. Information seeking consists of an information seeking event (the overarching proposition) that exists as a cultural phenomenon (education, librarianship, etc.), performed by experts (practitioners of a limited-membership group) interacting with non-experts (information seekers). The uncritical assumptions that both parties (expert and seeker) bring to individual information seeking events influence language use and interpretive behaviour. If we study how expert-seeker discourse actually unfolds during information seeking "events", it is possible to discover what behaviour hinders and which helps meaning construction.
It is assumed in this discussion that a recognizable "information seeking event" involves at least two parties: a speakerand hearer pair; one is an expert, the other the seeker. It is also given that all information seeking interactions, human-human or human-computer, are subject to communications analysis. The analytic method chosen for this project is founded upon critical theory.
Critical theory, as expressed by Frankfurt School philosopher, Jürgen Habermas, includes a formal pragmatic analysis of speech acts between hearer and speaker pairs. This theory situates his overall critique of communication in society which is considerably more sophisticated than the parts expressed here would suggest (Habermas, 1973; 1985). Secondly, it offers a framework to examine questions of relationships, "truth, rationality, action, and meaning" (Habermas, 1999). Habermas understands that no speech is purely rational or true, so he developed a model of idealized speech against which actually-occurring speech may be critiqued.
As language is used in information seeking both to interact with an information providing system to discover and interpret information resources, so it is used to establish and adjust relationships between speaker and hearer pairs. There is a large body of literature from philosophy, computer and information science that is restricted to analyzing information seeking language solely upon the semantic elements of human speech. But as information seeking is intended to aid one in one's life-world, the employment of language, in light of some context, influences the hearer's interpretation of speech. Consequently, the hearer's understanding of truth can be manipulated and so subsequent rationality, actions, and the ability to construct meaning from the interaction are harmed. We would like to investigate the presuppositions that guide linguistic exchanges in everyday processes of communication. It calls to mind that certain things must be presupposed if communication is to be successful. This look into the pragmatic uses of language by communicatively competent social actors lead Habermas to focus not on semantic primitives but on how social actors orient themselves for reaching understanding. All speech has built into it sound linguistic structure (semantics, syntax) but the ability to understand, and to trust language, will influence the hearer. These connections to validity and the consequences of accepting speech is what Habermas (1973, 1985, 1999) labels "pragmatic". Every speech act, then, entails the raising and responding to validity claims.
Concomitantly, validity claims in speech raises expectations of responsibility for the speech utterance and the speaker. Habermas identifies basic types of validity claims: comprehensibility, truth, truthfulness, and normative right. Comprehensibility addresses the semantic and syntactic perspective: can the hearer understand the utterance? Truth is simply that: are facts in the utterance "true"? Truthfulness is the sincerity of the speaker to present what s/he believes is true. Normative right addresses whether a speaker's relationship to the hearer justifies his/her right to make claims embedded in an utterance and particularly influences questions of "appropriateness" of interpretation. If, as Habermas believes, everyday linguistic interaction is more pragmatic than just semantics, raising and recognizing validity claims, then information seeking, like other speech, incorporates a network of social relationships between the speaker and hearer pair. In communication, the speaker or hearer is both the individual practitioner (the expert) and the seeker, as well as the relationship between the seeker and the group to which the practitioner belongs. For information seeking, then, the seeker's relationship is influenced by the expert's (designer's) trained scripts and schemata. The social relationship between speaker and hearer includes a "cooperative relationships of commitment and responsibility" (Habermas, 1999). In other words, speaker and hearer undertake their communication to behave in certain ways and that the interaction depends on the cooperation of both parties. A speaker acting on an open, rational basis agrees to undertake an obligation to provide reasons for the validity of the claims that are raised in an utterance. The hearer accepts or rejects the claim. This simple exchange Habermas labels "na�ve communicative action." As speech, especially speech between parties whose sense of relationship is unbalanced, is more complicated, there are more complicated speech actions intended to address the validity claims. This is the critical theoretic version of "discourse." "When the background consensus (the set of pretheoretical assumptions of relationships, motivations, and speech between parties) is shaken, participants may break off the conversation, begin it again on a different basis, or switch to "strategic action" (Habermas, 1999).
When parties meet for some purpose, say information seeking, it is often assumed by the parties that they share a common goal: that they share the common aim of reaching agreement with regard to the validity of the disputed validity claim, that no force except that of the better argument is exerted, that no competent parties have been excluded from the discussion, that no relevant argument has knowingly been suppressed, that participants are using the same linguistic expressions in the same say, and so on (Habermas, 1999). If institutions and individuals' beliefs in those institutions are part of every social order, a network of cooperating institutions involved in commitment and responsibility, then their actions influence society in a larger way.
Finally, individuals and institutions can act strategically: that is, the means and content of discourse are manipulated. Manipulation takes many forms: use of authority role to block cashing-in of validity claims, refusal to respond or the denial of ability to respond, ignoring requests to clarify language, and so on. In other words, attempts to restrict discourse only to selected linguistic elements or to the control of validity claims are means to achieve one's own goals, or "strategic communication", even if the context of the interaction is socially or individually premised on truth, rationality, and action to be mutually beneficial. Strategic action may be exercised deliberately, or accidentally. In the next section, it will be demonstrated how strategic action is accidentally incorporated into information seeking design by the group of designers.
"The central claim explored here is that the standard engineering design process produces a fundamental blindness to the domains of action in which the customers of software systems live and work." (Denning, 1994)
This quote was the keynote address by Denning in a conference on the education of software engineers. The themes of engineering's self-orientation and assumption of authority, that result in closed, self-validating system behaviour, are found both in the literature of software educators and in critiques of how engineers as a social group perform (Suzuki, 1998) or from their philosophical roots (Dick, 1991; 1993). Boardman and Mathur (1994) argue for a standard model of software engineering education in which designers follow a recipe for insuring the value of the software application:
"Reliability: steps to follow: Convince the participants of high program reliability with data from testing and the logic of the program"
"Robustness: Construct test cases that represent boundary conditions (e.g. an empty input file) and demonstrate the infallibility of your code to such inputs."
Throughout computer science and engineering education, there are conscious attempts to orient students who will become members of the group of engineers and software developers that their approach, based on logical or mathematical models (Dick, 1991; Boardman & Mather, 1994), is the sole legitimate choice in information systems design. This belief is manifested in system and interface design.
As Denning suggests, the role of the end-user is minimized in information seeking design. In the early days of publically-accessible information systems, the design did not consider end-user needs in a serious manner. Over the past thirty years, however, there have been (Guindon, et al., 1987) various introductions of the end-user into the design model (Boland, 1978; Dagwell & Weber, 1983; Hirschheim & Klein, 1989). Some approaches limit the user-role by assigning the user to a group of similar properties (e.g., Allen, 1990).
Attempts to overcome the boundaries of the system behaviour recently have focused on computer-human interaction from several perspectives: perception or cognition (Daniels, 1986; Ellis, 1992) or belief revision (Cawsey, et al., 1992; Schimera & Schneiderman, 1994). However pursued, end-users from the designers' perspective tend to be objectified as incompetent clients, dependent on the system for help for both information sources and how to interpret the sources property, regardless of individual or temporary need (Lipsky, 1980; Protas, 1979). The rationale for the approach is based on the designers' belief in their own methods as objectively valid and the designers' belief in the role and behaviour of language and communications during information seeking events.
There are two fundamental differences proposed here that put designers at odds with end-users. The first is that system designers and system users have a different approach to human understanding - the designer's rationale for truth, comprehension, and truthfulness is the degree of fit that a set of semantic primitives have to a mathematical model. The end-user's rationale includes comprehensibility, utility, trust and rationality. How well will this information, if adopted, help one's performance in one's life-world? The second is a difference in the unit of analysis. To most designers, language is subjected to tests of "technical rationality." Word units, or semantic tokens, are taken in groups removed from other contexts, and processed. To users, it is the utterance situated in a relational context. From this dichotomy perhaps the real difference is the ability to redeem validity claims as Habermas suggests: "Understanding an utterance is to prove its relevance" (Sperber & Wilson, 1986). A first step in understanding, then, is a channel to permit raising and redemption of validity claims. Likewise, there needs a conceptual model of appropriate interaction shared by all parties. Stated differently, both parties (information system and end-user) are viewed as self-advocating agents, capable of presenting utterances, accepting responsibility for the utterance's comprehension and truth, willing to permit challenges to the truthfulness (sincerity) of the utterance, and acting in accordance to normative right.
To examine this further, we need to establish what it means to be a communicatively competent agent, what is involved in the shift in relationship between speaker and hearer and the resultant shift in power, and situate this all within the technical confines of computer-based information systems.
Communicative competence is defined as covering "a person's knowledge and ability to use all the semiotic systems available to him as a member of a given socio-cultural community" (Hymes, 1974). Where rule-based grammatical approaches define all types of possible well-formed sentences, an individual's needs may defy the types of regularity found in such systems. Consider an individual using language to resolve his own immediate, temporary needs, the nature of the interaction with another person, and the possible outcomes of the interaction. The individual's limitations, purposes, and ability to interpret the implications of the other participant's speech in large measure determine the outcome of the interaction. The speaker (information seeker)'s communicative competence should, then, be part of any information systems analysis.
This raises several points to consider about any analysis of language that exists outside a socio-linguistic community:
It was suggested above that "experts" trained in scripts have an advantage in managing discourse, to control interactions to achieve ends quickly and efficiently. There are situations where control of discourse is acknowledged and valued, similar to Holmstr�m's discussion of asymmetric two-way relation (Holström, 1995). For instance, a teacher soliciting a correct answer from a class may ask leading questions and accept as correct an answer that s/he anticipates will help advance to the next action in the lesson plan. In a more dramatic example, operators of emergency rescue phone calls ("9-1-1" in the USA) are trained to solicit certain data as semantic primitives: address of the caller, keywords of the crime or accident. Once these have been collected, the operator determines what help to send. In human systems, control of the discourse is usually dictated by the authority (the one to whom the seeker turns for help) and what is extracted from the discourse are semantic primitives which help the authority shape the course of action. In a study of police discourse behaviour in non-emergency situations (Benoit, 1998), it was demonstrated that initially police will solicit primitives which can be converted into police action codes. Questions put to the officer which fall outside the narrow band of what the officer felt was appropriate sparked control behaviour (strategic action). In brief, challenges to authority were met with silence, reference to policy, or invitation to speak to a superior officer. [In some cases, the superior officer was called without telling the questioner.] Questions of linguistic comprehension (e.g., "I don't understand what you mean.") were rephrased. However, if the questioner continued not to understand, the officer's behaviour became defensive.
In communication between speaker and hearer pairs where one party is an authority or where the two parties sense a difference in relational status, the authority side tends to assume managing the discourse, without consent of the other party, through linguistic devices:
It has been discussed above that authorities will seek semantic primitives - usually unambiguous single tokens whose meanings when used in the expert's linguistic context provide sufficient meaning that the authority can pursue further action.
Commonly both human and machine-based information systems rely on semantic primitives as the unit of analysis. Even among sets of semantic primitives, there needs to be some coherency. Coherency relationships (Sanders, 1997) entail cause-and-effect relationships. Sanders describes how utterances' semantic elements can have implied cause-and-effect or speakers can choose to present explicit ones. The implied relationship is easiest to manage by a speaker in order to suggest interpretations by the hearer. For instance, "The neighbors aren't home. The lights aren't on." is interpreted by the hearer as "Because the neighbors aren't home, the lights aren't on." The problem is that the utterance could be counterfactual: the neighbors are indeed home, they just went to bed early.
Utterances are larger linguistic units than semantic primitives and also can be tied together through strategic action to lead a hearer to an interpretation. Explicit coherency relationships, on the other hand, are felt by the hearer to bind the speaker more tightly to responsibility for the utterance. Explicit coherency will shake up the discourse when semantic primitives and other evidence (such as that provided by private knowledge) seem counterfactual to the hearer. "The neighbors aren't home, I saw them leave, and the lights are out."
Habermas formalizes the coherency of utterances when he describes speech acts as bound to the speaker. "Close the door" for instance is interpreted as "[I hereby ask you] to close the door." Habermas emphasizes the multiplicity of ways people insert meaning into their use of language by the fact that the speaker made the utterance.
Similarly the utterance "it's cold in here" bears no semantic relation to the speaker's wish to have the door closed. (The speaker is relying on the illocutionary force of the utterance - that is the fact that the speaker says something is intended by the speaker to bring about a change in the state of affairs.)
Some effects of this understanding of how language is used are:
Truth, truthfulness, and normative right, then, can be claimed in communication, but the hearer's knowledge may not be able to evaluate the claims. Reduction of the claim to truth conditionals may yield results which are incomprehensible to the user, or whose "truth" runs counter to other evidence presented during the discourse or to the user's own knowledge. Computerized information systems evaluate truth-conditionals of language as semantic tokens and present retrieval evidence regardless of conflicts in the underlying evidence for the claim or contrary evidence in the display.
The only patently verifiable element in a communicative action is the hearer's sense of relationship with the other party. Utterances can mount challenges to the sincerity of the expert (system) provided the expert is willing to continue the discourse and not resort to strategic action.
The final challenge to communicative action is the physical channel between speaker/hearer and the use of the channel to permit discursive interaction. Without a channel for the presentation and redemption of validity claims, the communication is necessarily strategic. As the police example earlier suggests, there are degrees of amounts of interaction. Mere availability of a channel leads people to a different sense of relationships between parties. A web site, for instance, presents a company's products and services. Perhaps there are opportunities (links) to e-mail. These suggestions of interaction opportunity lead people to ascribe qualities to the company. But in the end it is the company which determines the type and amount of interaction and is actual strategically. Among peers, however, interaction tends to be more fluid - both parties tend to listen as much as they speak. In critical theoretical terms, the discursive interaction is the pragmatic context of justification.
Experts (and seekers) can minimize the need for challenges by incorporating into utterances, explicit coherency relations, and with that responsibility for the utterance. By operationalizing critical theory's perspectives on language to study real-world interactions between communicatively competent parties, it is possible to cast information seeking events along three axes:
Information seeking events are considered here as communication interactions based on three constructs derived from critical theory: relational status (RS), discursive interaction (DI), and language use (LU). Figure 1 displays an array of real-world interactions considered from the strength of each of these constructs. The degree of relational status and discursive interaction increase moving along the x-axis. Moving along the y-axis progresses from entirely semantic/syntactic to pragmatic language use. The axes do not reflect a chart with absolute x,y coordinates but suggest that information seeking in general occurs as individual events with various properties of RS, DI, and LU. In the chart symbolized by cubes is a variety of possible individual information seeking events, between a speaker and hearer pair, specifically an information source or expert and a seeker or end-user.
The placement of these information seeking events is theoretical, based on the above discussion. To investigate the structure of an RS/DI/LU oriented model of information seeking, a critical theoretic-based model hypothesizes that the cases may be grouped and labeled. For discussion, one group is labeled "high-quality" and the other "low-quality". The low quality interactions are those that cluster on the bottom right of the graph. By imagining that information seeking events have RS, DI, and LU properties, and theorizing that information seeking cases may be organized into two a priori categorizations of high- or low-quality interactions, eight possible combinations are possible. Table 1 outlines the interaction types.
| low DI | low LU | low RS |
| low DI | low LU | high RS |
| low Di | high LU | low RS |
| low DI | high LU | high RS |
| high DI | low LU | low RS |
| high DI | low LU | high RS |
| high DI | high LU | low RS |
| high DI | high LU | high RS |
A research project was conducted in two steps to test the validity of the alternative communications model and the individual communication behaviour in the model. The first was a content analysis of the information seeking interactions. The events were chosen from real-life examples that represent the range of interaction types described in Table 1. Each case involved two participants, a "seeker" and a "system representative" or expert. The cases were categorized a priori as being either "high quality" or "low quality" interactions, that is, likely to foster, or hinder, discourse and by extension meaning construction by the seeker, according to the expected levels of the three independent constructs (RS, DI, LU). In the content analysis a coding scheme was designed to measure DI, LU, and RS and used to analyze the transcripts. Code frequencies were tabulated (Benoit, 1998), and descriptive and comparative statistics, as well as intercoder reliability coefficients were calculated.
In the second step of the study, the hypothetical "high quality" and "low quality" categories were confirmed by a discriminant analysis.
Intercoder reliability was determined, code frequency indexes found in the content analysis were compared to the a priori categorizations using a chi-square to see whether the expected frequencies of DI, LU, and RS differed significantly from the actual frequencies found in the analysis (Hair, et al., 1994). The discriminant analysis, which highlights the differences between groups, suggested which of the three theoretical constructs (DI, RS, and LU) were likely to be the strongest discriminators between the hypothesized high- and low-quality interaction types. Table 2 outlines the distribution of test cases.
| Relational Status (RS) | ||||
|---|---|---|---|---|
| Discursive Interaction (DI) Less Equal | Discursive Interaction (DI) More Equal | |||
| Language Use (LU) | Low | High | Low | High |
| Semantic | - - | - + | - + | + + |
| (Case Nos.) | (1, 2) | (3, 4) | (5, 6) | (7, 8) |
| Pragmatic | - - | - + | - + | + + |
| (Case Nos.) | (9, 10) | (11, 12) | (13, 14) | (15, 16) |
| Note: + means likely to encourage meaning construction. - means less likely to encourage meaning construction. | ||||
Transcripts of each case from the above table were analyzed for specific linguistic and communicative behaviour. The choice of behaviour to look for derives from Habermas's communicative action, specifically content codes, purpose codes, and context codes. The specific codes were derived in accordance to normal content analysis methods (Sacks, 1972; Chafe, 1993; Croft, 1996; Blackmore, 1992).
Content codes address the language use construct: whether an utterance is interpreted primarily as a comprehensible and communicable semantic expression or a pragmatic expression.
Purpose codes consider the DI: whether an utterance is intended primarily to convey belief, or a clarification intended by the speaker to disambiguate some particular word or phrase; whether the utterance refers to shared knowledge evident in the discourse (is internally reflexive) or a reference to private knowledge, external to the shared discourse.
Context codes consider whether the utterance is used in a general way, to introduce redundancy in the discourse to help maintain its coherency, or is a specific appeal to the shared goal of the interaction, or is an assumption of the leadership role in guiding the information seeking event.
The transcripts were coded by volunteer codes, trained in definitions of each code and the use of a decision table in how to apply the codes. From their work, the constructs were given operational definitions, outlined in Table 3.
| Construct | Operational Definition |
| DI | greater amount of turn-taking; percentage of turns; relative percentage of statements coded for discourse coherencey (utterance contributes to evoluation of discourse; opportunity for warrants in the information seeking event. |
| LU | relative percentage of statements coded for goal-directedness; relative percentage of statements of semantic-based, internally-referential, and externally-referential codes |
| RS | relative percentage of statements speaking from "role" (authority function; utterance reflects sense of equity between participants); relative percent of pragmatic-based,clarifying, or belief statements |
Several statistical tests were performed to verify the utility of the data and the results. A Q-Q test validated the intercoder reliability. A paired-samples t-test indicates the normal distribution of the data (SPSS Inc., 1998). Two chi-squares were calculated to test whether the variable groups were statistically independent. The data gathered by the coders were deemed robust and independent and so meet the assumptions of the discriminant analysis.
The discriminant analysis tests the ability of cases to separate from each other and form different groups around a hypothetical center. Two results of the discriminant analysis are presented here: the correlation matrix and the discriminant function.
The correlation matrix shows the Pearson product-moment coefficient for each code. All correlations are significant at the .0005 level, unless otherwise noted. Strong correlations indicate related codes; negative correlations indicate an inverse relationship between the codes. Table 4 shows each code combination.
| s | p | b | c | i | x | m | f | g | |
| s | 1.00 | ||||||||
| p | .070 | 1.00 | |||||||
| b | .608 | .324 | 1.00 | ||||||
| c | .700 | .417 | .522 | 1.00 | |||||
| i | .067 | -.512 | -.317 | -.324 | 1.00 | ||||
| x | .268 | .789 | .170 | .410 | -.141 | 1.00 | |||
| m | .644 | .238 | .567 | .607 | -.174 | .452 | 1.00 | ||
| f | .298 | .466 | .288 | .269 | -.081 | .372 | -.075 | 1.00 | |
| g | .672 | .440 | .474 | .615 | .167 | .471 | .430 | .014 | 1.00 |
The canonical discriminant function organized the codes into groups (Klecka, 1990). Codes {p, i, c, g, (m)} formed one group; leaving {(m), b, x, f, s} in the other. The "m" code falls evenly between the two groups. The discriminant function suggests strongly that language use is the greatest influence in communicative interactions. The results also suggest that interactions are very sensitive to excessive references by speakers to private knowledge, belief statements, and the sense of role or function of the participants.
The test cases' a priori division was hypothesized as how well certain linguistic behaviour would foster or hinder successful interactions (the high and low quality groups). The discriminant analysis shows two clear divisions in the data, with a 90.65% agreement to the predicted group membership. Table 5 indicates the statistical predicted group members compared to the hypothesized division.
| _______________________________________________________________ | ||
| Actual Group | Predicted Group Membership | |
| (low) | (high) | |
| 1 | 87.5% | 12.5% |
| 2 | 6.3% | 93.8% |
| _______________________________________________________________ | ||
| Note: 87.5% of the low quality group was predicted
correctly. 93.8% of the high group was predicted correctly. Overall, the percent of "grouped" cases correctly classified was 90.65%. | ||
| _______________________________________________________________ | ||
Finally, the all-groups stacked histogram and distribution of cases within the two groups are plotted to associate the groups' behaviour as theoretically helping or hindering communications in information seeking events (Figure 2).
The purpose of this project was to establish elements of critical theory as a framework for analyzing communicative action between two parties, one an expert, the other a client of the expert. The project's goal was also to identify which elements of linguistic behaviour are most condusive to successful interactions between communicatively competent entities. Critical theory provided a theme that language use, relational status and discursive interaction are constructs which can be manipulated to achieve one's ends (strategic communication) or to achieve mutual understanding (communicative action). Properties of strategic communication were identified and labeled "low quality"; communicative action's properties were labeled "high quality." A statistical analysis of real world discourses that had high/low qualities of RU, DI, and LU demonstrated that certain linguistic acts actually do cluster. The high quality group emphasized pragmatic language use, utterances that clarified semantic primitives, expressed the speaker's belief, and permitted the "raising and cashing of validity claims." The low quality group emphasized using semantic primitives, references to private knowledge, did not clarify ambiguities, did not respond to attempts for cashing-in validity claims, and relied on the speaker's (authority) role to manage the discourse. According to Habermas, the high quality group would be communication oriented to emancipatory interests while the low quality group would represent a power imbalance.
The project also considered that information seeking was a special form of generalized communication. Information seeking by definition means one party is expert and the other is not; and that the client's reliance upon the expert places the expert in role where s/he may control the client's ability to understand. Avoiding responsibility and issues of control may be by accident or by design but in any case the opportunity is built-into the social role of the expert's group and accepted without critique by the client.
Therefore the expert's role warrants investigation. For information seeking contexts, the creators of information systems - the group identifiable by their work product, computerized information systems - accidentally influence the successful use of information system because of the designers' understanding of the role of information systems and of language use between communicative pairs.
If Habermas's expression of language use between expert and client is valid, which I believe has been demonstrated, then critical theory's model of language use may be a viable model for the design of information systems.
Critical theories on speech, especially Habermas's interpretation of linguistic behaviour, has been tested in many human-computer interactive environments. AuramŠki, et al. (1998) developed an office information system on speech acts. Dietz and Widdershove (1991, 1992) take the idea further into the use of computer-human interaction in cooperative work environments. Mingers (1980) attempts to integrate critical perspectives into software systems methodology and operational research.
Information systems design, despite considerable research into biological, psychological, cognitive and belief-revision aspects of human-computer interaction, has not fully explored information seeking from communications research. Human-human communication models have been shown to be difficult to implement or of limited utility (Lievrouw & Finn, 1990) in human-computer design. As this paper has discussed, some aspects of human-human information seeking interaction do significantly influence meaning construction. If these results were adopted into system design, they could serve as a via media between human-human models and deterministic human-computer ones. But a new model necessitates a shift in how designers perceive language use and the domain of action of the end-user.
The four themes of raising and redeeming validity claims that Habermas describes (truth, truthfulness, comprehensibility, and normative right) can be successfully integrated into an actual information system.
There are first some technical considerations to be mentioned. It is not feasible that computer interaction should mimic all the dimensions of human-human interaction, so a modest, computationally effective method needs to be proposed. The first element in this proposal is a valid philosophical orientation, such as the one offered in this paper, which can be integrated into the education of systems designers. The second is entirely a physical element: a communications channel. In this case we do not mean actual synchronous communication with the designer, but instead programming for interactivity built-into the application that anticipates and responds openly to the types of questions that may be raised by a user based on comprehensibility, truth, truthfulness and normative right.
Communication actions by the interface reflects decisions of the designers. As such, the designers bear the responsibility for validity and other entailments of a speech act, but do not under present design models. As Schmidt (1997) pointed out promoting user and designer to "sub-systems" in the same system may contribute to a "relational coupling" that encourages relationships.
Should designers view information seeking as a product of their relationship with the end-user, we avoid the pitfall attendant in questions of whether machines can communicate, and focus on the influence relationship has on communication and meaning construction.
The first entailment is comprehensibility, a feature addressed through natural language use (dialog boxes, labels, etc.) or the selection, design and layout of semiotic elements in the interface. Does the set of utterances (visual and textual) represent a set of semantic primitives whose message is unambiguous? Are there accidental (or deliberate) coherency relations intended by the designer to influence interpretation of the message?
Second is the issue of truth. In an on-line information system, the retrieval set determined by the system and sent to the user is response to the query is a communicative act with entailments of truth. Does this set constitute verifiably true elements? For instance, are these resources truly about the topic of interest? In other words, a channel is included in the behaviour of the system so users can redeem claims when counterfactual evidence is presented.
Truthfulness is the sincerity of the intention of the on-line system's designers. End-users of computerized information systems do enter into a relationship of trust with the designers to the extent that the system has been programmed with the interest of successful, useful information retrieval. Moreover, the system-side truth-conditions (its own conditions of coherency) can be provided so the user can interpret the rationale for a system-provided speech act.
Normative right, like truthfulness, speaks to the larger social concerns and may be viewed as part of the schemata of information providing experts. Information systems present themselves as service-oriented, socially-recognized opportunities for expert-seeker interaction. But as has been suggested, it is the system-side that dominates and influences the interaction by controlling opportunities to raise and redeem claims. An alternative is to reconceive of the role of a given information. By making plain the purpose and possibilities of the information system, both user and designer create a shared schema, whose scripts are open to both parties.
These elements have been successfully introduced into an on-line systems (). The use and success of the interface with end-users is the subject of a different project and its results are not presented here. In brief, the information resources are XML files with embedded triggers for hyperlinks or dialog boxes.
Hyperlinks are used to link the retrieval result to support evidence of truth. For instance, if the screen text presents a proposition (e.g., the capital of Norway is Oslo), the predicate is linked to another form of validation (in this case, an encyclopedia article).
Dialog boxes are called when the user questions truthfulness, comprehension, and normative right. The current version of the interface is limited to responding to questions of "why", such as "why are the data presented this way?", "how was this retrieval set prepared?" and the like.
Among the variety of models used in information systems design, most emphasize information seeking as manipulation of semantic primitives, whose goodness of fit to the query representation is determined unilaterally. The loss of linguistic elements necessary for interpretation is addressed through research in end-user behaviour controlled through the interface. An alternative model was presented based on the properties of actual communicative acts between information seekers and expert providers that shifted the goal of the interaction to mutual understanding between communicatively competent agents. The research strongly suggested which linguistic elements favour end-user meaning construction and, by extension, the critical theoretic goal of emancipation.
Note: when this paper was published in 2001, the author was Assistant Professor, College of Communications and Information Studies, University of Kentucky, USA.
How to cite this paper:
Beno�t, Gerald (2001) "Critical theory as a foundation for pragmatic information systems design.". Information Research, 6(2) Available at: http://InformationR.net/ir/6-2/paper98.html
© the author, 2001. Updated: 2nd January 2001
Contents |
|
Home |