Vol. 12 No. 1, October 2006 Contents | Author index | Subject index | Search | Home

Researchers and practitioners talk about users and each other. Making user and audience studies matter—paper 1

Purpose

This paper is an initial report from an empirical study that is part of a continuing project for which the ultimate goal is to make recommendations for structural and procedural changes in the communicative aspects of the user and audience studies enterprise.

What we mean by communicative aspects is that research consists of many acts of internal and external communicating and, hence, can be examined systematically as communication activity. It is communication activity that at least in part builds and destroys bridges between disciplines and perspectives; it is communication activity that allows us to advance our work; it is also communication activity that deters us from advancing our work. It is communication activity that drives every step of the research process. Rarely, however, does one find systematic attention paid to how communicating is done in the research enterprise. This, at the most general level, is the purpose of this project. (1)

This paper is the first of two interconnected papers appearing in this issue of Information Research. This paper reports on a qualitative study that compared what experts in three fields, library and information science, human computer interaction and communication and media studies, described as their big unanswered questions about users and audiences of information, library, electronic, communication and media systems and texts. It also compares what these experts thought about each other and the difficulties of crossing the divides between disciplines or fields and between research and practice.

The second paper consists of a philosophic commentary focusing on research as communicating, drawing out implications for collaboration across disciplinary and research-practice divides for those of us who conduct and apply user and audience studies. It is in the second paper, for example, that we attend to the philosophic legacies that are at one and the same time strengths of scholarly efforts to acquire more useful pictures of the world and weaknesses inhibiting our attempts to learn from each other and build on each others efforts.

The impetus for the first paper came from the authors' attempts to bring to bear on central communication questions literature from different fields that were, to our minds, addressing the same questions. Thus, for example, if we look at studies on information seeking and use in the fields of communication and library and information science we find that the two rarely meet. Or, we examine the empirical work derived from different theories of what audiences get from media, again the two rarely meet. Or, if we look at work on user uses of information systems versus audience uses of media systems, again the two do not meet. One does not have to look far to find researchers lamenting how we are drowning in avalanches of disconnected empirical findings and producing work that 'piles up, but does not add up'. Further, one does not have to look far to find researchers lamenting the fact that even our many attempts at building theories, models and syntheses to bring order out of the chaos are, in fact, adding to the cacophony. The intent of this paper is to document in the terrain of user studies the fact that, while individually some of us may feel as if we are making progress, collectively we appear to be in disarray. Worse, as we will note below, that disarray embeds within itself some dangers to user and audience studies enterprises. (2)

The purpose for the Keynote paper is to bring to bear on the portrait we develop from this paper a perspective we call a communicative view of communication. Along with the endless calls for more collaboration across our divides, there are, of course, endless calls for more communication as if these processes magically happen by intention alone. Evidence clearly suggests that they do not, that communicating and collaborating across divides is hard. Further, evidence suggests that many of the normative views we have of communication prevent us from looking at communication as process rather than outcome. In this paper, we propose a way of looking at communication that refocuses our attention away from the idea that communication is something we can do to do research better, to the idea that research fundamentally involves a large repertoire of potential communications, some often used, some originally mandated for good scholarship but now too often ignored and some in need of invention.

Rationale and background

Why user and audience studies: given a user by any other name; given information by any other name

In this paper we will apply the term users to refer to a wide variety of persons who have been conceptualized in one way or another as the voluntary or intended users of information, media or communication systems: citizens, employees, patients, patrons, audiences, students, clients, customers, constituencies, recipients and so on. Every institution whose efforts point in some way at groups of such users is building electronic interfaces for doing so; users, too, are building their own interfaces that, in turn, impact upon institutions. For simplicity's sake, we will use the term users throughout this article although we are deliberately enlarging the intended meanings for the term as discussed in this section.

The largest traditional cut across all these persons has traditionally been the focus on users versus audiences with user and audience studies being historically disparate enterprises. Few researchers have focused on both; few have defined the two as synonymous. Users have traditionally been those persons who voluntarily made use of information and communication systems; e.g., library patrons, museum attendees. Or, they have been those persons that an information or communication system was in some way mandated to serve; e.g., the citizens 'served' by a city's or museum system; the students at a university served by an academic library; the employees intended to use an organization's knowledge management system. Users have been thought of traditionally as individuals each of whom at least conceptually was served as an individual and for whom the bottom line has been whether that person was served. Systems have traditionally cared about meeting individual user needs. (3)

The literature is filled, of course, with nuanced arguments which trouble this attention to users. Two primary ones are: a) whether we should be focusing on users, use, uses, usability and so on and what differences these differences make; and b) whether these user-oriented terms privilege systems over people by rhetorically making systems and use of them the center of attention. We acknowledge these complexities although for our purposes here will focus on the primary overarching polarization. (4)

In contrast, audiences have traditionally been defined as amorphous groups of individuals that communication, media and information systems attract or entice with arrays of offerings of particular genres, programme types, or content. Traditionally, systems have not worried about whether any given specific member of an audience was served but rather how many paid attention. The bottom line has traditionally been audience counts even to the extent to which political economists have argued that such systems have little interest in serving individual audience member needs. Rather, the focus has been on audiences as commodities; as units of economic value to the system. Sometimes this economic value is profit oriented, as in television network programming; sometimes it is oriented towards societal cost reductions, as in public communication campaigns. (5)

While, clearly, there have been researchers who have defined these divisions between users and audiences as system-imposed artificialities, there is little doubt that, traditionally, these divisions have driven research in the library and information science versus communication fields with the former focusing on users, the latter audiences. Human-computer interaction is a 'Johnny-come-lately' to user studies transitioning its attentions from physioloigical and functional usability per se to users in response to its enlarged and diversified role in the face of the remarkable diffusion of electronic technologies.

Of course, another polarity, that between texts and messages defined as information versus those defined in some way as not information, is a large part of the foundation on which the distinctions between users and audiences have rested. Traditionally, users were voluntarily getting informed, while audiences were either voluntarily getting entertained or being persuaded or educated to become informed in particular and sometimes deliberately biased ways. These traditional facile distinctions between kinds of information, kinds of media content and kinds of utilities or gratifications users and audiences get from systems have long been challenged both by information science and communication theorists as well as by a host of philosophers. (6)

In the fast-moving maelstrom of the advance of the electronic technologies, these traditionally neat divisions between users and audiences, information and not-information are clearly falling apart. Audiences and users are becoming one. What was traditionally defined as information and its opposite not-information have, much to the distress of those interested in information authority, become, in the practical world of everyday experience, a jumble.

Presenting a detailed review of the literature supporting these premises is beyond our purpose here. Suffice it to say that evidence supports three observations:

1. That users by any other name, citizens, lay persons, patients, patrons, participants, attendees, viewers, game players and so on, increasingly have greater and greater control over their access and use of all manner of information and entertainment systems. In this sense they are no longer best conceptualized as users or as audiences but rather as persons with agency. While in the past access was highly constrained both in space and in time by institutional availability, now these independent agents may surf the planet and beyond at their own whims. This is so even when we acknowledge that all users are unwittingly or wittingly constrained by political economies that limit both arrays of offerings, modes of access and availabilities of specific contents.
2. That information, communication and media institutions are themselves undergoing a maelstrom of change whose directions and ends are predictable in only the most limited of ways. The traditional task divisions between institutions are collapsing; the traditional safe market 'niches' are being eroded; the traditional divisions between kinds of content (e.g., entertainment versus information) are collapsing.
3. That in actuality, when examined from a communication perspective, the historical divisions between kinds of users, kinds of systems and kinds of content were never tenable. The distinctions looked as if they were tenable given the constraints of the designs of information, media and communication structures. But individual sense-makers have always struggled to navigate the crisp divides that institutions have attempted to maintain between their edifices. And, individual sense-makers have always had to struggle with the gaps between and the ways in which societally imposed beliefs and normative forms have been less than useful to their lived experiences. What is happening as the result of the maelstrom of changes impelled by the emerging electronic confluence is that these communication realities have become more obvious. They were in the past relatively quiescent. Now they bubble to the surface in numerous ways. (7)

It is small wonder that in the face of these changes we find ever increasing emphasis being placed in all the professions that create systems designed to serve and/or attract users on understanding users, by whatever names they may be called in order to construct more useful communication systems, designs and messages and in order to find a competitive niche whether for purposes of service and/or profit. (8)

It is also small wonder that researchers in virtually every field that supports these professions are jumping on the user studies bandwagon. This is as true, for example, of the fields of education, art and social work as it is for the three fields, library and information science, human computer interaction and communication, that form our primary focus in this paper. (9)

Do user and audience studies matter? To whom?

The issue of whether user and audience studies matter has for purposes of this paper three interrelated manifestations. One is whether user and audience studies conducted in one discipline matter to researchers in different disciplines. The second is whether these studies matter to practitioners who are the ones designing and implementing systems. The third is whether they matter to society as a whole.

The first two divides are the empirical focus of this paper. There is much informal talk and a smattering of empirical work, which suggests that user studies completed in one discipline matter little to researchers in other disciplines and that practitioners find little value in the results obtained from user studies. Because of the dearth of the evidence amid much informal talk, we explicitly asked both researchers and practitioners in the three fields most involved with user studies, library and information science, human computer interaction and communication, what they saw as the barriers hindering collaboration and research utility across disciplinary and research-practice divides.(10)

While it is beyond our empirical purpose here, it is important to focus attention on the third divide, that between user research and society. We found no commentary that addressed this issue specifically in the context of user studies, but we found a great deal addressing it in the wider context of the social sciences. In general, it can be said that there is increasing lack of interest and increasing distrust in the social sciences, worldwide, by policy makers and the general citizenry.

Indeed, society does ask social scientists to rise to instrumental challenges: find out why bad events happens so we can fix or eliminate them; figure out how to persuade people to do x and stop doing y. Even here, however, the fact is that social sciences produce an avalanche of studies on virtually every topic and that these often disagree not only in method and approach but results. The plethora of studies and the disagreements coupled with the highly specialized vocabularies in which social scientists speak within their discourse communities further exacerbates the problem of the gap between social science researchers and society.

The situation is further compounded by an increase of commentary arguing fervently either for or against the social sciences. Amongst the former, we see arguments for a return to the traditional emphases of the social sciences on more humanistic issues of equity and human struggle and less engagement with nation states attempting to control and manipulate citizens. Among the latter, by far the loudest and most fervent, are those who are either against the social sciences for any but the most instrumental purposes and those who are entirely against them. A sub-argument within this debate is between those who see value in the social sciences as tools of social engineering and those for whom the goals of societal engineering are seen as irrelevant, even detrimental to society. (11)

Increasingly, then, the question is overtly being raised: 'Do the social sciences matter?' While most generally manifested as an argument between more liberal versus more conservative observers, the increasing frequency and fervour of the debate cannot be ignored. It is itself a maelstrom in which user studies are caught. It is certainly a maelstrom which has impacted the availability of funding for user studies as well as restricted the range of questions considered fundable.

It is both the fortune and misfortune of those interested in conducting and applying user studies to be doing so at this juncture in history. The assumption behind this current study is that the forces and tensions enumerated above plague user studies and have given rise to a pulse of interest in finding ways to make user studies matter by fostering collaboration between researchers in multiple fields and practitioners in multiple fields. While there are rampant disagreements on what it might mean for user studies to matter, there is clearly widespread agreement that those interested in user studies want them to matter more. It is this well-meaning pulse with which we agree that was impetus for this study.

Methods

The dialogue project

The study reported here is part of a larger multi-stage project, a dialogic surround of how researchers and practitioners in three fields look at: a) the big unanswered questions about users; b) the gaps that stand between them in finding value from each other's work; and c) the barriers to collaborating in the application of user research to system development, implementation and design.

In order to fully explain why this study and analysis have been structured as they have, it is necessary to set these efforts in the context of the larger project. The project has been designed as a dialogic surround in six stages. The concept of a dialogic surround is illustrated first by describing the project's stages. In the next section, a dialogic surround is defined conceptually in terms of the particular relevance of the concept to the analysis presented here.

• Round 1: the interviews that we are analyzing here constitute the first of the six stages.
• Round 2: volunteers from among the interviewees (working with graduate students and invited colleagues) wrote impressionistic essays focusing on their interpretations of divergences and convergences in the interviews. The opportunity to write impressionistic essays will continue to be open for at least a year. As of this writing, forty-eight impressionistic essays have been written and are posted online. Stage 2 essayists were asked to identify convergences and divergences between the interviews. They were asked not to evaluate or propose their own solutions.
• Round 3: this article is the first systematic thematic analysis drawn from the Stage 1 interviews. It has been designed explicitly to serve the purposes of early dialogic stages. Like the stage 2 essays, it does not propose 'right' answers or take sides. Rather, it focuses very generally on what we learned from our interviewees about their convergences and the extent and diversity of their divergences.
• Round 4: panels focusing on stages 1 and 2 are already included in the programme for the 2006 meeting of the American Society for Information Science and Technology and tentatively planned as a submission for the 2008 meeting of Association of Computing Machinery, Special Interest Group Computer Human Interaction. Full-day working symposia are planned for the 2007 meetings of both the International Communication Association and the American Society for Information Science and Technology. Working titles for all, with relevant field adaptations, focus on: Making user, audience and usability studies matter: Relevance and irrelevance to the designs, policies and practices of library, information, communication, media and electronic systems.
• Round 5: organizing a team to design and seek funding for an international multi-disciplinary dialogue and world conference. (12)

Intertwined between these stages are planned systematic analyses of the more traditional scholarly kind, i.e., those that qualitatively and/or quantitatively critique, evaluate and propose solutions. These will be executed separately and referenced back to the stages above. The stage 2 impressionistic essays and stage 3 thematic analysis focusing on communication issues are defined as necessary for later stages.

The nature of a dialogic surround and its relationship to this study

Drawn from Dervin's development of the Sense-Making Methodology, dialogic surround, as used in this article, is a concept intended to capture in one term the need to procedurally implement repeated rounds of communicating in dialogues where the intent is to enable people to hear how others construct their worlds in such a way that the hearing can become fodder for active sense-making rather than knee-jerk argument and resistance. (13)

It is beyond the purpose of this paper to explicate fully the empirical, theoretic and philosophic underpinnings of the Sense-Making approach to dialogue. Sufficient for our purposes here is specification of these ten procedural mandates which serve as underpinnings for all stages of a Sense-Making informed dialogue. A central assumption behind these mandates is that communicating activities, the verbs of communication, can be explicitly tailored to communicative purposes. Evidence from Sense Making studies of dialogue suggests that there are at least three basic needs in dialogue: 1) understanding and thinking about self; 2) understanding and thinking about others; 3) sharing and advancing own views to others. Our research suggests that these rarely can happen simultaneously. The procedure mandates in a Sense-Making-informed dialogue are:

1. Include both intra- and inter- personal communicating as legitimate aspects of total dialogic processes.
2. Mandate participants to strengthen their communicative repertoire capacities so they can use different behaviour when purposes are to understand and comprehend others versus when purposes are to critique, evaluate, teach, or propose solutions.
3. Implement specific procedures for some rounds in the dialogue in which the only purposes are listening and analyzing as free of judgment as possible.
4. Bracket, for the purposes of dialogue, the traditional emphases in social science interviewing on controlling the assumed biases of selectivity processes, selective perception, retention and recall, in favour of procedures that invite talkers to explain fully how they came to their views.
5. Break out of habitual and often hegemonic modes of communicating by asking different kinds of questions based on human universals of movement through time-space rather than based on the nouns of any given discourse community or institutional formation.
6. Build enforced anonymity into some communicating processes, particularly at the earliest stages of dialogue, so participants will be able to speak freely about both their struggles within systems and their successes.
7. Allow participants (including interviewees) to discuss multiple facets and be free to express inconsistent views which they are invited to navigate with their own explanations.
8. Build in questions that iteratively invite participants to make connections between their internal sense-making and their understandings of the material and structural conditions in which they are embedded.
9. Implement multiple rounds of communicating both within single interviews and within collective dialogues to facilitate the interplay between the conscious and the unconscious in human sense-making.
10. Invite participants to talk not only about what is but also about what might be particularly in the context of the ways in which they have struggled with specific hindrances they have identified.

In the project for which the study reported here serves as stage 1, these procedural mandates were applied. Specifically, they have informed both our approaches for interviewing and analyzing the stage 1 interviews as reported in this paper.

The stage 1 informants

Two groups of informants were interviewed in stage 1. One group, consisting of eighty-three informants, is called the international expert sample. The second, consisting of thirty-one informants, is called the local expert sample. Each is described briefly here. For more detail, readers may consult the dialogue project Website.

International expert sample

The international expert sample is a sample that must be admittedly described as having come together by implosion. The original plan was to use a set of ten to twelve interviews with national and international expert researchers and practitioners in our three focal fields to inform an IMLS-funded research project focusing on the hows and whys of user satisficing of their information needs. By the time interviewing was completed, there were, instead, eighty-three interviews with an effort supported about 10% by grant funds and 90% by extensive volunteer efforts and contributions of resources by some 180 people at multiple institutions. (14)

After completing the first twelve interviews, it had become obvious that there was both more agreement than we expected as well as more disagreement. We did not expect informants, for example, to concur so much on the struggles of crossing field and research-practice divides in user studies. At the same time, we did not expect such widespread divergence in what informants said, the premises from which they started and the very ways in which they talked. Nor did we expect that the divergencies did not link clearly to particular fields or even to practitioner versus research differences.

In addition, unexpectedly, the first twelve informants were extraordinarily excited by their interviews and suggested other possible informants. Given a lack of funds for paying informants and our desire to interview luminaries in the three fields, our primary means of identifying possible informants was by nomination and personal appeal, primarily from the senior author but also by each successive group of informants. Every effort was made, within resource limits, to achieve a reasonably balanced sample. Further, the standard qualitative rule for interviewing cessation was applied; interviewing continued until we concluded that additional effort could yield little additional diversity in input given available resources.

There were some obvious limitations in the process. While the final sample consisted primarily of well known top-level experts, many considered international luminaries, only ten were not current residents of the U.S. On the other hand, an examination of dossiers showed that an additional twelve were recent residents and substantially more than half had had non-US experiences, some quite extensive. Informants insisted we extend our interviewing outside the U.S. because of the obvious globalization of the electronic confluence and known differences in how researchers on different continents conceptualize users and audiences. We accomplished this to a small extent given budget constraints.

The sample also clearly favoured academics, most of whom had current or former practitioner experience. However, only 17% of the sample consisted of experts currently employed full-time as practitioners. While the communication and library and information science field samples were relatively 'pure', i.e., peopled mostly by individuals with advanced degrees at least in these core subjects, the human computer interaction sample was less so. Only about two-thirds of these informants came from computer science and engineering backgrounds. The rest were individuals highly involved in system design who had gravitated there from other backgrounds.

We see these limitations as having relatively little impact on the analyses we present here which focus on communication issues rather than substantive differences. These limitations will need to be actively engaged in planned further analyses of this data. Table 1 describes the resulting sample of international experts.

Local expert sample

The local expert interviews consisted of a total of thirty-one individual and seven focus group interviews with directors of the academic and public libraries serving the forty-four colleges and universities in central Ohio, the designated institutions from which the user sample was drawn for the IMLS-funded research project. (15)

A total of seventy-nine academic and public libraries served these forty-four institutions. Of these, 31 (39.2%) directors or their representatives attended a day long session at which both the individual and group interviews were collected. Of the 31, 24 (77.4%) were academic librarians; 7 (22.6%) were public librarians. Table 2 compares the academic librarian distribution by the institution's student enrollment focus and by institution support base. Other than academic rank (faculty, graduate student, undergraduate), an institution's student enrollment focus was the primary factor used in constructing the stratified proportionate sample of informants in the user study. Institution support base was also an important consideration in representing institutional diversity, particularly in the central Ohio context.

Stage 1 interviews

Interviewing for both the international and local experts was informed by Sense-Making Methodology's interviewing approach. While the venues for interviewing differed, the focal questions aligned as closely as possible. Both sets of interviews focused on how informants viewed gaps: a) in understanding users; b) between researchers in different fields; and, c) between researchers and practitioners. They were also asked for their magic wands for research or procedures that would be especially helpful in alleviating gaps. Each set of interviewing approaches is described briefly here.

Interviews with international experts

A few of the international experts were interviewed in person; most by phone; and all at times set by prior appointment. Interviews were recorded and took, on average, forty-five minutes. Interviewers included both senior project personnel as well as Ohio State University students at all levels, undergraduate, Master's and doctoral.

Interviewers were trained in navigating the Sense-Making approach to interviewing with its emphasis on gaps, how gaps hinder, how participants struggle with gaps, how gaps are or might be bridged and how bridging gaps might help. In Sense-Making's theory of dialogue, it is answers to these gap identifying and bridging queries that allow people from different discourse communities to begin to understand each other. The interview script provided the essential structure organized in four sets of questions as follows:

• About users and audiences
  1. What in your judgment are the big unanswered questions about users and audiences and their uses of library, information and communication systems? For each big question: How would an answer to that question help or facilitate your work or research? In your judgment, what explains why this question is not yet answered?
  2. What are the biggest challenges the advance of electronic information systems present to library/ information/ communication systems in serving users/audiences well? For each challenge: What would help or facilitate facing this challenge?
  3. What would make user research more useful to library/ information/ communication system design and practice? For each suggestion: How would this in particular help?
• About gaps between fields and disciplines
  4. Have you found the many different approaches and fields focusing on user and audience research hindering in any way? For each hindrance: How has it hindered? If you could wave a magic wand, what would help?
  5. Have you found the many different approaches and fields focusing on user/ audience research helpful in any way? For each help: How has it helped?
• About gaps between research and practice
  6. Have you found the different ways in which practitioners vs researchers look at users/ audiences hindering in any way? For each hindrance: How has it hindered? If you could wave a magic wand, what would help?
  7. Have you found these differences in how practitioners vs researchers look at users/ audiences helpful in any way? For each help: How has it helped?
• About the ideal study
  8. If you could wave a magic wand and had any amount of money, what would you like to see a big study of users or audiences do?

The interviewer's goal was to cover all the focal questions without behaving like an interviewing robot because persons operating in different fields do not think about users and audiences in the same way. Often the interviews started with a negotiation about what 'users' or 'audiences' to talk about and what 'systems' and how to name these entities. This was a particular requirement in interviewing some communication academics who defined themselves not as focusing on users or audiences but rather on human beings (in various sub-categories) who happen to use systems in their life struggles and journeys.

Interviews with local experts

In the Sense-Making approach to focus group interviews, informants are simultaneously involved in self-interviewing (through keeping diaries or journals) and group discussions. For this study, on the day the local experts convened, informants were given individual workbooks with one page for each focal question. Individual journal entries involved informants in entering their 'sense-makings' as they came to them in four different boxes as each focal question was discussed.

1. My thoughts in answer to these questions.
2. What I strongly agreed with and why.
3. What I strongly disagreed with and why.
4. What I would most like to see your project do [in its user study].

The ten focal questions replicated the focal questions used for the international expert interviews although stated somewhat differently so as to apply more directly to the practice context.

1. Big unanswered questions about users
   1. What are the big unanswered questions about users?
   2. How would answers help?
   3. What explains why these questions are not yet answered?
   4. What are the biggest challenges the advance of information systems presents to library and information science in serving users well?
   5. What understandings of users would help us handle challenges better?
   
   
   
2. Gaps between research and practice
   6. What would need to be done to make knowledge about users more useful?
   7. What would make the mountains of conflicting evidence about users more useful?
   8. Are differences between how practitioners and researchers look at users a barrier and how?
   9. Are there strengths in the diversity of understandings about users and how have these been useful?
   
   
3. The ideal help from research
   10. If you could wave a magic wand, what would help you to apply different ideas and research about users to your work?

Participation then proceeded in rounds first focusing on questions 1-5, then 6-9 and finally 10. Each round consisted of solo journal keeping, group discussion coupled with self-journals and group breakout reports coupled with self-journals. At the end of the day, informants handed in their workbooks and the posters from group breakout reports were collected. These once transcribed and edited formed the local expert database.

Interview transcriptions

Both sources of interviewing input, the audio tapes from the international expert interviews and the written input from the local expert interviews, were initially transcribed using the smooth verbatim approach, eliminating repetitions and non-fluencies. Standard quality control procedures usually used in qualitative research were applied. The final transcriptions were content edited to remove non-fluencies and grammatical errors and repetitions common to oral talk while still retaining the informant's words. Final versions removed all nouns and descriptors that might identify informants, their institutions, their specific projects, or their specific specialties when these were unique to them. This was done for two reasons: 1) Because the attention to communication gaps, which was the mandate for this project, required deep anonymity by many of the informants, particularly some of the international expert luminaries. 2) Because the first stage of a Sense-Making-informed dialogue almost always involves anonymity. In this study, this was particularly important because it was vital that interpreters in latter rounds would not be spending their time focusing on which luminary in which field said what and then using stereotypes to frame their interpretations of what informants said. All informants were sent their transcriptions for final consent to participate. All gave consent. Institutional review of contact procedures, consent forms and informant protections were duly approved at Ohio State University for both sets of interviews.

The thematic analysis

The thematic analysis presented here was developed to highlight issues of communication as these informants described them. It was not our intent to fix a systematic description of what informants saw as the substantive nature of their gaps and solutions to them, or to do a comparison of what was said between fields, work foci, or institutional contexts. These analyses are planned for later efforts. Our purpose was to capture a broad impressionistic portrait of the struggles these informants reported having across disciplinary and research-practice divides in their internal and external sense-making efforts, in short in their communicatings.

As an example of our emphasis, readers will see that in the thematic analysis we do lay out a number of themes enumerating various ways in which informants disagreed with each other on what they thought would make user studies better. Our intent in developing this enumeration was to provide a portrait of the quantitative extent to which informants disagreed and the large variety of differing solutions they were playing with. Having a survey of these kinds of differences is an essential starting place for beginning to design more advanced stages of dialogue.

Our approach to extracting themes was based on intersecting the inductive procedures of the method of constant comparative coding developed in early grounded theory with the deductive, meta-theoretic emphases of Sense-Making on gaps and gap-bridgings. Because it is polarizations, both explicit and implicit, enduring or temporary, justified and imagined, that most hinder communication, we adopted the approach often used in applications of the Delphi Method and applied a dialectical lens to zero in on polarizations that leapt off the transcription pages in three senses: 1) those where informants were explicitly comparing two different approaches; 2) those where informants were comparing what is to what might be; 3) those involving absences versus presences in talk, as, for example, where there was much emphasis in the interviews on problems but despite specific interviewing queries relatively little attention to solutions. (16)

Because our emphasis was placed on communication struggles, it was vital that we not homogenize differences. For this reason, we chose the quotable quote as our unit of analysis and selected quotable quotes representing themes roughly in proportion to their presence in the 602 single-spaced manuscript pages of transcription (545 for international experts; fifty-seven for local experts).

Arriving at an approach for presenting the results was not straightforward because our specification of themes was necessarily conceptual and informed by a body of work on the structuring of dialogues for bridging gaps. Because of this our attention to the transcripts was necessarily pointed several abstraction levels above where the usual substantive thematic analyses start. In judging both what major themes to use and what sub-themes to nest within them, we required 100% agreement between the two authors.

The resulting thematic analysis is organized into twelve major themes with from three to nine sub-themes each for a total of seventy-five sub-themes. In order to emphasize the ways in which the themes address our central foci on research as communication and communication as research, we present the themes as if the 114 informants are speaking to us personally, describing themselves in terms of their agreements, disagreements and struggles.

While at a meta-level our themes were necessarily anchored on abstract communication principles, we also deliberately allowed the warrant of the data to mandate detailed sub-themes. We chose this strategy because we wanted at one and the same time to present a thematic analysis which would allow us to draw out implications for communication and to show the diversity within. For this reason we have separated our thematic presentation into two parts. The first is Table 3 below which lists the themes and sub-themes. Deliberately in the main body of this paper we did not attempt to explain and explore the substantive meanings from which we gleaned these themes. In the Appendix we again list the themes and sub-themes, this time illustrated with the 320 quotable quote units which we mined and ascribed to each of them. The Table 3 items are linked to the Appendix so that readers may navigate between. Deliberately again we did not apply another level of conceptual definition and analysis above the exemplar quotes and the themes. In line with the procedures mandated for the dialogue project of which this thematic analysis is a part, the intent is to present the portrait of themes in as non-judgmental and dialogic a manner as possible and to retain not only the most centrally relevant quotes within a sub-theme but those at the margins as well.

Because the theory of dialogue on which this project is based mandates several rounds focusing on listening and understanding before any attempts to engaging in substantive comparisons and evaluations, the thematic analysis presented here is considered one of many. As described above, readers can find on the dialogue project Website a series of stage 2 impressionistic essays mandated to essentially the same task that this thematic analysis addressed with the exception that this thematic analysis represents 100% coverage of the stage 1 interviews and an imposed requirement of 100% agreement between the co-authors. Any reader who wishes to may join the dialogue by applying to become a stage 2 essayist. Instructions are given on the project Website. (17)

We already know, because as of this writing some 48 impressionistic essays have been contributed, that there are mind-boggling differences in what essayists see as the covergencies and divergencies between our stage 1 informants. For this reason, we do not present our thematic analysis as the thematic analysis but rather as one analysis of potentially many that is at one and the same time both highly informed by and highly prejudiced by an anchoring in interpretive communication theorizings as well as untold biases in our own perspectives of which we are not aware.

Conclusion

For those of us interested in user and audience studies, many of us in the three communities studied here naturally feel impelled to intervene with our many suggestions, our well thought out theories, our arduously collected data, of why this or that concept or approach will resolve this or that difficulty or bridge this or that gap.

But, alas, that is communicatively the crux of our problem. Yes, our informants were calling for synthesis but at the same time they were decrying their inability to comprehend syntheses written in other tongues by alien beings. Yes, some wanted more theories and models but some, and sometimes the very same persons, said they were drowning in disconnected theories and models. Yes, they found many things to criticize, about communication across divides, but in actuality they had few solutions. Yes, most of our informants wanted to communicate and share with each other but they saw formidable barriers of power, structure, tradition and habit standing in the way.

Yes, one could see evidence of stereotypes between our divides, how practitioners were more likely to distrust researchers and how researchers were more likely to think practitioners uninformed; how HCI people were accused of caring only about their machines and user studies people were accused of being fluffy; or, how communication researchers did not seem to talk to practitioners hardly at all and were so internally divided that they formed disparate sub-fields within themselves; or, how people in librarianship and information science were accused of being too obsessed with their library institutions.

But, the fact is that these stereotypes were a minor key. In fact, perhaps what struck us most was how more often than not we could not predict what field an informant was from by what the informant said; or how often informants were self-reflexively critical of the constraints embedded in their own discourse communities and purposes. Likewise, we were struck by an overall sense of a collision between negatives and positives. On the negative side, there was befuddlement about the chaos of the state of affairs in user studies, an almost resigned disappointment that communication was not going well either across fields or across the research-practice divide and a sense of hopelessness about the speed and demands of current conditions that could prevent things from improving. Yet, on the positive side was the extraordinary committed engagement of the entire informant pool and the palpable wish to make things go better.

We can offer no magic wands for this scenario. But as communication specialists focusing on the requirements of dialogue, we can propose that the traditional modes of communication serving the user studies research enterprise and, in fact, all social science research enterprises, are not doing the job that needs doing.

It is at this juncture that we turn to our second article, a commentary that focuses on research as communicating. To project ahead to our bottom line, we draw in our second essay on a body of work primarily from the fields of philosophy and communication that says we must find ways to re-establish the importance of some communicating practices that used to be normative in the social sciences and have become marginalized in current research practice; and we must find ways to invent new communicating practices so that that communication we do about and for our research is more fruitful.

In the stage 1 interviews, we could sense the two polar dialectics that confound human efforts to communicate. One is the polarity between uniformity and diversity. On the uniformity side, there is the emphasis on homogenization, on orienting towards correct and right answers and on achieving consensus and avoiding dissensus. On the diversity side is the emphasis on hearing different interpretations so often challenged as ending in the chaos of solipsistic individuality.

The second polarity is between humanistic approaches to communication and instrumental approaches. The former focus on building empathy and deeper listening; the latter on structuring communications so as to achieve explicit ends, either manipulative (as in propaganda and much advertising) or well-intended (as in public communication campaigns and some management leadership styles).

We will develop an argument in the Keynote Paper which suggests that the choices offered at the ends of these dialectical polarities do not address communication in communicative ways. Rather, we propose that we need to focus on how symbolic interpreting humans move between the individual sense-making and sense-unmaking necessary for collective effort; and how humans are capable of flexibly designing and employing their communication so that these serve different ends.

Acknowledgements

Resources supporting the dialogue project as reported here came primarily from: a) senior author Dervin's Joan N. Huber Fellowship fund; b) the Ohio State University School of Communication; and c) volunteer efforts of some 180 volunteers , faculty, students, consultants, administrators and practitioners in three fields (library and information science, human computer interaction and communication and media studies) located at some seventy-five institutions (universities, libraries, corporations, consulting firms, governmental agencies) in twenty US states and eight countries. About 10% of the support came from the Sense-making the information confluence project which was funded by a grant from the Institute of Museum and Library Services to Ohio State University and by in-kind contributions from Ohio State University and the Online Computer Library Center. That project was implemented by Brenda Dervin (Professor of Communication and Joan N. Huber Fellow of Social & Behavioral Science, Ohio State University) as Principal Investigator; and Lynn Silipigni Connaway (OCLC Consulting Research Scientist III) and Chandra Prahba (OCLC Senior Research Scientist), as Co-Investigators. More information on the IMLS project may be found at its Website The authors owe special thanks to: a) the some 190 students and volunteers who assisted in various ways on the international-local expert dialogue, their names are listed at the project Website; b) OCLC Consulting Research Scientist Lynn Silipigni Connaway who completed six of the international expert interviews; c) the Online Computer Library Center for providing the venue for the local expert focus group meetings and Connaway for facilitating. Special thanks are due to Noelle Karnolt and Tingting Lu, Ohio State University students, for assistance in preparing the final manuscript.

Notes

(1) This paper's impetus rests on extensive reviews and the senior author's continuing 20-year emphasis on the problematics of effective communicating in different contexts, in particular the intersections between all manner of systems (e.g., media, information, library, communication, government, medical, service) and their users (by any other name, e.g., audiences, patients, patrons, participants, citizens). A recent turn in that emphasis focuses on dialogue between experts, across disciplinary and research-practice divides. The project reported here is one outgrowth the article 'Human studies and user studies: a call for methodological inter-disciplinarity' (Dervin 2003) which appeared in this journal. Other recent works in this emphasis include: Dervin (1999); Dervin (2001); Romanello et al. (2003); and Schaefer & Dervin (2003). In addition, Dervin & Foreman-Wernet (2003) compiled a series of earlier works focusing on dialogue as a central requisite of the design of effective: democratic systems, education campaigns, programmes to address literacy issues, Third World development, information systems design and organizational-constitutency relationships. Central to this effort has been a focus on conceptualizing communication as communicative, as internal and external activities that humans pursue to make and unmake sense of their worlds and to both fall in line with and fall out of line with the collective structures in which they find themselves. This view of communication mandates that we look at communication not as an outcome but rather as a series of activities or step-takings (conscious or unconscious, habitual or capricious, designed or duplicated) that may have different outcomes. This formulation suggests thinking about communication as repertoires of internal and external behaviour. Theorists who have informed this position include, in particular: Carter (1974, 2003), Stephenson (1967) and Thayer (1987, 1997). For comprehensive literature reviews, see Dervin & Foreman-Wernet (2003).

(2) One example of a comparison between the library and information science and communication literatures on information seeking and use in the context of health information seeking is in a paper commissioned by the US National Library of Medicine (Dervin, 2001), An example of a comparison of twelve theories of media uses/effects is in Dervin et al. (2005). Examples of authors who have referred to the growing body of disconnected offerings in the social sciences include: Carter (2003) who has admonished for years that we need to stop doing research that 'piles up, but does not add up'. (Carter 2003: 360); McGuire (1999: 386) who called for senior scholars to stop adding more undigested pieces to the pile but rather to turn their attention to observation, syntheses and interpretation; and Hjørland (1996: 52) who challenged that 'We must cease the overproduction of unrelated facts'.

(3) As usual, the academic literature is not a good indicator of trends. The current emphasis in the different fields on the concepts user(s) versus audience(s) supports the traditional divisions of attention. We did a title search for two top-tier journals in each of the three fields for two time periods (1990-95 and 2000-05) looking for use of the terms user(s) or audience(s). In the library and information science field (tapped with the Journal of the American Society for Information and Technology and Information Processing and Management), 100% of 48 articles in the earlier time period used the terms user(s) as opposed to audience(s) and 98% of 86 articles in the later period. For the human-computer interaction field (tapped with Human Computer Interaction and International Journal of Human-Computer Interaction), the comparable figures were 100% of 7 articles for the earlier period; 100% of 40 articles for the later period. For the communication and media studies field (tapped with the Journal of Communication and Journal of Broadcasting and Electronic Media), the comparable figures were 10% of 30 articles in the earlier period; and 14% of 21 articles in the later. For the communication and media studies field, it seems that emphasis on users and audiences has gone down but, in fact, what has happened is that an array of new journals in the field has diffused attention to users and audiences and the dust has not yet settled on the impact. Two examples include: New Media & Society and Journal of Mediated Communication. In general, however, communication and media studies field articles have overwhelming focused on audiences, the library and information science and human-computer interaction fields on users. Where the implosion between these tidy divisions shows is in the trade press, in, for example, the special section in The Nation, 2006 on the 'national entertainment state', which documents not only the increased control over all information, communication and media systems by a few mega-multinationals but also the qualitative blurring of the previously separated functions of information versus entertainment. At the same time the seeming freedoms which the exploding internet offers means we see average citizens taking on functions as journalists , 'a journalism without journalists' as Lemann (2006: 44) called it; or as their own librarians, what Bates (2006a: 29) termed the 'disintermediation of information'.

(4) Numerous authors have interrogated the fundamental concepts: use, uses, users. Especially helpful have been: Capurro et al. (2002); Frohmann (1992); Julien (1999); Pettigrew et al. (2001); Savolainen (2000); Talja (1997); Vakkari (1997); Wilson (1994, 2000). Also helpful has been this edited compilation: Olaisen et al. (1996).

(5) This line of inquiry has a robust tradition. These chapters provide a helpful overview: Murdock & Golding (1995); Smythe (1995).

(6) Considerations of the terms information and knowledge have been fodder for continuing debate in the library and information science field from its origins. Some particularly useful recent writings: Bates (2006, 2005); Hjørland (2002a); Capurro et al. (2002); Olsson (1999). In the communication and media studies field the distinction between being informed and being entertained has been maintained with relative constancy in a genre of research usually labeled as "media uses and gratifications". In this work, information or being informed is considered a separate function of media (usually conceptualized as channels) than escape, diversion, or entertainment. A recent example is Song et al. (2004). A seminal example: Blumler & Katz (1974). Signs that this distinction is being eroded show, however, in a number of terrains. Just as librarians worry about how to maintain an address of information authority, the media world is focusing more intently on such issues as 'the future of fact' (Strange & Katz, 1998).

(7) The literature support for these three premises is given in the notes above. An important point is that from a communication perspective the world as it has been constructed and divided into pieces by our disciplinary boundaries and the systems they support is not the world in which everyday actors make and unmake sense. The structures created by collectivities (e.g., governments, cultures, systems) may be seen, depending on your theoretical perspective, as tight or elusive boundaries within which humans experience the everyday. Most extant approaches to communication focus on one or the other end of this structure-agency polarity. This point will be addressed in our second paper in this series.

(8) Of the three fields that are the foci of this paper, two have had a traditional direct relationship with users or audiences, library and information science and communication and media studies. The impact of the exploding electronic confluence on these fields is awesomely visible. A Google search completed October 8, 2006 using the term 'future of libraries' yielded 60,000 hits; 'future of newspapers', 100,000; 'future of media', 260,000. Examples of articles from library and information science addressing issues relating to whether and how libraries and information systems can compete: D'Elia et al. (2002); Hjørland (2002b). Examples from communication and media studies focusing on media systems: Cohen (2002); Webster (2005).

(9) Examples of studies from other fields that have been hopping on the user study bandwagon: education, McDonald (2004); medicine, Chapple et al. (2002); nursing: Edward & Staniszewska (2000); art: Hsi (2003); Marty (2006).

(10) All three of our focal fields have had a long-time emphasis on applying research to practice, to designing and implementing systems and services as well as campaigns and education efforts aimed at users and audiences. All three have also been traditionally embroiled in tensions that characterize the research-practice divide essentially along four polarities: a) research versus practice; b) research versus design; c) design versus service; and d) theoretical research versus applied. Examples of recent articles in library and information studies and communication and media studies attending to these issues include: Booth (2003); Bryant (2004). Of particular relevance to this study is an article by Smith (2006) which proposes that the standard divide between positivism and interpretivism feeds theory-practice inconsistencies. He proposes critical realism as a resolution. In the earlier stages of our dialogue project, we have deliberately asked interpreters not to propose solutions. The practice-research divide plagues all the applied social sciences: see, for example: Hammersley (2003); Small (2005). A literature review is available in Dervin, et al. (2003). Our study reported here focuses on polarities a, b and d as listed above. We did not explicitly ask informants about gaps between design and service needs although some frontline practitioners did speak directly to this issue. For most, however, the issue of the utility of design to service was foundational to their other attentions.

(11) This, we submit, is the divide that social scientists and their applied fields have been least paying attention to but is probably having the largest impact on our fates. We provide documentation for this argument and discussion of the issues it raises in our second article. We have been informed particularly by: Dreyfus & Dreyfus (1986); Flyvbjerg (2001); Johnson (1991).

(12) The stage two essays are posted online. Descriptions of panels, workshops, symposia and other project developments are available at the same link.

(13) See references listed in Note (1). The Sense-Making approach to dialogue draws upon many foundational sources particularly as referenced in Dervin & Foreman-Wernet (2003). Three that deserve special mention are: 1) The Delphi method, which has been used in research although was developed by the Rand Corporation as a technique aimed at building consensus in policy groups during the cold war (Turoff 2002). A major thrust of its application, in addition to utilizing the idea of rounds, was the bracketing of power relationships by mandating anonymity among participants. This is an aspect Sense-Making has adopted for early dialogic rounds. As normatively used Delphi focuses on building consensus and partitioning dissensus. A later development by Tapio (2002) focuses on establishing not a central consensus but diverse scenarios in an approach he calls 'disaggregative policy Delphi' although the goal is still to solve all inconsistencies. Because of studies (e.g., Schaefer and Dervin 2003) that show that an emphasis on achieving consensus and avoiding dissensus may inhibit communication processes necessary for effective community dialogue, the Sense-Making approach to dialogue deliberately attends to both convergences and divergences and does not demand homogeneity either between or within participants except as it emerges naturally from rounds of sense makings and unmakings. 2) Flanagan's (1954) 'critical incident technique', the cutting edge for its time approach for anchoring interviews to the material situations of interviewees' lives. In a Sense-Making dialogue this is implemented by asking participants to articulate the connections between their own views and opinions and how these relate to their material circumstances. 3) The Frierean theory of dialogue (Friere, 1970, 1983) as developed in his approaches to literacy training and Third World development with its emphasis on the role of talking in generating raised consciouness (or what Friere calls conscientizing) and participatory readiness. In Sense-Making interviews, this is implemented primarily by asking informants not only to explain what they see but the bridges, gaps and inconsistencies in what they see and how these connect to their material conditions.

(14) A complete list of persons who served as advisors and volunteers for the international expert dialogue is available here

(15) A list of the institutions sampled and persons involved in the local expert dialogue is available at the URL listed above.

(16) There are numerous debates and interpretations of the Glaser & Straus (1967) 'grounded theory' approach from its original statement most popularly known as the 'constant comparative method' to more recent debates (e.g., Strauss & Corbin, 1990). Since we did not intend to develop theory but rather a descriptive portrait of the discourse covergences and divergences most applicable to considering problems of communication, we applied the method of constant comparison to our reading of the expert interviews intersecting these readings with Sense-Making's emphasis on gaps and gap-bridgings. Because Sense-Making's approach to dialogue mandates attention to absences as well as presences, our readings attempted to focus both on what was said and what was not said as well as the ways in which informant responses represented contradictory aspects of the same themes.

(17) Instructions for doing so are available.

Appendix

What researchers and practitioners in three fields said about users, audiences, research, service, design and each other: convergences and divergences focusing on research as communication and communication as research

Web Counter

© the authors, 2006. Last updated: 21 August, 2006

online web site analytics