Vol. 11 No. 2, January 2006
| Vol. 11 No. 2, January 2006 | ||||
Introduction. This study investigated unmonitored referrals in a nationwide, collaborative chat reference service. Specifically, it examined the extent to which questions are referred, the types of questions that are more likely to be referred than others, and the level of user satisfaction with the referrals in the collaborative chat reference service.
Method. The data analysed for this study were 420 chat reference transaction transcripts along with corresponding online survey questionnaires submitted by the service users. Both sets of data were collected from an electronic archive of a southeastern state public library system that has participated in 24/7 Reference of the Metropolitan Cooperative Library System (MCLS).
Results. Referrals in the collaborative chat reference service comprised approximately 30% of the total transactions. Circulation-related questions were the most often referred among all question types, possibly because of the inability of 'outside' librarians to access patron accounts. Most importantly, user satisfaction with referrals was found to be significantly lower than that of completed answers.
Conclusion. The findings of this study addressed the importance of distinguishing two types of referrals: the expert research referrals conducive to collaborative virtual reference services; and the re-directional local referrals that increase unnecessary question traffic, thereby being detrimental to effective use of collaborative reference. Continuing efforts to conceptualize referrals in multiple dimensions are anticipated to fully grasp complex phenomena underlying referrals.
The collaborative virtual reference service is an innovative reference service that can bring many benefits to both service users and libraries. Thanks to collaborations among libraries that share their staff, technology and other resources, no longer are library patrons confined to the services and resources of their local libraries. Patrons now have the opportunity to ask questions of institutions in remote locations for specialized information that would not otherwise be available to them. In many collaborative services such as 24/7 Reference, patrons can access the service without being restricted by time and location. Thus, library patrons could obtain help for their information needs via the help of participating libraries in a reference consortium even when their local libraries are closed.
From the library management perspective, libraries can deploy staff more efficiently by participating in a bigger pool and embrace otherwise more costly virtual reference services. Besides cost reduction, a member library of a consortium could answer difficult subject-based research questions more easily by referring the questions to subject specialists in other libraries in the consortium. However, it seems that a large proportion of questions asked in a collaborative service are simply referred back to the home libraries because they should be resolved at a local level (Coffman 2002; Helfer 2003). Unfortunately, there is little monitoring of further transactions to ensure whether the patrons received satisfying answers.
Unmonitored referrals, originally defined by Dewdney and Ross (1994), refers to a situation where the reference staff gives the patron a call number or refers the patron to a source in the library thought to contain the answer but does not follow up or check to make sure that the source is not only found but actually answers the question. These unmonitored referrals seem to be frequently observed in a collaborative virtual reference setting when 'outside' library staff refer the patrons back to local libraries. If a large proportion of those referrals are made back to the local libraries, however, that would not be an effective allocation of resources.
More importantly, the referrals, if unattended, can easily frustrate the users of collaborative virtual reference services. While reference questions are often answered by outside library staff, many patrons would still view the service as a local service. Whenever the questions require local resources and services to answer adequately, library staff have to refer patrons back to their local libraries to complete the answer. Would the patrons still feel satisfied with a delayed answer? In fact, patrons' frustration caused by being transferred from one point to another in the midst of referrals is not unfamiliar. This frustration is equivalent to our everyday experience from using automated customer services on the phone or asking directions on the street.
Although some studies have attempted to identify the types of questions that are effectively answered in virtual reference, few have examined the types of questions that are more prone to referrals, or user satisfaction with referrals in the context of collaborative virtual reference. Perhaps this is because referral are felt to be legitimate answers, although the notion has yet to be validated. Dewdney and Ross (1994) pointed out the problem of a high proportion of unmonitored referrals (42%) at the physical reference service desk. They maintained that reference staff tend to make unmonitored referrals as a way of circumventing the reference interview. Because few studies have examined the unmonitored referrals in collaborative virtual reference service settings, the present study investigates the extent of referrals and user satisfaction with referrals in these types of services. Specifically, the current study focuses on the following three questions:
Research Question 1: To what extent are questions answered as referrals and to what extent are questions answered completely during collaborative virtual reference sessions?
Research Question 2: Are certain types of questions referred more frequently than others? If so, what types of questions are they?
Research Question 3: Are there any differences in user satisfaction levels betweentransactions answered with referrals and transactions answered with completed answers?
The findings of the study will inform us about the question types that are frequently referred in the collaborative virtual reference service. The results will also inform of the extent to which referrals are made in collaborative virtual reference and the efficacy of the referrals as an acceptable answer category from the users' perspective. It will also help local libraries design their services for optimal question traffic management in the collaborative service.
Virtual reference services have become a critical component in current reference practice where the intent of libraries is to provide a fully functioning user service in the digital library environment. The rapid growth in the service has been made through both local and national efforts (Kresh 2002). Some critics question the cost-effectiveness of virtual reference services because of low usage and high cost (Coffman 2002; Coffman & Arret 2004). Based on a cost-benefit analysis, Coffman and Arret suggest the costs associated with software, training and staffing are 'a pretty expensive way to answer a question' and may not justify continuing the service.
Others argue that virtual reference is a niche service that will merge into a part of reference services just as happened to telephone reference services in the 1930s (Kern 2004). In particular, responding to the negative view toward virtual references, many suggest joining a consortium as a way for those libraries that cannot afford a stand-alone virtual reference service (Bailey-Hainer 2005; Kresh 2002; Tenopir 2004). Examples of collaborative models include an international library network project, the Collaborative Digital Reference Service (CDRS), the CLEVNET Library Consortium of Ohio public libraries, and AskColorado. Because of many benefits that collaborative service brings to both library management and user services, it is estimated that approximately seventy-six collaborative virtual reference services were in operation through the active participation of libraries of various types and sizes as of November 2004 (Sloan 2004).
Reference staff seem to make referrals for their patrons for various reasons. The cases can be divided roughly into two types: expert research and re-directional local referrals. Expert research referrals relate to the situations when the questions are referred to subject specialists who could answer with their high level of expertise and resources. Difficult research questions would fall into this type. The referrals in the other type, re-directional local referrals, relate to the situations when the answers require specific local knowledge or locally-restricted databases. Questions that would fall into this type include questions about local library events that may or may not be listed on the library's Website or patrons' requests to check overdue fines. Despite both being referral, each type of referral could have considerably different implications for reference service delivery. The former type of referral enables member libraries to coordinate human expertise and resources in order to answer the questions that are otherwise unanswerable. In this case, patrons are linked to richer resources and subject specialists.
In contrast, re-directional local referrals can be ineffective or even result in a disservice to the patrons. That is, patrons who access the inter-institutional collaborative network may not necessarily understand the seamless nature of the service that goes beyond the walls of their local library. It is possible that people who access this service through their library portal would ask simple, circulation-related questions (e.g., overdue status), anticipating an immediate answer. However, many times, these questions cannot be completed by outside staff who do not have authority to access patron records. In this case, the patron should be referred back to her or his local library to get the final answer. If a high proportion of the questions submitted to collaborative services are referred to a local library for this latter reason, it becomes an ineffective deployment of resources for the local library. Having this concern, some libraries may decide to remain local for their virtual reference service, understanding that a high proportion of their reference questions relate to their library resources and services (Kibbee et al. 2002).
In the current literature, there are not many empirical studies that have examined the proportion of referrals in reference services. Among the few, Ross and Nilsen (2000), in their unobtrusive reference service evaluation study, reported that 37% of the total reference transactions were unmonitored referrals. This finding is based on 261 reference transactions that occurred at physical reference desks in both public and academic library settings.
As libraries adopt collaborative virtual reference services, librarians tend to expect more research related referrals because collaboration and communication among institutions have become much easier and seamless in the online network. Interestingly, a rather reverse finding was reported in a recent study by Nilsen (2004) who compared the unmonitored referrals between physical and virtual reference settings. While more than one-third of the transactions have unmonitored referrals in the physical setting, the proportion was somewhat lower in the virtual setting, comprising 28% of a total of 40 transactions. In her updated study with more transactions, unmonitored referrals were reported as 38% of a total of 85 virtual reference transactions where chat reference comprised 25 transactions (Nilsen 2005). Since this small sample-size research appears to be the only one that investigated referrals in virtual reference setting, referrals in collaborative virtual reference are still largely unknown.
Are there any specific types of questions that result in more referrals than others in the collaborative virtual reference setting? In other words, would referrals be contingent on the types of questions? Indeed, most factual and subject-based research questions are expected to be answered during the session by most participating librarians, especially when the consortium is equipped with a shared online catalogue and consortia-shared databases. Among these questions, complex subject-based reference questions would be more likely to be referred to subject experts within the consortium. However, this type of question was reported to be very infrequent among the transactions. For example, Lee (2004) reported that research level questions comprised only 3% of all transactions in a virtual reference environment at an Australian academic library.
A previous study suggests another type of question that would result in high referral rates. According to Kibbee et al. (2002), a service that generates a high proportion of local questions argues against the inter-institutional collaborative service because the questions would not be effectively answered by outside librarians in the consortium. This is because questions that involve local knowledge (e.g., history, events, or services) or questions that are answerable only by local staff (e.g., use of locally restricted databases or access to patron accounts) tend to result in more referrals back to local libraries. Yet, these speculations should be empirically tested to identify the types of questions that produce more referrals in a collaborative service. The results will serve as important managerial information for designing the practice and policies for virtual reference services.
While the professional community generally encourages the use of referrals as an alternative to actual answers (Reference and User Services Association 2004), Dewdney and Ross (1994) revealed a pitfall of the referral practice by reporting its ineffectiveness from the users' perspective. In fact, this contention is supported by a collaborative online chat reference use study (Kwon 2004). According to the finding, a mere suggestion of alternative resources or making referrals was not a positive predictor of user satisfaction. That is, unmonitored referrals do not warrant user satisfaction because the question might or might not be ultimately answered.
A handful of studies has investigated the impact of referrals on reference effectiveness in traditional reference settings. In the study by Dewdney and Ross (1994), most patrons who received unmonitored referrals reported that they found nothing or very little helpful material when they had to search the sources to which reference staff referred them. In fact, Douglas (1988) categorized these referrals as one of the six types of reference service failures. Furthermore, Murfin and Bunge (1984) reported that the average reference success rate dropped from 69% to 22% when the librarian was busy and, as a result, made suggestions that were not followed up, instead of actually helping the patron with the search. Murfin and Bunge maintained that the practice, policy or necessity of directing the user rather than accompanying the user on the search was a factor that caused librarians to perform far below their potential. All of the above studies indicate apparent problems in the effectiveness of referrals in the traditional physical reference setting. Yet, there is still a question of how patrons would feel when their questions were referred as opposed to when their questions were answered completely. Furthermore, it is also a question how patrons would feel about referrals in a collaborative virtual reference setting.
While many studies have examined the user satisfaction with virtual reference in general, few researchers have investigated the satisfaction of the patrons whose questions are answered as referrals in the collaborative virtual reference setting. The present study investigated this issue empirically.
The present study examined chat reference services delivered through the Broward County public library system in Florida. As the largest library system in Florida, it has 33 regional and branch libraries. Since August 2002, the system has used the 24/7 Reference chat reference service delivered by the Metropolitan Cooperative Library System (MCLS), an association of libraries located in the greater Los Angeles area, funded by a Federal Library Services and Technology Act (LSTA) grant.
The data analysed for the present study were on-line chat reference transactions initiated by the patrons of the Broward Country library system, along with online survey questionnaires, which the service users completed voluntarily. The questionnaire was devised and supplied by 24/7 Reference and popped up upon completion of each transaction. While the patrons were mostly the users of the library system, the reference staff who provided the service were from forty-nine library systems across the United States participating in the MCLS 24/7 Reference programme. During the six-month research period between January and June 2004, a total of 1,667 transactions took place and 1,387 were analysable as being either completed or transferred transactions. In order to analyse the influence of referrals on user satisfaction, all 420 transactions that had a corresponding questionnaire that chat service users had submitted immediately after their session to report their satisfaction with the service were chosen. This amounted to 30.1% of the total analysable transactions. By having a chat reference session transcript with a matching user satisfaction survey questionnaire, the researcher was able to examine what question was asked in a reference session, whether the question was answered completely or referred, and to what extent the patron was satisfied with the answer or referrals she or he received.
User satisfaction was assessed by three indicators: satisfaction with the answer, perceived staff quality, and willingness to use the service again. These indicators were measured by patrons' responses to three questionnaire items. First, satisfaction with the answer was measured by asking the question, 'Were you satisfied with the answer you received to your reference question?' Respondents answered the question by choosing one from 'Yes', 'Not sure' and 'No'. The result shows that, among the 417 respondents who answered this question, 65.2% reported that they were satisfied with the answer received, and 21.1% of the respondents were not sure whether they were satisfied or not. Finally, 12.6% of the respondents reported that they were not satisfied at all.
Perceived staff quality, the second indicator of user satisfaction, was measured by the user response to the question, 'The quality of the library staff service in answering this request was _________'. The answer choices were 'Excellent', 'Good', 'Average' and 'Poor'. According to the finding (N = 416), 68.2% of the respondents answered that the librarians handling the reference questions were 'Excellent'. About 19.5% answered 'Good' and 11.3% answered 'Poor.'
Finally, Willingness to use the service again, the third indicator of user satisfaction, was measured using the questionnaire item, 'Will you use this service again?' The answer choices were 'Very likely', 'Maybe' and 'Never'. 77.2% of users answered that they were very likely to use the service again 19.0% answered maybe, and only 3.8% said they would never use the service again (N = 417).
The overall user satisfaction was computed by summing up the above three indicators (Footnote 1). Measurement research literature indicates that composite variables yield scores that are generally more valid and reliable than does a single item (DeVellis 1991).
In order to determine the reliability and validity of the scores yielded by the composite variable, a reliability test and a factor analysis were conducted, respectively. As a result, a Cronbach's alpha coefficient of .85 was obtained, indicating that the composite variable generated scores were reliable. The factor analysis revealed that the three items were represented by one factor, with structure-pattern coefficients of .868, .916, and .876. This factor explained 78.64% of the total variance. This high validity score confirms that the composite variable is measuring a single construct, suggesting that the composite variable is a valid measure of user satisfaction. Finally, the mean of the overall user satisfaction was 12.69 with a standard deviation of 3.44 in the range between a maximum value of 3 for 'highly dissatisfied' and the minimum value of 15 for 'highly satisfied.'
Answer completeness refers to the level of completeness that library staff answers to the patron's inquiry during the reference transaction. A content analysis of 420 online chat reference transaction transcripts was conducted in order to code the levels of answer completeness. A total of 16 different levels of answer completeness were identified from the initial coding (See the first three columns of Table 1).
Later, the initial sixteen levels of answer completeness were collapsed into four broader categories: completed answer, partial or no answer, referrals, and problematic ending (See the last three columns of Table 1). This reduction in the categories helped to remove some ambiguities in the micro-level coding. For example, when a transcript showed an abrupt ending, it was not clear whether it was patron's termination due to delay, librarian's premature ending, or technological failure. Thus, a broader category of 'problematic ending' incorporates all of these possibilities constituting an abrupt ending. A fewer number of answer categories also enables bivariate data analysis that is necessary to answer the research questions. Thus, the final coding for answer completeness categories were defined as follows:
- 4.7: Asks the patrons if additional information is needed after an initial result is found (e.g., 'is there anything else I can help you with?')
- 5.1: Asks patrons if their questions have been completely answered (e.g., 'did you find what you needed?,''does this completely answer your question?')
- 5.2: Encourages the patrons to return if they have further questions (e.g., 'If you don't find what you are looking for, please come back and we'll try something else.')
- 5.9: Takes care not to end the reference interview prematurely.
A content analysis of transaction transcripts was undertaken to identify the types of questions that are submitted to the collaborative virtual reference. A total of fourteen categories of question types emerged initially. Again, the fourteen categories were collapsed into five broader question types: simple, factual questions, subject-based research questions, resource access questions, circulation-related questions, and local library information inquiries. Below list the definitions of the five types of questions with examples:
Inter-coder reliability tests were undertaken to ensure the consistency of coding for both question types and answer completeness. The tests show the level of agreement between two independent coders. To test the inter-coder reliability, the primary researcher coded the entire 420 transactions. Subsequently, the second coder, a reference librarian who received training for coding the RUSA guideline categories, coded every fifth transaction of the 420 transactions (n = 84). This number comprised 20% of the total transactions, which is a recommended percentage for social science research (Neuendorf 2002). A widely used Cohen's Kappa (k) was used as the inter-coder reliability index for this study and was calculated using SPSS 13.0 (SPSS Inc. 2004). Cohen's Kappa coefficient was found to be 1.00 for question type and .75 for answer completeness, indicating high agreements between the two coders in classifying both variables.
Research Question 1 examines 'To what extent are questions answered as referrals and to what extent are questions completely answered during collaborative virtual reference sessions?' Both Table 1 and Figure 1 show the results of answer completeness based on total transactions (N = 420).
| Initial coding category | Freq. | Percent | Final coding | Freq. | Percent | |
|---|---|---|---|---|---|---|
| Answered completely | 204 | 48.6 | Completed answers | 237 | 56.4 | |
| Answered & transferred | 12 | 2.9 | ||||
| Answered & referred | 21 | 5.0 | ||||
| Ended with partial answer | 12 | 2.9 | Partial or no answer | 20 | 4.8 | |
| Unanswered or failed | 8 | 1.9 | ||||
| Part answered & transferred | 4 | 1.0 | Referrals | 122 | 29.0 | |
| Part answered & referred | 11 | 2.6 | ||||
| Unanswered & transferred | 6 | 1.4 | ||||
| Unanswered & referred | 5 | 1.2 | ||||
| Referred or suspended | 84 | 20.0 | ||||
| Transferred | 12 | 2.9 | ||||
| Patron disconnected before answered | 8 | 1.9 | Problematic ending | 41 | 9.8 | |
| Uncertain ending | 1 | 0.2 | ||||
| Disconnect due to delay | 19 | 4.5 | ||||
| Librarian's premature ending | 7 | 1.7 | ||||
| System failure | 6 | 1.4 | ||||
| Total | 420 | 100.0 | Total | 420 | 100.0 |
These findings revealed the answer to Research Question 1; referrals reached up to approximately 30% of all transactions in the collaborative virtual reference service. In contrast to referrals, completely answered transactions comprised 56.4% of the transactions, which is very close to the well-known 55% reference success rate (Hernon & McClure 1986; Saxton & Richardson 2002)
A profile of questions that public library patrons asked of collaborative virtual reference was found from a content analysis of transaction transcripts. Figure 1 presents the findings (N = 415).
As shown in Figure 1, circulation-related questions were the most frequently asked questions of all (48.9%), followed by subject-based research questions (25.8%), simple factual questions (9.6%), resource access (8.9%) and local library-related information inquiries (6.8%). It should be noted that the high proportion of circulation-related questions seem to partly attribute to the fact that the circulation page has a link to the service.
Research Question 2 relates to whether certain types of questions are more frequently referred than others. If there are, what types of questions would they be? Figure 2 shows the degree of answer completeness by question types.
Figure 2 demonstrates that both simple factual and subject-based research questions were completely answered over 70% of the time and their average referral rates were 5% and 11.2% of total transactions, respectively. In the case of circulation and local library information inquiries, answer completion rates were approximately 50% and referral rates were 44.8% and 39.3%, respectively.
To determine a systematic association between answer completeness and question types, a chi-squared test was conducted using a significance level of .05. According to the test result, there was a statistically significant difference in answer completion rates across the five question types, χ2 = 71.616 (12 d.f, N = 415), p < .001. The effect size (Footnote 2) associated with this relationship, as measured by Cramer's V, was .24, which suggests a small to medium effect size. This result indicates that the degree of answer completion is highly correlated with the types of questions. Specifically, this result shows that referrals occurred far more frequently in circulation and local library information inquiries while substantially fewer referrals were made in both simple factual and subject-based research questions.
Research question 3 relates to whether there is any difference in the user satisfaction between referrals and completed answers? To answer this question, the level of user satisfaction was compared across the four answer categories. User satisfaction was highest when the questions were completely answered (Mean = 14.21; SD = 1.88; n = 237), followed by referred (Mean = 11.83; SD = 3.40; n = 122), partial/unanswered (Mean = 10.68; SD = 3.91; n = 20), and problematic endings (Mean= 7.54; SD = 3.87; n = 41). With these results, an analysis of variance (ANOVA) test was conducted to determine if the user satisfaction of referrals is statistically significantly lower than that of completed answers (Figure 3)
The ANOVA test results revealed a statistically significant difference in user satisfaction across the four answer categories at the significance level of .05, F(3, 416) = 80.341, p < .001. A post-hoc test of the overall ANOVA test was conducted to determine how the four groups differ by employing Games-Howell test. The Games-Howell test is considered to be robust when sample sizes and variances are not equal across compared groups (Field 2005; SPSS Inc. 2004). The test result revealed that user satisfaction was rated into three different levels, as shown in Table 2.
| Answer completeness | User satisfaction | ||
|---|---|---|---|
| High | Middle | Low | |
| Completed answers | 14.21 (1.88)* | ||
| Referrals | 11.83 (3.40) | ||
| Partial or no answer | 10.68 (3.91) | ||
| Problematic Endings | 7.53 (3.87) | ||
| *The numbers in parentheses are standard deviations. | |||
As Table 2 shows, patrons who obtained complete answers were most satisfied among all groups. Patrons who received referrals were significantly less satisfied than the patrons whose questions were completely answered; however, patrons who received referrals fell into the same satisfaction level as those whose questions were partially answered or unanswered. More importantly, the current study revealed that users were far less satisfied with the referrals than with completed answers and experience only about the same degree of satisfaction as users receiving either a partial answer or no answer at all. Finally, all three groups were statistically significantly more satisfied than patrons who experienced a problematic ending.
This study investigated referrals in a nationwide, collaborative virtual reference service. Specifically, it examined the extent to which questions are referred in the collaborative virtual reference environment, the types of questions that are more likely to be referred, and whether patrons would be equally satisfied with the referred answers as with complete answers. This study employed both content analysis of the transcripts of 420 virtual reference transactions that occurred between January and June 2004 and corresponding self-reported survey questionnaires that were submitted by the patrons after each transaction.
According to the results of this study, approximately 30% of the questions were answered with referrals. Circulation-related questions were more frequently referred back to the local libraries than simple factual or subject-based research questions, mostly due to restriction of access to patron records. The latter two types of questions are non-local, generic questions in nature. These types of questions tend to be more completely answered than either circulation or local library information inquiries which involve more locality-specific knowledge. These findings indicate that questions involving local resources tend to create more referrals than questions that involve generic knowledge. More importantly, the current study revealed that users were far less satisfied with the referrals than with completed answers and experience only about the same degree of satisfaction as users receiving either a partial answer or no answer at all.
Regarding this problem of lower user satisfaction with referrals, libraries participating in collaborative services could consider several approaches for handling this problem. An immediate measure would be to specify the scope of questions explicitly and effectively on the virtual reference service Web site. As another approach, Braxton and Brunsdale (2004) suggested that virtual reference services ask users to check their question types and to describe their questions using a Web form where users can also read the scope of questions available from the service.
Probably a more effective measure to correct the phenomenon of problematic referrals would be to place the links to the virtual reference service in the most natural and pertinent locations, that is, at the point of need. A problem that seems to have contributed to the high volume of circulation-related questions in this study is the location of the link placed on a circulation page. Whenever patrons encountered a circulation-related problem, they seemed to click on the link to get help. However, a large proportion of these circulation-related questions were answerable only by authorized personnel in the patrons' home libraries.
This finding supports Wells's (2003) argument that proper location of the links to the virtual reference services can enhance service effectiveness. According to Gray (2000), both the placement of links to the services on individual Web pages and the number of links to the service throughout the Web site directly affect the amount of question traffic. All these findings in the literature suggest a possible influence of link locations on the types of questions asked of virtual reference.
However, the results of this study do not support Gray's other recommendation, 'services that provide links on the bottom of all pages, on a main navigation bar, or on a help menu featured throughout the site will provide users with more opportunity to ask questions at the point of need' (Gray 2000: 369). According to the findings of this study, not all links placed at the point of need helped the patrons especially when it does not guarantee answers. For instance, the present study reported 45% of questions asked via a link on the circulation page were answered as referrals and therefore patrons had to contact their home library again. These findings also suggest that a link on the circulation page unnecessarily increases circulation question traffic within the collaborative service. As the current findings indicate, user satisfaction with referrals was significantly lower than completed answers. Patrons seemed to have experienced confusion and frustration or felt they had wasted their time in this process. Referrals made back to the local library would work against the anticipated benefits of effective allocation of resources. Regarding this finding, a usability study could be conducted to empirically examine relationships between the locations of links to virtual reference and the service effectiveness.
Despite the apparent problems in some referrals, they seem to be generally encouraged and reference staff seem to believe referrals are legitimate answers to reference questions (Reference and User Services Association, 2004). However, the fact that librarians are advised to offer referrals when they do not provide the answer does not necessarily mean that referrals are effective in all circumstances. The findings of this study suggest that librarians should be able to distinguish between two types of referrals: the 'expert research' referrals conducive to collaborative virtual reference services (e.g., sharing expertise and resources in answering difficult subject-based research questions); and the 're-directional, local' referrals that increase unnecessary question traffic, thereby being detrimental to effective use of collaborative reference (e.g., referring patrons back to local libraries to resolve circulation disputes). A practical implication of this finding is that two types of referrals should be properly addressed in reference staff training so that staff can make referrals most effectively. A future study could compare the two types of referrals to further understand contingent conditions of those referrals. In relation to this, another interesting agenda of further research would be to compare the patrons and the reference staff for their perceptions of and attitudes toward referrals.
While there are many expectations and promises in the emerging collaborative virtual reference service, the present study demonstrated some gaps in the current service delivery by investigating referrals. One limitation of the present study, however, is the fact that the study was conducted at a single public library system although the librarians investigated were from forty-nine different library systems participating in a nationwide collaborative chat reference programme. Thus, the findings of the present study should be further confirmed by replicating the study in libraries of different types and sizes.
As another limitation, it should be noted that the present study included only the transactions that had a completed questionnaire. These transactions comprise only 30% of the total of 1387 analysable transactions. Considering the possibility that the patrons who completed the questionnaire may not represent the entire user population accurately, the findings of the present study should be substantiated by further research.
Despite these limitations, the results of this study provide several implications for future collaborative virtual reference practice and research. First, from the practical perspective, the current study empirically demonstrated that generic reference questions, such as simple factual and subject-based research questions, are more effectively answered than locality-specific questions. These findings can help local libraries make informed decisions on such issues as whether they go global or remain local in providing the service. The study also demonstrated empirically that the patrons did not value referrals to a great extent. This finding suggests that librarians should not regard all referrals as equally good alternatives to completed answers. Most importantly, local libraries can minimize confusion and frustration of their users by evaluating the current locations of links to virtual reference and manage the question traffic to provide proper directions for the patrons. This should also reduce the frustration of reference staff who might have to end their sessions without offering satisfying answers and help them to spend their time and expertise more wisely with the questions that they could better answer.
From the reference research perspective, it can be contended that the present study addressed complex phenomena underlying a seemingly simple behaviour of making referrals. The findings certainly redirect our attention to the concept of 'unmonitored referrals' that Dewdney and Ross (1994) had proposed from their research in the physical reference setting. While supporting their claims about the problems in unmonitored referrals, the present study suggests that referrals could bring both positive and negative implications to collaborative virtual reference services. In this regard, the use of referrals must be one such area that Whitlatch (2004) pointed out to assure patrons' satisfaction in the current reference service environment where new technology can create both opportunities and threats. Since further understanding of referrals is critical to maximize the user satisfaction in collaborative services, continuing efforts to conceptualize referrals in multiple dimensions are anticipated to fully grasp complex phenomena underlying referrals.
1. In creating a composite variable, a series of computations was conducted because the three items in the survey questionnaire were measured on a three-point or four-point ordinal scale (e.g., 'satisfied,' 'not sure,' and 'not satisfied' for the 'satisfaction with the answer' item). These ordinal level measures are not suitable to undertake inferential statistical tests, such as ANOVA, that are necessary to answer research question 3 of the present study. To resolve this conflict, the existing three questionnaire items were recoded and transformed into a composite variable to increase the variability of the measure. First, the ordinal level data variables were rescaled on a five-point scale as shown below:
| Negative | Positive | ||||
|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | |
| Satisfaction with the answer | Unsatisfied | Not sure | Satisfied | ||
| Perceived staff quality | Poor | Average | Good | Excellent | |
| Willingness to use the service again | Never | Maybe | Very likely |
As shown on the above diagram, for satisfaction with the answer item, 1 was assigned for 'unsatisfied,' 2 for 'not sure,' and 5 for 'satisfied.' For perceived staff quality item, 1 was assigned for 'poor,' 3 for 'average,' 4 for 'excellent.' For willingness to return item, 1 was assigned for 'never,' 3 for 'maybe,' and 5 for 'very likely.'
After the three items were rescaled on the same five-point scale, they were summed as a single composite variable, with a minimum value of 3 and a maximum value of 15. This procedure of data management allowed the researcher to conduct the necessary inferential statistical tests.
2. The 'effect size' measured the degree of the strength of the association between the two categorical variables. Many behavioral science research papers now report effect size along with significance test result because of the limitations of the significance test. Effect size is useful information especially if the significance test result is sensitive to the sample size, as with Chi-squared tests. For this paper, I measured the effect size using Cramer's V which is a measure of the effect size of the Chi-squared tests (Kline 2004).
| Find other papers on this subject. |
|||
© the author, 2006. Last updated: 10 December, 2005 |
|
