Vol. 12 No. 2, January 2007 Contents \| Author index \| Subject index \| Search \| Home

Students seeking information for their Masters' theses: the effect of the Internet

Paulina Junni
Hanken - Swedish School of Economics and Business Administration, PB 479 (Arkadiankatu 22), 00101 Helsinki, Finland

Abstract

Introduction. The Internet has radically changed the global availability of scholarly publications. Today, a substantial part of the resources accessible for researchers and university students are offered through electronic site licenses, making the supply of easily obtainable information larger than ever. This brings forth an important question: what are the qualitative and quantitative effects of this development on the use of reference material in research and studies?
Method. To address the research question, reference lists of Masters' theses from 1985, 1993 and 2003 were studied in three disciplines: economics, psychology and mathematics, followed by semi-structured interviews of students who had finished their thesis in 2003.
Analysis. The quantitative data were analysed using analysis of variance (ANOVA) with the statistical program SPSS, where the significance of the results was measured with Dunnet's t-test.
Results. The findings showed a substantial increase in the use of scholarly articles as references throughout the studied time periods, although the amount of other information sources had remained largely unchanged. There were also significant differences between the three disciplines in the contents of their reference lists, the amount of Internet resources that students used, how they sought and obtained publications, and how they selected their sources.
Conclusions. The Internet appears to have had a profound effect on the type and quantity of information that students use as references in Master's theses. One of the main problems that students reported was a lack of training in information seeking, and the abundance of irrelevant information on the Internet. Many respondents would have needed additional training on using library databases.

Introduction

There has been a paradigm shift in the process of scholarly communication. Until recently, end-users (readers) were still physically quite restricted to scholarly publications that could only be obtained on paper, for instance in libraries. Today, with the rise of the Internet, the end-user is no longer physically bound to material that is close by, but can easily search, obtain and download scholarly papers in electronic journals, electronic books, electronic archives etc. The Internet is a relatively new channel for scholarly resources, and contains vast quantities of information that vary a great deal regarding its contents, aim, target group, reliability etc. Hence, it is important that the end-user is aware of the diverse information available on the Internet, and educated in the criteria by which the information content should be assessed (Chapman 2002).

Students have only recently received the opportunity to use the Internet to seek and obtain scholarly material and, consequently, knowledge on how effectively they make use of this channel is limited. Students' information seeking culminates as they work on their Masters' theses. In this sense, studying the effect of the Internet on students' work in this phase is particularly interesting.

The aim of this article is to report on a study of the effect of the Internet on students' choice of information sources and the methods they used to find and obtain publications. The customs and the requirements of students when writing a Master's thesis may significantly differ between disciplines; therefore, this study covers students from three different disciplines: economics, psychology and mathematics. The topic is examined both on a general level, as well as individually for each discipline.

The research questions that guided this study can be grouped into two sets. The first set was examined by studying reference lists of Masters' theses from the years 1985, 1993 and 2003. The specific research questions concerned how use of the Internet has influenced the end-users choice of information sources. More specifically: Have the different types of references available changed in popularity? Has the number of references changed? and Has the average age of the references read changed? The second set of research questions was examined through phone interviews with respondents who had written their Master's thesis in 2003. More specifically: Which methods do ends users apply when seeking or retrieving information? To what extent do they use the Internet to make searches? and Which variables affected the end-users' choice of information sources? It was also of interest to find out what formal training students had received in information seeking, and whether they perceived their skills to be sufficient in order to find the information they needed? These research questions were formulated following a review of the literature, which is presented in the next section.

New channels for scholarly publications

The Internet is an attractive medium for seeking and obtaining information, for the following reasons:

The Internet is accessible twenty-four hours a day.
The end-user does not have to visit a library.
It is possible to find and obtain information relatively quickly and conveniently.
The end-user can choose between saving, printing or reading the information from the computer screen.
Sources on the Internet are often more up-to-date than sources in paper format.

The Internet has made it possible to disseminate information electronically.As a result, the end-user has more options in seeking and obtaining informationthan ever before. He or she can, for instance: borrow material from libraries, download free information on the Internet, or obtain information from databases on the Internet that Universities subscribe to (i.e., full text databases such as Proquest, Ebsco and Science Direct). The end-user can also obtain information by e-mail about potentially interesting articles through article alerting services of scholarly journals.

The Internet has in particular affected the supply of scholarly journals to university students. End-users have a much larger selection of scholarly journals to choose from than, for example, ten years ago. The Internet has affected the supply of scholarly journals in the following way:

The supply of scholarly journals has increased. The bundling of journals as part of license agreements gives students the opportunity to seek and browse through thousands of journals.
The supply of other scholarly publications, such as conference proceedings, working papers, manuscripts etc., has also increased, as end-users can browse them on the Web. Scholarly journal articles can also be obtained from non-commercial, open access e-journals, or from authors' or universities' home pages or open archives.
End-users can also obtain information from companies' Web pages, news Web pages, communities of interest or practice etc.

Recently, universities in Finland have gained access to large databases containing scholarly journals, through the FinnElib consortium. These packages include various databases, such as Proquest, Ebsco and Science Direct (from now on referred to as university databases). Since students and the personnel of the universities are the only ones allowed to make use of the databases, the supply of scholarly journals has only increased in the universities of Finland. As a consequence of these developments, Hedlund (Personal Communication) argues that the amount of irrelevant information has greatly increased. In order not to be overwhelmed by the vast amount of information it is crucial that students are aware of the information sources that are relevant for their discipline, as the supply of scholarly journals for university students is now larger than ever.

According to Chapman's (2002) study, students are often not able to make sound judgments about the quality and relevance of information. This may be a problem for many end-users, since the quality, aim and target audience of the information may vary significantly. Griffiths (2003), who studied users' perceptions of JISC's (Joint Information Systems Committee) information services, found that students are not very adept at evaluating the quality of online scholarly publications, since they seemed confused about the actual meaning of quality scholarly publications. In this sense it can be considered very important that students are taught to seek and evaluate information on the Internet (and elsewhere) effectively.

Researchers' use of scholarly publications on the Internet

Many studies have been conducted regarding the type of information the end-users seek and obtain on the Internet and in which circumstances they prefer electronic sources to paper sources ( Applebee et al. 1997; Applebee et al.. 2000; Björk and Turk 2000; Fescemeyer 2000; Savolainen 1999; Katz and Aspden 1997; Tenopir et al. 2003).

Björk and Turk (2000) studied how researchers in the domains of construction information technology and construction management obtained publications on the Internet and how they choose their sources. According to their survey, approximately 50% of all the scholarly publications were obtained from the Internet. Fescemeyer (2000), who studied geography student's use of sources on the Internet by analyzing citations from a test, found that 51% of the citations referred to sources on paper, 47% of the sources were from the Internet and the rest was course material.

Applebee et al. (1997; 2000) studied Australian academics' use of the Internet. In 1997 less than 50% used the Internet and only 13% used it daily. In 2000 more than 92% used the Internet and about 25% used it daily. Applebee et al. (1997) found significant differences between how much people from different disciplines used the Internet. The respondents from the Institution of Information Technology used the Internet more often than others. They stated fewer problems with using the Internet, as compared to respondents from the other disciplines and showed more patience with for instance waiting while downloading information. Over 80% of the respondents thought that they had too little time to use the information they had obtained on the Internet. Over 90% said that they did not have enough time to seek various information sources on the Internet, which might be relevant for their field. This might be a sign that the respondents are having difficulties either with finding the appropriate methods for seeking information, or with distinguishing irrelevant sources. On the other hand problems in information seeking can also stem from having insufficient knowledge of the field of study.

Students seeking scholarly publications on the Internet

Several theories and models have been developed on end-user information seeking; among others Ellis's and Haugan's (1997) model on search patterns indifferent phases and types of projects, Wilson's (1999) problem-solving model and levels of uncertainty in information seeking, Kuhltau's (1993) six-stage information seeking model in relation to performing a task and Chu's (1999) model consisting of different phases in the research process with regard to researchers' information needs in each phase. Choo et al. (1998; 2000) combined Ellis' model of information seeking strategies with Aquilar's modes of organizational scanning and extended the new model to include information seeking on the Web. According to their study, the end-user can apply a variety of search methods in different stages when seeking information on the Internet, in order to find the information that satisfies their needs at the time.

Schmidt and Spreng (1996) studied variables that influence end-users' information seeking patterns. Among these are personal variables, or internal variables that vary among different end-users, for instance motivation, attitudes and experience (Schmidt and Spreng 1996; Kling and Callahan 2002), as well as end-users' personality traits (Heinström 2003). An individual's knowledge about the subject and of seeking information online also influences which methods he uses to find information (Kwong 2002; Hölscher and Strube 2000) and how successful he is in finding relevant information (Kuo et al. 2003). In this sense, end-users who have received formal training in information seeking may use different methods for finding information than those who have not. Pennanen and Vakkari (2003) also found that students' ability to turn their conceptual constructs of their research proposals into appropriate query terms significantly affected their search success. Situational variables, or external variables connected to the environment, also affect individuals' search patterns. For instance, time limits, information needs, the supply and access of paper sources versus sources online and access to the Internet (Burnett and McKinley 1998; Crisp et al. 1997; Kling and Callahan 2002; Kwong 2002; Savolainen 1999; Thong et al. 2002).

According to Schmidt and Spreng (1996) the end-user will perform an additional search as long as the marginal benefit of performing the search exceeds the marginal experienced cost. The experienced cost is described as a combination of an individual's access to information, degree of uncertainty and degree of pressure. According to Wilson (1999) information seeking involves uncertainty, which decreases as more information is gathered.

In summary information seeking can be described as a task-related, stepwise process, which is affected by both personal and situational variables, as well as an individuals' experience and knowledge about information seeking and the task at hand. The Internet provides a means to seek and obtain information suitable for information needs in different stages of the search process.

Methods

This study was made as a cross-sectional analysis, with the aim of mapping out and describing where and how end-users (students who are seeking information for their Master's thesis in three different subjects) seek and obtain scholarly publications, what affected their choice of methods when seeking information, retrieving information and their choice of sources. The study that was carried out constituted of two parts, which are presented in the following sections.

Part 1: Study of reference lists in Masters' theses

The data gathering process started in the author's own university, the Swedish University of Economics and Business Administration, in Helsinki, Finland, since one aim was to study students? information seeking and usage patterns in the university's library and it was expected to get a good response rate in the phone interviews in the second part of the study. Since all students in this university were in the field of economics, it was of interest to include other fields as well. The University of Helsinki, which is the largest university in Finland, was chosen as another target. Two other subjects were chosen as comparisons to economics: psychology and mathematics. These fields were chosen as comparison groups because the aim was to compare information seeking patterns and citing patterns of students in different fields, in the hope of getting more reliable and generalizable results. Although the data were gathered from only two universities in Helsinki, the sample represents the population well. First, the University of Helsinki is large and the main university in this region. Second, it can be expected that there are greater differences in information seeking patterns between fields than between universities.

In this part of the study 219 reference lists from Masters' theses were gathered from the subject areas of economics, psychology and mathematics in three time periods: seventy-one from 1985, seventy-five from 1993 and seventy-three from 2003. The aim was to gather twenty-five reference lists from each subject for each year, using random sample as the method for data selection. In some cases there were fewer than twenty-five available masters' theses in one year and in these cases all samples were included. These three time periods were chosen in order to at least partly control for factors, other than the effect of the Internet, which could induce changes in citing patterns. The year 1993 was chosen as a comparison year because this was one of the last years before the World Wide Web began to have effects on information availability. The year 1985 was chosen in order to compare changes between 1993 and 2003 and to find out whether the changes were linear between 1985 and 2003, or whether changes were different or more significant between 1993 and 2003. In other words the aim was to find out which changes could be ascribed to the effects of the Internet and which could have been induced by other factors. For a summary of the data collected, see Table 1:

**Table 1: The sample**
Year	Economics	Psychology	Mathematics	Total
1985	25	25	24	71
1993	25	25	25	75
2003	25	25	23	73
Total	75	72	72	219
Interview sample (2003)	25	25	23	73
Interview participants	15	17	16	48

Based on the reference lists the different types of information sources were divided into the nine categories below and the contents of the reference lists from 1985, 1993 and 2003 were compared.

Monographs and course literature, which includes course books, compendiums and student?s notes from courses.
Scholarly journal articles.
Conference publications, working papers and reports.
Theses (masters', licentiate and doctoral theses).
The news press, i.e., popular articles that are not academic.
Grey literature, i.e., separate reports, talks, lectures and unpublished papers, which are not controlled by any (often commercial) publisher.
Company information, i.e., brochures that describe the activities of a company, annual reports etc.
Official sources, i.e., laws and official statistics.
Other/undefined, i.e., sources whose references are not comprehensive enough and sources that do not fit into the categories described above (for instance computer programs or code).

These sources were either in paper or electronic format, i.e., the grouping of the sources was based on their contents, not on the medium in which the publication was obtained. This was done because the use of electronic sources naturally has increased dramatically since 1993, as electronic sources practically were not used at all at that time. However, it was of interest to analyse which sources had increased or decreased in number between 1985, 1993 and 2003 and the students' preferences for sources. The data were analysed with analysis of variance (hereafter ANOVA) using SPSS (Statistical Package for the Social Sciences).

Part 2: Interviews

The population, from which the interview sample was drawn, included seventy-three respondents who had written their Masters' theses in 2003. Of these forty-eight respondents took part in the interviews (making the response rate 66%); namely fifteen from economics, seventeen from psychology and sixteen from mathematics (see Table 1). Thus the distribution of respondents from the different disciplines was quite even. The distribution of the sexes in the sample was also even: 46% were male and 54% female, although it ought to be noted that all of the respondents from psychology were female. But since the proportion of males in the psychology population was two only out of twenty-five (8%) the sample represents the population quite well. During the first conversation a time was booked with the respondents when they would have access to their Master's thesis. This would help them to refresh their memory and thus give more accurate answers. Three of the respondents were abroad and preferred to answer by e-mail.

The phone interviews addressed questions that could not be answered by analysing the reference lists. These included: the methods used to seek and obtain information and whether the user had received formal training in information seeking. Both quantitative and qualitative data were gathered in the interviews. Likert-scales, ranging from 1-5 (where 1 = never and 5 = very often), were applied to the questions on the methods the respondents used to seek and obtain sources. The differences between the disciplines were analysed using ANOVA with the statistical program SPSS. More specifically, the significance of the results was measured with Dunnet's t-test. Since part of the aim of this paper was to map out the current situation, it was necessary to collect quantifiable data. In these semi-structured interviews open questions were also included, where the respondents were encouraged to tell more or specify their answers (Lundahl and Skärvad 1999).

The interviews with respondents from the Swedish School of Economics and Business Administration were conducted in Swedish, whereas those with respondents from the Helsinki University were conducted in Finnish, because the respondents had different mother tongues. Finally, all answers were translated into English.

Findings

In accordance with Applebee et al.'s (1997; 2000) and Griffiths's (2003) findings there were significant differences between the respondents? information seeking methods, methods of retrieving information and the use of information sources between the disciplines. The data were analysed with ANOVA, in order to find out whether there were any significant differences between the fields and whether there were any significant changes in citation patterns between 1985, 1993 and 2003. The findings are presented in the following order: 1) Methods that students used when seeking information; 2) methods that students used to obtain information; 3) how this is reflected in changes in reference lists between the years 1985, 1993 and 2003 and finally 4) students' training in information seeking.

Information seeking methods

During the phone interviews the respondents were asked what methods they used to seek and obtain information. The goal was to find out how much they used the Internet to find sources and how many sources they actually obtained from the Internet. The respondents were asked to give their answers as an integer, ranging from 1 to 5, where 1 = never and 5 = very often. The results are presented in Table 2. Significant results between individual groups (as obtained from the ANOVA) are marked with a star (*), in which case the marked groups differ significantly from each other. There are two cases, 1. and 3., where the differences between the fields do not stand out from the table. In these cases the analysis presented below the table provides further details.

**Table 2: Methods of seeking information**
Information seeking method	Average	Economics	Psychology	Mathematics
1. Follow up references	3.8	4.2*	4.3*	2.9*
2. Seek in library (goal oriented)	3.7	3.9	3.8	3.4
3. Subject specific bibliographic service	3.5	3.8*	4.3*	2.3*
4. Browse in library	3.3	3.9*	3.2	2.8*
5. Received source	2.8	2.1*	2.8*	3.5*
6. Follow hyperlinks	2.7	2.3	3.1	2.8
7. General search engine	2.6	3.0	2.3	2.5
8. Hints from others	2.6	2.4*	2.4	3.1*
9. E-mail alerting services	1.3	1.1	4,3*	1.1

The results are presented ranging from the most popular to the least popular method. First the average results are presented, where after the differences between the fields are described in more detail:

The most popular method for finding information was to follow up references in other publications (see 1 with 3.8 in the table above). Many respondents claimed this method to be especially useful in the beginning, when they were not yet familiar with the literature of their subject field. A few respondents said that following references is easy on the Internet, because some references are linked to the cited papers. In this case, although respondents from psychology looked up more sources from reference lists (4.3), as compared to the other fields, the only statistically significant difference was between economics (4.2) and mathematics (2.9). This finding might be explained by the fact that the nature and contents of the Masters' theses in mathematics differs from that in economics and psychology. Whereas the aim in economics and psychology theses is to contribute to and extend existing research, Masters' theses in mathematics can focus on problems that have been solved before. This because in some theses in mathematics students need to prove they have the ability to solve a complex mathematical problem, rather than contributing to existing streams of research. A respondent from economics said: 'I always looked into articles' and books' reference lists. That was one of the best ways of finding more sources'.
All of the respondents sought information in the library in a goal-oriented fashion, using the author's name, the title or the subject (see 2. with a score of 3.7 in Table 2). There were no significant differences between the fields, indicating that the use of library services has remained popular among students, despite the growing availability of information on the Internet.
It was also common among the students to seek information sources through subject specific portals or search engines. These include so-called open access sources (which are free of charge to the end-user) and databases with articles from scholarly journals from commercial publishers (like Ebsco, Psychlit), which are not free of charge but paid for by the Universities (often by licence), see 3 with a score of 3.5 in the table above. There were some differences between the fields in the use of this method for finding information; respondents from mathematics used this method the least (2.3), i.e., less than those from economics (3.8) and psychology (4.3), whereas the respondents from psychology used this library service most. Some respondents from psychology and economics commented that it was unnecessary to use general search engines on the Internet, because it was much easier to find good articles through the databases subscribed to by the University.
Although it was more popular among students to make goal-oriented searches in a library (see 2 in the table) instead of just browsing through information (either by being physically present in the library or by searching a database that covers the library's contents), this method was still quite common (see 4 with a score of 3.3). Respondents from economics relied on this method the most (3.9) and significantly more than those in mathematics (2.8). Some respondents in economics and psychology reported that they visited their university's library regularly to browse through new journals or to get new ideas.
Some respondents received sources directly from professors or friends (see 5 with 2.8). Whereas many respondents in mathematics received most of their sources from their professors or friends (3.5), this was far less common in economics (2.1).
Although some respondents did not like to use general search engines on the Internet, because the searches yielded so many results, some spent a lot of time following hyperlinks on the Internet (see 6 with a score of 2.7). There were no significant differences between the fields in using this method for seeking information and many of those who did use it often, did so in connection with following up references. Yet, many respondents complained that they thought that searching this way was too troublesome and took up too much of their time, since some of them could end up following different links all day and finally only end up with a few relevant information sources.
It was not as common to seek information using general search engines on the Web as one might have expected (see 7 with a score of 2.6 with no differences between the groups). Many of the respondents only looked for information in the Universities' databases, because they experienced it as difficult to find reliable scholarly publications by using for instance Google or Altavista. In contrast, two of the respondents had looked for almost all of their sources on the Web, because there were not enough relevant sources on their subjects in the subscription databases. Two psychology students commented: 'I used this method in the beginning when I didn't know much about the topic, to get hold of background information. But I did not include these sources later on in my Master's thesis'. and 'There was so little valid information on the Internet. I rather searched Psychlit [the university's database] with the use of key words'.
One of the least used methods for seeking information was to look for sources based on advice from friends, professors or experts, or by simply remembering a source (see 8 with 2.6). Respondents in mathematics received significantly more advice and hints about relevant information sources (3.1) than did respondents from economics (2.4). A few respondents from mathematics even claimed that they had found all their sources based on their supervisors' recommendations, or that they simply had received all sources directly from them.
It was least common to use alerting services for articles in scholarly journals (see 9 with 1.1). This method was used extremely rarely. Only two respondents from economics had used alerting services and both said that they had not found many relevant articles this way. As they advanced in their research process, however, they claimed that this service did provide some additional information. This result resembles Choo et al.'s (2000) study, where respondents also used alerting services scantily. This result might be explained by the fact that students who are writing their Master's thesis work quite intensively for a short time, whereas for instance PhD students and researchers want to keep up with development in their field(s) and thus may well find this method more useful.

Use of the Internet and methods for obtaining information

Comparing the share of sources obtained from the Internet to the amount of paper sources was interesting. A problem emerged from the manner in which some sources were referred to in the reference lists. Students are required to include the Web address of a source that has been obtained from the Internet, but this only applies to sources that cannot be found in paper format. For example, students are not required to include the Web address of an article that has been downloaded from one of the databases subscribed to by the University. In this sense the analysis of the extent to which electronic sources appear in the reference lists suffered and the true number of electronic sources remained unclear. Thus, this issue was addressed in the telephone interviews and the respondents were asked to estimate, expressed as a percentage, the sources they had obtained from the Internet.

It was found that the respondents had not included all the Internet addresses of the sources they cited and that they had obtained more sources than presented in the earlier results. Based on the citations in the Masters' theses few sources referred to information on the Internet (approximately 8%). Based on the respondents' own estimations, however, approximately 32% of the sources were obtained from the Internet. The respondents from economics obtained most publications from the Internet (44%), whereas the others obtained less (psychology 28% and mathematics 25%). The share of sources the respondents obtained from the Internet was far less than found, for instance, in Fescemeyer's (2000) study, where the students obtained 47% of their sources from the Internet.

The following table presents the different methods the students used for obtaining information. The methods are presented in order of precedence, ranging from the most popular to the least popular method. Significant differences between the disciplines are marked with a star (*) and are discussed below.

**Table 3: Methods of obtaining information**
Methods of obtaining information	Average	Economics	Psychology	Mathematics
University library/institution	4	3.4*	4.7*	3.8*
University's database	2.7	3.3*	3.6*	1.3*
Publisher's home page	1.7	1.9	1.4	1.8
Author's home page	1.4	1.5	1.4	1.4
Library distance loan	1.4	1.5	1.3	1.3
Ask for paper copy	1.3	1.3	1.4	1.2
Ask for electronic copy	1.1	1.0	1.1	1.1

The respondents obtained most of their information sources from University libraries or University institutions, either by borrowing or copying them (average answer was 4, see table 3). This method was most popular among respondents in all of the disciplines, although respondents from psychology (4.7) used this method significantly more than those in economics (3.4) and mathematics (3.8). The respondents also obtained many publications from full-text databases that the Universities subscribed to (2.7). Respondents in psychology used this method more often (3.6) than did the respondents from economics (3.3), whereas respondents from mathematics used it the least (1.3). Generally the respondents were satisfied with their full-text databases and thought that it had helped them find many relevant sources. In was not uncommon to retrieve sources from publisher's home pages for free (1.7). Respondents mostly obtained these free sources from universities' home pages or from open access e-journals or databases. Some respondents said that information on universities' home pages is more reliable than other information on the Web. Two less popular methods were to obtain information from authors' home pages or to order it from the Internet, through inter-library loans (1.4). Some respondents commented that they though that this method took up too much of their time or was simply too cumbersome. The respondents did not ask for many sources from authors, neither electronic copies (1.1) nor paper copies (1.3). Some said that they did not want to bother the authors, or they thought it took up too much time. Others felt that they should take responsibility of obtaining relevant sources themselves. On average, respondents who had received some formal training in information seeking obtained more publications from commercial publisher's home pages.

Findings from the reference lists

The study of reference lists between the years 1985, 1993 and 2003 showed significant changes between 1993 and 2003, (i.e., when the Internet started to have an effect) in the following ways: the number of sources in reference lists has increased dramatically, the average age of sources has decreased and the types of sources used have changed. These findings will be discussed in the following sections.

Number of sources in the Masters' theses

The average number or sources has significantly increased as an effect of the Internet, being 30 sources in 1985, remaining at 30 in 1993 and increasing to 47 in 2003. When studied separately the number of sources has only increased significantly in economics and psychology between 1993 and 2003, while remaining quite constant between 1985 and 1993: from c.25 in 1985 and 27 in 1993 to c.45 in 2003 and from c.54 in 1985 and 53 in 1993 to c.79 in 2003, respectively. The number of references has not increased significantly in mathematics (being c.13 in 1985, 10 in 1993 and 16 in 2003), which might be due to the fact that the contents, purpose and structure of Masters' theses in mathematics differ from the two other fields. See the following figure:

Figure 1: Number of sources in Master's theses

Although the number of references in Masters' theses has increased as an effect of the Internet, the number of pages in Masters' theses has remained quite constant, (being 70 pages in 1985, 55 in 1993 and 58 in 2003). There was a significant decrease in the number of pages in Masters' theses between 1985 and 1993, although the Internet did not induce it. In other words the respondents referred to more sources, without increasing the length of the Masters' theses, see figure 1.

Average age of sources

On average, the age of the sources read has significantly decreased, probably partly as an effect of the Internet. The average age of sources was 12.0 years in 1985, 12.8 years in 1993 and 9.9 in 2003. Individually the age of the sources read has only decreased in mathematics (from 17.2 and 18.6 to 12.0 years) and psychology (from 11.2 and 12.8 to 9.8), whereas it has slightly increased in economics (from 7.5 and 7.0 to 7.9 years). The decrease in the average age of the sources is not substantial, but has remained quite stable over the past 20 years. These results resemble Tenopir et al.'s (2003) study of scholarly journals, where the age of the sources also remained relatively constant over a period of approximately 10 years. One reason for the small change in the age of the sources may be that the literature in the disciplines has not evolved rapidly and classic publications are still regarded as relevant.

The percentage and number of different types of sources

It was found that the Internet had affected the student's choice of different types of information sources for their Masters' theses. First, the changes in the percentages of different source types in the reference lists are presented, where after the changes in the number of different sources are presented. Comparisons are made between the disciplines. The most pronounced changes in the percentages of different source types in the reference lists are presented in Table 4 below, ranging from the most common to the least common source types cited. Significant differences are marked with a star (*). Note that there was no difference in the use of sources between 1985 and 1993, i.e., all stars refer to differences between 1985 and 2003 or between 1993 and 2003. Below the table follows a discussion about the differences between the disciplines' use of sources.

**Table 4: Percentage of different source types**
Types of sources (%)	1985	1993	2003
Scholarly journal articles	23.2*	27.9	37.3*
Monograph/Course literature	60.4*	58.0*	45.8*
Company information	0.1*	0.5*	2.0*

There were significant variations between the types of sources that were used in the three studied fields. Psychology students used more journal articles (on average 55% of the total number of sources), than those from economics (21%), whereas students in economics used more articles than those from mathematics (13%). On the other hand respondents from mathematics used most monographs and course literature (79%), whereas respondents from economics used less (48%), and psychology students used the least (37%). Although these were the most significant findings, some other differences between the fields appeared as well: Students from economics had more thesis citations (Masters' theses, licentiate theses and doctoral dissertations) than did those from mathematics (2.6% vs. 0.9%). They also cited the news press (i.e., non academic papers), grey literature, company information and official sources (8.2%, 4.7%, 2.3% and 6.9% respectively) more often than did those from psychology (0.8%, 1.6%, 0% and 0.3% respectively) and mathematics (1.1%, 1.2%, 0.3% and 0.4% respectively). These differences in the citation patterns between the fields can probably be explained by the fact that the nature, structure and demands of Masters' theses differ between fields. It is no surprise that economics students, for instance, rely more heavily on official sources, the news press, and company information, since these sources may provide information about the business field or company of interest. Psychology students, on the other hand might rely more on clinical studies cited in prestigious journals, whereas those from mathematics might derive their mathematical formulas and basic theories from monographs and course material.

Despite the differences between the three fields, the most significant affect of the Internet was found to be citations to articles from scholarly journals. As mentioned earlier the supply of articles from scholarly journals has increased among researchers and students, as they now via the Internet can access full-text databases such as Proquest and Ebsco that aggregate publications from thousands of journals. This increased supply seems, in view of the empirical results, to have significantly affected the student's choice of information sources for their Masters' theses. The figure below presents the changes in the number of references between 1985, 1993 and 2003:

Figure 2: Changes in the different types of sources, 1985 to 2003

As illustrated above, the average number of articles from scholarly journals has been 9.6 in 1985, 11.5 in 1993 and has increased to 23.5 in 2003. The number of monographs and course literature in the reference lists has remained quite stable during the last 18 years, as have the other sources. This is not surprising, since students still frequently use the library (see methods for retrieving information) and the Internet has not affected the availability of monographs or course literature. Rather, the Internet has mostly increased the supply of available articles from scholarly journals, which, according to the findings from the interviews, are used a great deal by the students. It might have been expected that the number of grey literature sources would have increased, as they are readily available on the Web. This, however, is not the case. Some respondents said that they did seek background information on subjects through general search engines on the Internet, but that the quality and reliability of these sources was not adequate enough for them to be included in the reference lists. They stated further that these types of sources gave them a good insight into their subjects, especially in the beginning of the research process.

If the disciplines are studied separately it is clear that the respondents from psychology used most articles from scholarly journals (24.2 in 1985, 27.8 in 1993 and 55.1 in 2003), as compared to those in economics (4.6 in 1985, 4.9 in 1993 and 11.2 in 2003) and mathematics (1.3 in 1985, 1.7 in 1993 and 2.7 in 2003). The increase in the number of articles from scholarly journals was only significant in psychology and economics. Again, this might be due to the fact that Masters' theses in mathematics differ in purpose, structure and content from those in economics and psychology.

Training in information seeking

The respondents were asked whether they had received any formal training in information seeking, in order to find out if it affected how students searched, gathered and chose their sources. About half (48 %) of the respondents had attended a course in information seeking and 39 % of the respondents said that they had not received enough training. Some respondents were unhappy with the timing of the training and though that more information about information seeking ought to be given in connection with a seminar or a course in methodology. One respondent from economics commented: 'The training was sparse. I have learned everything myself through trial and error. I received some information in the beginning of my studies, but nothing after that.'

It was found that respondents who had received some formal training in information seeking obtained more sources from the Internet on average. They also cited fewer course literature sources than respondents who had not received any training. But this might also be dependent upon the fact that the respondents from mathematics had received least training and used the largest amount of course literature. Respondents who had attended a course in information seeking generally also used subject specific search engines and portals and the Universities' databases to find relevant information. But these differences might also be ascribed to the fact that respondents in mathematics received least training, used the largest amount of course literature and used subject specific search engines and portals the least to find relevant information.

One finding, which shows the importance of teaching students different methods for seeking and retrieving information, is that many psychology students believed that full text articles could only be attained through computers in the university's library and as a result did not use the library's databases much for seeking or retrieving information from home. In fact students have the possibility of logging on to the university's network from home and thereby getting accesses to the full text articles. Problems or barriers the respondents often mentioned against effective information seeking included, among others:

Difficulties in articulating queries for search engines.
General dislike of using computers and lack of experience.
Information overload: One search on the Internet can provide hundreds of sources of which perhaps just 1 % is relevant.
Availability: Students in psychology felt that access to full-text articles in the subscription databases was too limited.

Students' opinions of the Internet

The respondents had quite different opinions about information on the Internet. Respondents who were not fond of using computers had, quite naturally, a less positive opinion about electronic information sources as compared to those who also surfed the Internet often for other purposes. Respondents in psychology and economics were quite pleased with the academic resources that were offered by the universities' libraries. On average, the respondents were not as fond of searching the Web through general search engines, as they did not think they could find quality material there and the searches took too much time. There were, however, a few respondents who said that the information on the Web was of crucial importance for them, as not much had been published on paper relevant to their subject field.

Some respondents mentioned difficulties when seeking on the Internet. Many said that it is hard to find relevant sources, as a search engine can produce thousands of results. Many pointed out that it is important to be very critical regarding sources on the Internet, as their origin can be quite obscure, although some thought that it is 'up to you' to successfully use the Internet when seeking sources. A respondent commented:

'If you're good at searching you can find pretty good sources. It's up to yourself if you can pick and choose [among the information sources]. Most of my sources are from famous authors. They are more trustworthy and you know what types of sources they are.'

Even though the respondents used the Internet often to seek and get hold of information, it seemed as though electronic sources were used more as a complement, in contrast to a substitute, to sources in paper format, in accordance with the findings of Savolainen (1999). A respondent commented:

Ninety peercent [of the information you find on the Internet] is totally useless when you search it using key words, most of it is useless... It is good as a complement for library services, but cannot replace it.

Some respondents said that they prefer to read their sources in paper format, whether they have obtained it from the Internet or not. In summary it can be stated that electronic sources are not a threat to sources in paper format, but indeed mostly used to complement paper sources.

When you obtain sources from the Internet and print them out you get such large amounts of paper that it gets hard to keep track of the most important paragraphs and texts. In books it is easy, you just put a post-it note between the pages

Summary and conclusions

The findings showed significant differences among the disciplines regarding the methods used to find information sources, although the respondents used very similar methods to get hold of publications: the majority of the information sources are still obtained from a physical library, even though the respondents like to seek information on the Internet. Respondents in economics and psychology obtained more sources from the Internet, whereas those from mathematics did not use the Internet as much for this purpose. The respondents in mathematics also cited the smallest number of sources (sixteen, on average), whereas those in psychology cited the most (79, on average). These differences might be explained by the different research traditions of the disciplines and their demands on the quality and structure of Masters' theses. The most central results are, in summary:

The average number of sources in reference lists of Masters' theses has increased between 1993 and 2003, in the disciplines of economics and psychology. The number of pages in the Masters' theses has not increased in the same period. In other words electronic sources are not substituting other sources, but rather complementing them.
The average age of the sources in the reference lists has decreased in the disciplines psychology and mathematics. Specifically, the age of sources in the reference lists has decreased as an effect of the Internet, as sources on the Internet generally are more up to date in comparison to those in paper format (Chapman 2002).
The number of articles from scholarly journals has increased significantly between 1993 and 2003, in economics and psychology. This effect might be ascribed to the fact that the availability of articles has increased dramatically for students and researchers, who have obtained access to full-text databases such as Ebsco, Psychlit etc.
The number of other types of sources has remained quite stable. This might be explained by the fact that the Internet has most significantly increased the availability of electronic articles from scholarly journals.
Formal training did not affect the information seeking methods the respondents used or how they obtained their sources. But respondents who had received some training did use more articles from scholarly journals and less course literature.
The respondents were not satisfied with the amount of training in information seeking they had received from their university. There seems to be a clear need for more training on information seeking. Many respondents were irritated by the fact that they had received some training in the beginning of their studies, but not much afterwards. Some suggested that training could be given in connection with methods courses or when writing a term paper or Master's thesis. Education concerning technical search methods is not sufficient, as many respondents had problems with dismissing irrelevant information when searching, forming good queries and evaluating the relevance of publications. Training and information about how to be critical when selecting sources and how relevant material can be found is also important (i.e., information literacy).

In summary, it is quite common that end-users search the Internet or online databases in order to find relevant information, thus they need to be taught how to make effective searches and how to evaluate information on the Internet. There has been a general concern regarding the quality and reliability of information on the Web and how skilled students are at evaluating these sources. This study shows that students are sceptical towards information found on the Web, for instance by using search engines like Google or Altavista and generally do not include many of these types of sources in the references of their Masters' theses. The availability of scholarly journal articles has increased as students and faculty have received access to full text databases, such as Proquest and Ebsco. The number of scholarly journal articles in references has increased dramatically, showing that students make good use of these high quality academic resources.

As the availability of different types of information has (and is) increasing, as an effect of the Internet, it is important that students are taught to seek information on the Internet, use the subscription databases and evaluate sources. The fact that many psychology students assumed that full text articles could only be accessed through computers in the university's library and thus did not search for them from home, shows that an increase in available information as such (through the Internet or elsewhere) does not increase usage. In order to increase the effectiveness of end-users' information seeking they also need to be made aware of all the different methods available for seeking and retrieving information.

The major finding of this study was that the citing patterns of students (when writing their Master's thesis) have changed to include more articles from scholarly journals, because the Internet provides students (and other academics) with more sources than any single library has before, which also are quite easy to access. Although the findings were quite clear there are some limitations that should be taken into consideration. First, other factors might have affected student's choice of information sources. As Finnish universities are aiming at internationalizing their research, students might be encouraged to use more information from international journals. The quality requirements of Masters' theses might also have increased between 1993 and 2003, pushing students to obtain information from international journals that are highly regarded. Conversely, the access to large amounts of scholarly journals, provided by the Internet, might also have heightened quality requirements, since it is now much easier for students to obtain high quality information sources. Nevertheless, the causal relationships are unclear and were not studied in this article, which only provides evidence of an increased usage of articles from scholarly journals. Other studies could address this limitation by studying the factors that have affected citation patterns in different subjects. Second, in this paper only students (who were writing their Master's thesis) were included, so information seeking and citation patterns found in this study might not necessarily be generalizable to other information users, for example academics or industrial researchers. Other studies could include respondents from other categories as well, in order to shed some light on the similarities and differences in information seeking behaviour between different groups of people.

References

Applebee, A., Clayton, P. & Pascoe, C. (1997). Australian academic use of the Internet. Internet Research: Electronic Networking Applications and Policy, 7(2), 85-94.
Applebee, A., Clayton, P., Pascoe, C. & Bruce, H. (2000). Australian academic use of the Internet: implications for university administration. Internet Research: Electronic Networking Applications and Policy, 10(2), 141-149.
Björk, B-C. & Turk, Z. (2000). How scientists retrieve publications: an empirical study of how the Internet is overtaking paper media. The Journal of Electronic Publishing, 6(2). Retrieved 24 December, 2006 from http://www.press.umich.edu/jep/06-02/bjork.html
Burnett, K. & McKinley, E. (1998). Modelling information seeking. Interacting with Computers, 10(3), 285-302.
Chapman, L. (2002). Russian roulette or Pandora's box: use of Internet as a research tool. Paper presented at VALA 2002. 11th Biennial Conference and Exhibition, 6-8 February, 2002, Melbourne. Victoria, Australia: Victorian Association for Library Automation Inc. Retrieved 24 December, 2006, from http://www.vala.org.au/vala2002/2002pdf/18Chpmn.pdf
Choo, C. Detlor, B. & Turnbull D. (1998). A behavioral model of information seeking on the Web; preliminary results of a study of how managers and IT specialists use the Web. Paper preseented at the 1998 ASIS Annual Meeting. Toronto: University of Toronto, Faculty of Information Studies. Retrieved 24 December, 2006 from http://choo.fis.utoronto.ca/fis/respub/asis98/
Choo, C. Detlor, B. & Turnbull D. (2000). Information seeking on the Web: an integrated model of browsing and searching. First Monday, 5(2). Retrieved 24 December, 2006 from http://firstmonday.org/issues/issue5_2/choo/index.html
Chu, C. (1999). Literary critics at work and their information needs: a research-phases model. Library & Information Science Research, 21(2), 247-273.
Crisp, C., Jarvenpaa, S. & Todd, P. (1997). Individual differences and Internet shopping attitudes and intentions. Austin, TX: University of Texas at Austin. Retrieved 15 April, 2003, from http://ccwf.cc.utexas.edu/~crisp/Indiv_Shop.htm [Now available at: http://InformationR.net/ir/12-2/Crisp.html
Ellis, D. & Haugan, M. (1997). Modelling the information seeking patterns of engineers and research scientists in an industrial environment. Journal of documentation, 53(4), 384-403.
Fescemeyer, K. (2000). Information-seeking behavior of undergraduate geography students. Research Strategies, 17(3/4), 307-317.
Fidel, R. & Green, M. (2003). The many faces of accessibility: engineers' perception of information sources. Information Processing & Management, 40(3), 563-581 Retrieved 24 December, 2006 from http://projects.ischool.washington.edu/fidelr/RayaPubs/ManyFacesofAccessibility.pdf
Griffiths, J.R. (2003). Evaluation of the JISC Information Environment: student perceptions of services. Information Research, 8(4), paper no. 160. Retrieved 24 December, 2006 from http://informationr.net/ir/8-4/paper160.html
Heinström, J. (2003). Five personality dimensions and their influence on information behaviour. Information Research, 9(1), paper no. 165. Retrieved 23 December, 2006, from http://informationr.net/ir/9-1/paper165.html
Hölscher, C. & Strube, G. (2000). Web search behavior of Internet experts and newbies. Computer Networks, 33(1-6), 337-346.
Katz, J. & Aspden, P. (1997). Motivations and barriers to Internet usage: results of a national public opinion survey. Internet Research: Electronic Networking Applications and Policy, 7(3), 170-188.
Kling, R. & Callahan, E. (2002). Electronic journals, the Internet and scholarly communication. Annual Review of Information Science and Technology, 37, 127-177/
Kuhltau, C. (1993). A principle of uncertainty for information seeking. Journal of Documentation, 49(4), 339-355.
Kuo, F-Y., Chu, T-H., Hsu. M-H. & Hsieh, H-S. (2003). An investigation of effort-accuracy trade-off and the impact of self-efficacy on Web searching behaviors. Decision Support Systems, 37(3), 331-342
Kwong, B. (2002). Toward a theoretical framework for understanding the relationship between situated action and planned action models of behavior in information retrieval contexts: contributions from phenomenology. Information Processing & Management, 38(5), 613-626.
Lundahl, U. & Skärvad, P-H. (1999). Utredningsmetodik för samhällsvetare och ekonomer. Lund: Studentlitteratur AB.
Pennanen, M & Vakkari, P (2003). Student's conceptual structure, search process and outcome while preparing a research proposal: a longitudinal case study. Journal of the American Society for Information Science and Technology, 54(8), 759-770.
Savolainen, R. (1999). The role of the Internet in information seeking. Putting the networked services in context. Information Processing & Management, 35(6), 765-782.
Schmidt, J. & Spreng, R. (1996). A proposed model of external consumer information search. Research notes. Journal of the Academy of Marketing Science, 42(3), 246-256.
Tenopir, C., King, D., Boyce, P., Grayson, M., Zhang, Y. & Ebuen, M. (2003). Patterns of journal use by scientists through three evolutionary phases. D-lib Magazine, 9(5). Retrieved 21.5.2003, from http://www.dlib.org/dlib/may03/king/05king.html
Thong, J., Hing, W. & Tam K-Y. (2002). Understanding user acceptance of digital libraries: what are the roles of interface characteristics, organizational context and individual differences? International Journal of Human-Computer Studies, 57(3), 215-242.
Wilson, T.D. (1999). Exploring models of information behavior: the 'uncertainty project'. Information Processing & Management, 35, 839-849.

Find other papers on this subject

Web Counter