This article is from the June 2004 issue of Update.
The news that the British Library is attempting to demonstrate its value to the UK economy1 reminds us that making an effective case for support for libraries and information services has long been a concern of the LIS profession.
Making the case
The traditional approach to justifying an increase in the budget — or defending it — has been to highlight the factors that influence the demand for library and information services, such as growth in the size or sophistication of the client population, increased educational provision, or scientific and technological change. These are generally easily quantifiable, and have been the subject of regular research intended to support a case for funding. The results have been presented to the decision-makers so frequently that they are now widely accepted by them, but this has not necessarily prevented budget cuts.
Creating an understanding of the impact of libraries in terms of the social, educational or economic aspirations of the politicians may have even more influence on budgetary allocations to libraries and information services. But, demonstrating how libraries make a difference is a more complex issue than undertaking simple studies of demand factors.
Recent British governments have, however, increasingly made it clear that cases for extra spending will only be considered when an organisation delivers value for money, supported by evidence that it makes an impact in the government’s priority policy areas. The Museums, Libraries and Archives Council (MLA, formerly known as Resource) has, therefore, been particularly interested in encouraging the sector to develop and use measures which demonstrate the impact of those services in a way that means something to politicians.
As a first step towards fostering an evidence-based approach to policymaking and professional practice in library management, Resource commissioned an evaluation of the Value and Impact projects funded by its predecessor, the Library and Information Commission. This found that those projects, while generating a significant amount of useful information, did not provide compelling evidence of impact.2
Subsequently, to relate to the government’s education agenda, two projects were carried out examining the impact of school libraries.3,4 These found that there was a body of research supporting the view that school libraries can have a positive impact on academic achievement, particularly at the primary and early secondary level. However, there was no clear evidence that might have identified best practice by differentiating the contribution made to learning by the various models of school library provision to be found throughout the UK.
A fourth project funded by Resource examined the sector as a whole. This was a desk-based critical review of the literature of impact evaluation in the museum, gallery, archive and library domains, concentrating mainly on evidence from the UK over a period of five years. Some significant work outside that period or outside the UK was also noted.
It pulled together the published evidence from research undertaken by both academics and practitioners, and assessed the methodologies used so that future impact evaluations might be based on appropriate, tried and tested methods.5
Impact on personal development
This review found evidence from projects examining the social impact of libraries that indicated that they do make a positive impact in supporting personal development, social cohesion, community empowerment, local culture and identity, and health and well-being. The most compelling evidence was in the area of personal development, possibly because the immediate outcomes are more easily identified and less problematic in terms of establishing causality. However, the evidence presented varied in the extent and rigour with which it was reported. The literature included ‘evidence’ in the form of project cameos, short anecdotal quotes, and brief accounts of engagement with specific groups in the community. This does more to demonstrate potential for a wider range of social impacts than provide consistently convincing results.
Impact on learning
A number of studies were found that examined the relationship between libraries and learning. These indicated that libraries had made a positive impact on reading development in young children; on academic achievement, particularly in terms of acquisition of language and ICT skills and information literacy; on the acquisition of broader aspects of learning, such as increased motivation for learning, self-confidence and independence; and on enjoyment and choice of leisure reading material.
All the studies examined reflected the complexity of the learning process, the difficulties of isolating the impact of the library from other significant influences, and the challenges in establishing causal relationships. Again, however, the evidence presented varied in rigour, and most evaluations concentrated on the short-term impact of special projects, rather than the continuing impact of core services.
The majority of studies of the sector’s economic impact came from outside the UK, and/or concentrated on the arts and cultural industries in general. They also concentrated on the public sector and major public spending initiatives. Research into the economic impact of public libraries conducted in the UK claimed to present evidence of direct and indirect impact on job creation, and visitor spending in town centres. Evidence that has appeared in the public domain on the impact of public or private sector information services on the business community has been limited, and has not been related to areas of public or government interest such as the effect of library services on inward investment in a particular area, or the relationship between public libraries and town centre regeneration. Research in this area has been best described as being at the stage of investigating possible approaches to measuring impact, and demonstrating potential rather than actual impact.6 The British Library’s study, for example, employs a methodology previously little used in the library domain, contingent valuation, which is apparently widely accepted as a means of evaluating the benefit to a nation of its publicly funded institutions, but which appears to be little more than a sophisticated form of estimating. Can anything more precise be devised?
Quality of practitioners’ evaluations
At present, practitioners’ project evaluations are the most readily available source of evidence of libraries’ impact. They provide a useful means of raising awareness of what can be achieved and help practitioners understand how it can be achieved. However, the quality of these evaluations varies tremendously. The reports tend to be descriptive, with results that are difficult to replicate. The failure of many practitioners (and most independent consultants) to publish the results of their research not only suggests a reluctance to get into a debate about the robustness of their methodology and data, but also raises doubts about the generalisability of any conclusions that might be drawn from their work.
The MLA’s interest in evaluating impact compels practitioners as well as academics not only to seek greater rigour in research, but also to re-assess the relationship between research, policymaking and the development of the profession. It underlines the fact that research aimed at evaluating the role of library and information services could have a significant influence on the perception by citizens and policy makers of the role that information does or could play in promoting social and economic well-being. However, it is also clear that if the results of research are to be influential, more attention needs to be given to helping policy makers to find and absorb the knowledge they need for decision-making.
Traditional approaches to publishing research results are probably inadequate for this purpose. As a recent report for the National Audit Office on how British government departments commission, manage and use research observed, ‘dissemination of research findings is not sufficient to ensure that research findings are used to improve service delivery and to inform policy’.7
The British Library has not yet published the full report of the consultants’ research, but promises that papers based on the research will be published in scholarly journals in due course. Whether anyone will pay any attention to them remains to be seen. Scholarly journals are not widely available, and even less widely read.
A recent study also noted a reluctance among the LIS community to acknowledge the results of relevant research, particularly work undertaken by those researchers who were not themselves members of the LIS community.8 This is especially significant as many of the studies of libraries’ impact, particularly in the economic field, have been conducted by specialists from other disciplines rather than by information professionals. In some other professions, it is accepted that a multi-disciplinary approach can deepen understanding of the way complex issues can be tackled,9 and there is clearly a need to examine what underlies LIS professionals’ attitude, particularly as so many of them already have a degree in another discipline.
A change of approach?
Reflection on the outcome of the impact studies undertaken for Resource strongly suggests the need for a greater emphasis on impact research if the profession is to make and be seen to make a significant contribution to social and economic development. For the LIS research community, both academics and practitioners, an immediate challenge must be to begin to determine meaningful and measurable indicators of the impact of libraries and information services in different fields. It is also important that the LIS community as a whole takes research more seriously. If LIS practitioners themselves do not value and make good use of research results, how can we expect policy makers outside the profession to pay any attention to them?
The other challenge is to use the outcomes of research effectively to influence policy making, and for that the LIS community will need an understanding of not only the political environment, but also the policy-making process, and the information-processing habits of the policy makers. The LIS community must also review how it can make use of more and different pathways for disseminating the results of research, such as policy networks, so that any objective evidence that it might produce can play a more effective role in strengthening library and information services.
References
1 British Library. ‘Measuring our value.’ (www.bl.uk/cgi-bin/news.cgi?story=1398 [broken link removed on 14 April 2005]).
2 D. Streatfield and S. Markless. Critical Evaluation of the LIC Value and Impact Programme. Information Management Associates, 2002 (www.resource.gov.uk/documents/ev_impevb.pdf).
3 D. A. Williams, C. Wavell and L. Coles. Impact of Secondary School Libraries on Attainment and Learning: critical literature review. Final report for Department for Education & Skills and Resource: the Council for Museums, Archives and Libraries. The Robert Gordon University, 2001 (www.rgu.ac.uk/files/Impact%20of%20School%20Library%20Services1%2Epdf).
4 D. A. Williams, L. Coles and C. Wavell. Impact of School Library Services on Attainment and Learning in Primary Schools: final report for Department for Education & Skills and Resource: the Council for Museums, Archives and Libraries. The Robert Gordon University, 2002 (www.rgu.ac.uk/files/ACF1C8D%2Epdf).
5 C. Wavell et al. Impact Evaluation of Museums, Archives and Libraries: available evidence project. The Robert Gordon University, 2002 (www.resource.gov.uk/information/evidence/ev_impev.asp).
6 W. Warr. ‘A framework to measure impact.’ Information World Review, July/August 2003, pp. 8.
7 National Audit Office. Getting the Evidence: using research in policy making. Report by the Comptroller and Auditor General. The Stationery Office, 2003 (www.nao.gov.uk/pn/02-03/0203586.htm).
8 S. McNichol and C. Nankivell. The LIS Research Landscape: a review and prognosis. University of Central England, Centre for Information Research and Teaching, 2003 (www.cie.uce.ac.uk/cirt/projects/past/LISlandscape_final report.pdf ).
9 Y. Dror. ‘Science advice: tasks, preferable features, impact assessment.’ The IPTS Report, 72, March 2003, pp. 13-19.
Ian Johnson is Assistant Dean, Aberdeen Business School, The Robert Gordon University (i.m.johnson@rgu.ac.uk).