This article is from the March 2004 issue of Update.
City University's Ciber research group has recently completed an evaluation of websites, sponsored by the Diana, Princess of Wales Memorial Fund as part of its effort to improve access to information for isadvantaged young people.1 The overall aim of the investigation was to review the extent to which the aims of the funded organisations have been realised through the development of their websites, and to develop a 'template' to provide for best practice within the fund and sector. More specifically, we wanted to look at:
- the extent and pattern of website use;
- whether the site met the needs of its users;
- if there were any barriers to use, in terms of information retrieval
or understanding;
- the extent to which users (especially young people) were involved in the design and evaluation of the site;
- how the exploration of the above issues addressed and informed good practice.
A multi-method approach was used, incorporating log analysis of the use of the various sites, online questionnaires, usability sessions, and in-depth interviews with charity staff and site developers.
Despite the fact that, as one commentator put it, 'the internet has the potential to revolutionise almost every aspect of the work of voluntary organisations',2 the investigation found that many of the bodies involved did not fully understand the significance or implications of having moved their services into a digital environment. They were not fully aware that their audience has not only grown enormously and widened, but has changed. This raises fundamental questions about their remit, organisational objectives, etc and the services they should be providing for the web. A lack of monitoring and evaluation mechanisms (and culture), together with a belief on the part of managers that things have not really changed, is largely to blame.
Albert Kennedy Trust (supporting gay/lesbian
young people)
http://www.akt.org.uk/
Just one main menu, very prominent.
In order to help organisations improve their digital thinking and performance we offered them some robust and practical advice, derived partly from the study itself, and partly from previous evaluative work undertaken by the team. A shortened version is reproduced below.
Digital users (consumers)
Remember your users will be global
Whatever you understand as your geographical remit,
in the case of the web you are on an international platform and serving a need that is international too. As a voluntary body/charity involved with vulnerable people with urgent needs you cannot simply shrug these people off. By going on the web you really have invited this use.
Our experience shows that in many digital environments
international users will dwarf UK users in number. It may be that you don't have the staffing resources to deal with everyone, in which case you really are duty bound to provide links on your site to the equivalent organisations in your visitors' home countries and to provide as much material as you can in a digital form. On a practical level you need to:
- determine the extent and nature of international usage (and formulate plans accordingly); define terms or explain the location of places; make it clear on your site that the service is primarily for UK residents but that you recognise the need is global and you have international partners or can steer enquirers to similar information sources closer to, and perhaps even more geared towards, their needs.
If you really are genuinely overwhelmed (a sign of your success), it is not difficult to 'bounce' messages from email addresses which include a national domain indicator (e.g.'nz', 'au' etc) or other indication of location (such as 'edu' for US academic sites). The message should be accompanied by a pre-prepared and automatically added list of other information providers as mentioned above and an explanation of why you cannot deal with the query. This does not work, of course, for hotmail or other non-location specific addresses.
Remember your users will be larger in number, different in character and behave differently
If you look at your computer-generated transaction logs - about which more, below - you will probably find that many more people reach you via the internet than by phone, letter or physical visit. These people are likely to be very different in character and behaviour from those you are used to. The audience, by definition, is far more heterogeneous and - in information terms - 'promiscuous'. More than likely they will be 'bouncers' and 'checkers', sent to your site by a search engine like Google. They will use the search engine to make a choice, will check your site quickly and bounce out again to do the same to some other organisation's website. And, with such an abundance of choice, they are unlikely to come back unless you offer something special. Digital users are more likely to be students, academics, professionals and carers than those with a specific condition, patients etc. This digital audience is fickle and dynamic, and it is helpful to think of them as consumers rather than users
Remember you have entered a dynamic and interactive
environment
Most organisations we have researched use the web simply to increase their presence or to reach a larger audience, more cheaply. It is the broadcasting potential that fascinates, and that constrains their vision. Few people grasp the fact that it is a fast-changing environment that has to be monitored constantly and the feedback used to refine and improve the digital platform. Interactive does not simply mean replying to users by email or running a chat room, it also means utilising user feedback to tailor the service accordingly, something almost impossible to do in a hard-copy environment.
Child Bereavement Project (c/o National Children's
Bureau)
http://www.ncb.org.uk/cbn/directory/feedback.htm
www.childhoodbereavementnetwork.org.uk
[visited November 2006]
Good example of feedback form for site users, including a free-text facility so people are not constrained by questionnaire wording.
Evaluation, evaluation, evaluation
Always monitor
Your users will be a remote, new, volatile and relatively anonymous group, and the sheer weight of numbers means that you will have to gather information on them if you want to maintain a grip on events. If you don't, a gulf between provision and need is likely to be opening up, which could lead to serious problems for the organisation.
Data can be gathered from a number of quarters — from computer logs, client databases, chat rooms and questionnaires. Logs — the digital fingerprints of your users — when configured correctly, should tell you where your users are coming from, how often they visit, what pages they are accessing and how deeply they engage with your service. Logs are usually supplied by third-party organisations, maybe an internet service provider.
Logs have a major advantage in the caring sector. Here, users are traditionally reluctant to offer their views and experiences — probably because they are feeling vulnerable and like the anonymity of the internet. Logs tell (anonymously) the stories that users themselves are reluctant to discuss. So get a dialogue going with your log provider to make sure you maximise the yield from the logs (or if you need a deeper, bespoke analysis contact Ciber3 for more information).
Of course, logs can only help you evaluate what is provided and you still need to ask people whether the information is useful/at the right level etc, and what else they would like from your organisation. This is where other methods come in — online questionnaires, interviews and usability tests.
Surveys allow a broad sweep of users, and are good for number crunching and for interpreting log results. You can find out the main reasons for going to the site, the most popular pages and why, and any aspects of the site that users do not like or find difficult to use (frames, flashy graphics etc). Interviews enable you to explore individuals' viewpoints more thoroughly. Often it is only by talking to people that important aspects about their perception of your site (or organisation) may be teased out. Finally, having small groups look at the site and undertake set tasks, for usability purposes, also provides rich and unexpected data useful for improving layout and design.
Content
Try not to be too prescriptive
You may want to protect your readers from unpalatable facts (e.g. the mortality rate of a particular condition) but this is extremely difficult to do in the digital environment. The information you do not publish will be available to them on other internet sites, and they will make unfavourable comparisons with yours. Rather than censor, we suggest that you accentuate
the positive.
Don't shovel content online
Too often people use the web as a document archive or graveyard without understanding the impact on the user. Too often it gets in the way of information. Use log data to spot which areas need more coverage and which need less. Change the content as needed, just as shops do — you have become a digital information shop. It is also expensive keeping unused data online.
Put the most important
information up front...
Log studies have shown that the further down the digital hierarchy
content is situated, the fewer 'hits' it receives —
fewer people get to the information. We call this aspect 'digital visibility'. The obvious message is try not to have too many levels on your site, and put the most important information where people are most likely to see it.
Most of all, monitor page use and do something about it when your flagship pages are not being accessed. Do not let your digital service become set in concrete: move content around in response to your users' online behaviour and to their best advantage.
… and update it!
Do not put information on the site if it will 'decay' and you don't have the resources to update it. There is nothing worse than an electronic resource that hosts superseded information, and the bouncing/checking user will inevitably find you out through site comparisons. Your authority will be undermined. Currency is as much king as content, so update and date stamp all pages - and don't forget to check those web links and contact details of other organisations.
Finally, be contactable
Do include a full postal address and tele-phone number on your website. It is amazing how many organisations assume people only want to use email. Often, of course, websites are consulted by people wishing to visit, to find out where the information comes from, geographically, or to have hard-copy material sent to them. A physical address tells people a lot, and can be reassuring to those who are all too aware of transient, fake or otherwise less than adequate sites. Also, with the rise of the internet café/phone booth etc, it is possible that some web users may not have email accounts to receive replies to any message they might send through a 'mailto' link.
Who Cares? Trust (project to improve training
and employment opportunities for young people with learning disabilities)
http://www.thewhocarestrust.org/
One main menu list (across the top)
'Click here' entry points well explained, with full text as part of the link, plus good orientation.
Usability/accessibility
It is very important, particularly for charities and similar
organisations working for or with vulnerable people, to check for usability. Apart from the usability sessions mentioned above, we recommend the following:
Bobby guidelines
Pages can be 'Bobby' checked for free on the net. Bobby is a web accessibility tool designed to help expose barriers to accessibility and encourage compliance with existing accessibility guidelines. It offers prioritised suggestions based on the World Wide Web Consortium's (W3C) Web Access Initiative. Bobby allows developers to test web pages and generate summary reports highlighting critical accessibility issues before posting content to live servers.
To run a Bobby scan:
The BBC online news site is often mentioned in reviews as an example of how a well-designed, easily usable site should look. However, running BBC News online through the Bobby check gave the following advice (and more!), useful for anyone developing a site:
- provide alternative text for all images;
- check that the foreground and background colours contrast sufficiently with each other;
- separate adjacent links with more than white space.
Each point is accompanied by an explanation and a rationale. For example, with regard to the last point, Bobby says: 'One common problem for many blind web users is identifying which pieces of text are links and which are not. If there is no text or image separating two adjacent links, even if they occur on consecutive lines, some screen readers will incorrectly read adjacent links as a single link.'
Avoid these …
Apart from the usability guidelines noted above, our own website evaluation work has indicated a number of things to try and avoid in website design.
- Multiple menu lists. These confuse people, especially when the same entry appears in two or more places. In a book, you would never see a list of entries both running across the contents page and down the side of it — so why do this electronically?
- Using the 'open new browser window' facility. Some
users are not aware the new window sits on top of the original one and are confused when the 'back' button is greyed out (because the new window is showing its first page).
- Using frames (which are out of favour these days). Putting pages in frames means that the URL of your site stays at the 'Home' page location, and does not resolve down to the actual page people might reach through your menu. This is to be avoided because people might want to bookmark specific pages. Also, frames confuse printers, which often print the wrong page or — infuriatingly — say 'nothing to print'.
- Including unexplained/poorly explained 'click here'
links. It is always better to give people plenty of detail about what page or information to expect. It is important also to include explanatory text as part of the link — W3C guidelines say that 'link text should be meaningful enough to make sense when read out of context'. This is because of the way some screen readers work.
In conclusion, it is only fair to say that, in general, the sites we evaluated were well designed, and their use was associated with positive outcomes. What was very clear, however, is that none of them were really benefiting from robust user input/evaluation, either in the form of questionnaires or computer logs — the CCTV of the internet. Few of the organisations appeared to be obtaining valuable intelligence as a result of their foray into a digital and very challenging environment.
The web, with its enormous reach, multimedia character,
youth popularity, anonymity and relative low cost, offers massive opportunities to charities. Generally, however, it appears that these are not being taken. The art of good website practice on the part of the provider is to seek user input at every stage. This includes information needs research, usability feedback, or the study of those digital footprints left by computer logs.
Only by proactively engaging with their users and their behaviours will producers be able to say with any confidence that their service meets the real needs of its target group of information-seekers.
Young Minds (a children's mental health charity)
http://www.youngminds.org.uk/contactus/
Comprehensive 'Contact us' list, which includes phone, fax, opening times, and details of different offices (below this screen-capture there are details of administration and management, fundraising, and more).
References
1 The full report will be posted on the Ciber website (www.soi.city.ac.uk/organisation/is/research/ciber/).
2 J. Saxton and S. Game. Virtual Promise: are charities making the most of the internet revolution? Third Sector, 2000 (p. 6).
3 nicky@soi.city.ac.uk
Peter Williams is a Research Fellow in the Department of Information Science, City University and founder member of Ciber. David Nicholas and Karen Dennis both work for Ciber.