Database-generated Web pages: the Norris Medical Library experience

INTRODUCTION

The Norris Medical Library at the University of Southern California serves the faculty, staff, and students of the schools of medicine, pharmacy, and allied health. The library launched its original Website in 1994, focusing the site upon library information such as its hours and policies. A small part of the site was devoted to librarian-recommended Web resources, arranged by discipline. As the Web grew in popularity and available full-text electronic resources proliferated, these lists of selected sites were growing unwieldy. All the pages on the Norris Medical Library Website (NMLweb)† were hand-coded hypertext markup language (HTML), including the lists of electronic resources. Changes to one resource could have repercussions on other lists, and one alteration could result in many pages requiring attention.

When the library decided to redesign its Website in 1998, it planned to shift the emphasis from library information to electronic resources. The new site would not only promote the electronic resources but would increase the number of them, particularly the electronic journals. The amount of time needed to maintain these lists was a concern to the library Web committee. In partnership with University of Southern California's Internet Publishing Center (IPC), the library committed itself to a database solution [1–3].

DATABASE-DRIVEN SOLUTIONS

A database for electronic resources functions like an online catalog, in which a record is created for each resource, and then a search identifies appropriate records based on criteria such as subject, keyword, or format. Although databases for Web pages may range in size and mechanics, virtually all share a basic structure. The information needs to be entered into a database software program, such as Access or FileMaker Pro. Then, a script must be employed to extract the data from the database and insert it into Web page templates. In the past, common gateway interface (CGI) scripts residing on the Web server were the dominant tools. “Middleware” programs, such as Cold Fusion and Lasso, can be used to create the necessary CGI scripting to connect the Web page to the database. Finally, these finished pages must be made available on a server for Web accessibility.

Making changes to hand-coded Web pages is time consuming. If the uniform resource locator (URL) of a resource changes, the librarian responsible for maintaining the Website must make corrections to each page on which the resource is located. With a database approach, altering that resource's record, such as the URL, can quickly modify a resource's properties on numerous pages. Web pages generated from a database are globally updated when the database is updated. Staff involved with the database need to exercise caution when creating records, as the flip side to global change is global error. If any piece of information, such as an URL, is wrong in the record, it will be incorrect wherever it appears. Such problems often occur in pages coded by hand and are not unique to databases.

Another advantage to databases is the option for online forms. Librarians may submit records through a form on the Web, adding to the database without having to know HTML. The database creates the resulting Web page with a script, eliminating much of the need for hand coding. Those responsible for adding items to the database will need to learn how to use the form, but the learning curve is much shorter than it is with HTML. At Norris, the librarians have found completing a Web form easier than modifying HTML code, which has led to an increased willingness to update their pages and an interest in the database development.

Database-generated pages can be static or dynamic. Static pages are generated offline and uploaded to the Web server as finished HTML pages. To the Web user, the database component is invisible, as completed pages look identical to pages created by hand coding.

With dynamically generated pages, users initiate the pages' creation. Users make a request to the server for items of interest, either through a Web-based form or by clicking on a provided Web link. The server then runs the script, which accesses the database, extracts the matching records, and creates corresponding Web pages reflecting the users' criteria. The corresponding pages are also known as “on-the-fly” pages; they do not exist until users request certain information. This technique allows for a very current page; new modifications and resources can be accessed immediately after their addition to the database.

Such interaction has its downsides. Response time can be very slow causing users to become frustrated waiting for the script to build on-the-fly pages. CGI scripts can also create security holes in servers, making institutions very leery about adding them. To date, the University of Southern California has not allowed any department to place its CGI script on the university Web server. The library could place scripts on its own small Web server, but the machine is not robust enough to handle our Website traffic.

Because of these limitations, the Norris Medical Library chose to use a database to create static pages. The FileMaker Pro database runs on a Macintosh computer. The librarians complete an online form for each electronic resource, creating database records. The library's Web manager accesses the database and issues the command to generate Web pages based on established criteria. The database creates the pages, and then an Applescript uploads the pages onto the Web server. This static Web page is available online and does not change until the pages are regenerated.

ROLE OF LIBRARIANS

The librarians at the Norris Medical Library worked extensively with the IPC's database manager in the development of the database. The library's Web committee met numerous times to determine the necessary information to be included and the best presentation of the information. The database manager created the database structure and wrote the necessary scripts. The librarians decided that the database record for each resource needed an URL, title, brief description, notification of access restrictions, and meta tags for keywords. Descriptions of resources were vital but consumed space on the Web page. The lists of resources would be significantly lengthier if descriptions were visible. Instead, the library chose to create a “More Info” link next to each resource title, giving users the option to read further about the resource.‡ The “More Info” descriptions provided use restrictions, dates of coverage, and a librarian contact.

Classification of resources became a challenge. The original, hand-coded electronic resources pages had served as a template, but inconsistencies among the pages were rampant. On the earlier pages, the same resource was labeled on differing pages as a government resource, a federal resource, and a conference resource. Some resources were identified by content, others by format. The committee struggled to come to a consensus regarding format definitions: both Yahoo and AMA Online Doc Finder could be called directories, but should they be classified as the same format? What should be considered a database? If a journal has a search feature, is it a database? How should a resource calling itself a “textbook” but lacking a print version be defined, as a book or Website? The database and online form would impose a certain level of consistency, but the librarians needed to create the policies. The classification by subject and format determined which resources appeared on the various pages and the arrangement of the resources on each page. Those keywords were included for each record. Some new categories added to the library Website, such as personal digital assistant resources, continue to raise new questions, as the original decisions on format and subject keywords do not provide the best descriptors.

CONCLUSION

Launched in March 1999, the library's database approach is considered a success. Although not suitable for Web pages that are text heavy and address items that do not contain common elements, such as policies and library hours, a database is an excellent method of handling resource lists. The librarians responsible for electronic resources have responded to the simplified method of adding records by increasing their contributions substantially. Before the database, the library site had approximately 300 suggested electronic resources. In November 2000, this number had increased to nearly 1,500 resources. As the e-resources lists has grown, so has their usage. Statistics show that the old e-resources lists had about 2,000 to 3,000 hits per month; the latest numbers reveal more than 125,000 hits per month. The increase in statistics most likely reflects the greater wealth of information now available. However, the ease of maintaining the resources through a database approach has meant that more resources are provided and in a more timely manner.

The library initially questioned the decision to create static rather than dynamic pages with the database. However, users have responded positively to the quick download speed of the static pages. Search engines index static pages, while most do not index records stored only in a database. Usability studies conducted on the library site§ have shown that users often prefer to use “search-the-site” systems to trying to click their way to a page; dynamic pages are not indexed and cannot be searched through such a mechanism.

The Norris Medical Library database was developed using resources readily available at the time. Although the university licenses Sybase, the availability of programmers is limited. As a result, the library used FileMaker Pro, which the university also licenses and the IPC staff supports. The original development was done on a hand-me-down Macintosh but now resides on a Power Mac G4 with 450 MHz. Lasso is used for scripting the pages created from the database, and Applescript and file transfer protocol (FTP) place the pages upon the university server, eliminating the need to install scripts or software on the Web server itself.

Although during implementation of the database system many choices were forced upon the developers, the resulting product has proved useful to both library users and staff. This success has encouraged the library and other university departments to launch more Web projects using databases, such as a faculty profile database and a searchable clinical trials site. The library is currently considering more ambitious database uses, such as personalized “MyLibrary”–type portals. These projects demand a more stable set-up than currently provided by our FileMaker Pro–Lasso combination. The library is investigating other options, including ColdFusion and Hyperwave solutions. As with most Web projects, this one will continue to evolve with time as software, needs, and the institutional environment change.