Reprocessing metadata

Metadata, not only for digital libraries but also for regular library catalogs, is increasingly going to be collected from heterogeneous sources in various formats. With a plethora of metadata standards around, what could a framework “that likes any metadata it sees” (Roy Tennant) look like?

If you’d like to learn more about how metadata travels from harvesting from different sources to aggregated presentation, one good place to start would be “Strategies for reprocessing aggregated metadata” by Muriel Foulonneau and Timothy W. Cole (PDF, published in Research and Advanced Technology for Digital Libraries (proceedings of the 9th European Conference on Digital Libraries, ECDL 2005), Heidelberg: Springer, ISBN 3-540-28767-1).

Abstract:

The OAI protocol facilitates the aggregation of large numbers of heterogeneous metadata records. In order to make harvested records useable in the context of an OAI service provider, the records typically must be filtered, analyzed and transformed. The CIC metadata portal harvests 450,000 records from 18 repositories at 9 U.S. Midwestern universities. The process implemented for transforming metadata records for this project supports multiple workflows and end-user interfaces. The design of the metadata transformation process required trade-offs between aggregation homogeneity and utility for purpose and pragmatic constraints such as feasibility, human resources, and processing time.

In the process of looking at transitioning from MARC, I hope the Library of Congress will also consider an infrastructure that is designed to facilitate linking, ingestion and reprocessing of a variety of metadata from other information providers. Maybe it really is time for a meta model?

Simplification as an opportunity – redux

About a month ago I posted about George Oates’ concept of a “minimum viable record”. The topic has continued to float around in my mind and was also taken up by Christine Schwartz.  So I thought I’d highlight two additional related items from the library community that corroborate the “MVR” idea:

• Karen Calhoun, in her 2006 report for the Library of Congress entitled “The Changing Nature of the Catalog and its Integration with Other Discovery Tools”, suggests to “simplify catalog records to a set of basic elements to support discovery, browsing, identification, delivery, resource sharing, linking, and inventory control” and to “monitor RDA, keeping in mind the need for simplicity and the larger role that automated metadata creation, batch processing, linking, and Web services will play” (p. 17).
• William Moen, who was involved in the MARC Content Designation Utilization project (MCDU), asks the following pertinent questions in one of his presentations on the project: What is needed in a bibliographic record? Can you argue persuasively for the cost/benefit of your existing practice? Should the focus be on high-value, high-impact, high-quality data in a few fields/subfields? Can you identify these few fields/subfields?

These questions and considerations should inform how we envision our (future) cataloging practice.

Automatic subject cataloging at the German National Library

The German National Library has been working on a project for automatic subject classification that is expected to go live at the end of this year. This project, called PETRUS, is explained in detail in this article (in German). Part of the Library’s mission is the creation of the German National Bibliography, and to keep pace with the onslaught of (digital) resources more efficiency is needed. The main areas where human intellectual work is supported by machines in the PETRUS project are indexing, classification, metadata extraction and metadata generation.

At the moment, the project covers full-text electronic / online publications in German and English. Subject headings from the SWD (Schlagwortnormdatei) and DDC numbers are automatically assigned to the resources. I assume when the project becomes operational more will be reported on precision rates. To be sure, the more full text is available, the more material there is for the computer to process and learn from and the more successful automatic classification will be.

Machine-learning and linguistic as well as semantic analysis are at the heart of the PETRUS project. The Library collaborates with Averbis, a company that has already built several information retrieval platforms on the basis of Lucene, e.g. Medpilot for the German National Library of Medicine. Their technology makes use of various NLP methods for assigning classification numbers and headings from subject authority files.

The German National Library will pursue a two-pronged approach to cataloging digital materials: publishers and other suppliers of publications are encouraged to already include metadata when submitting resources to the Library as legal deposit. This metadata can be reused, repurposed or iteratively enhanced. Second, the Library will make use of automatic tools to extract and/or generate metadata from machine-processable data. Human intervention will only take place on specific occasions, for quality control or clarification of difficult cases.