Library Technical Services

Creating and Maintaining Euclid Subscription Metadata (LTS Procedure #105)

Scope: This procedure outlines the procedures for the creation and maintenance of Project Euclid Subscription Access metadata. NOTE: As of 07/18/08, MSU is in the process of transferring the responsibility for Prime subscription metadata to Duke University Press.  Once that transfer is complete and DUP has tested their environment, the transfer of responsibility for Select subscription metadata will take place.  As of 11/01/08, MSU will no longer be responsible for IMS subscription metadata, and this procedure should be archived.

Contact: Nancy Solla

Unit: Metadata Services

Date created: 07/12/07

Date of next review: July 2009

Introduction

List of Prime Journals

List of Select Journals

List of IMS Journals

Receiving Notification of an Update Request

Entering Metadata into XML Records

About Prime/Select Metadata

Note About Prime/Select IPs

Editing XML for Prime/Select

About IMS Metadata

Note About IMS IPs

Editing XML for IMS

Differences Between Prime/Select and IMS Metadata

Submitting Metadata

Notifying Concerned Party

Recording Hours Worked

Introduction

The single most important function of a subscription metadata record is to turn on access for current, paid-up subscribers, and to turn it off for those with lapsed subscriptions.

Maintaining Euclid subscription metadata means keeping, updating, and submitting freshly updated XML access records for:

1. Euclid Prime - a group of journals
2. Select - journals not included in Prime
3. International Mathematical Statistics (IMS) journals

Here is a complete list of all of the journals with the respective publication ID:

Select Journals (euclid.xxx.select)

IMS Journals

 Receiving Notification of an Update

Metadata Services is contacted by Project Euclid staff or a client --with an email posted to <metaserve-L@cornell.edu>--to do one of two kinds of metadata tasks:

1. Add metadata for new subscriber(s)

2. Revise metadata for current/renewing subscriber(s)

 

Prime/Select

Metadata comes from the following Project Euclid staff, in this order of frequency and preference:

Jinhee Roper, Account Representative

Terry Ehling, Director, Center for Innovation Publishing

Michael Wakoff, Production Administrator-E-Publishing Technologies

David Ruddy, Head of Systems, Development and Production

Metadata comes via email with or without PDFs, Word docs, or Excel spreadsheets. It is then copied into the appropriate directory \Euclid_DPUBS\Euclid_access_data\Prime+Select\Data_files_new on Library21.

 

IMS

Metadata comes from Elyse Gustafson, Executive Director, International Mathematical Statistics, or from one of several accounts representatives at FASEB, the organization which is now handling IMS journal subscription orders.

Metadata comes via email for small updates and from Excel spreadsheets for larger updates. It is then copied into the appropriate directory \Euclid_DPUBS\Euclid_access_data\IMS\Data_files_new on Library21.

  Entering Metadata into XML Records

With the update(s) in hand, the next step is to transcribe this “raw,” unencoded metadata into the appropriate XML record (one for Euclid Prime, one for each Select journal, and one for each IMS journal). This will almost always involve copying and pasting. For a subset of metadata contained in Excel spreadsheets there is a way of automatically transforming it into the desired XML-encoded metadata, but even here copying and pasting will be involved.

Prime/Select

Open all the records previously copied into \Euclid_DPUBS\Euclid_access_data\Prime+Select\Data_files_new on Library21.

Generally, if you receive an email with a PDF, Word doc or spreadsheet, most if not all of the metadata will be contained in the accompanying PDF, Word doc, or Excel spreadsheet. Regardless of the source, the untagged metadata it contains should be easy enough to identify and match up with the corresponding XML element.

First find out which Prime/Select journal(s) is being modified (information supplied in the email, the accompanying electronic record, or both). Next open an XML editor (Oxygen is currently our editor of choice) and access the directory with the path name \Euclid_DPUBS\Euclid_access_data\Prime+Select\XML_Files on Library21. Finally, open the corresponding XML record (recognizable by the file name, which simply consists of the publication ID together with the file name extension).

Euclid Prime = euclid.prime.xml

Select

1. Bulletin of Symbolic Logic & Journal of Symbolic Logic = euclid.group.asl.select.xml

2. International Statistical Review = euclid.isr.select.xml

3. Journal of Integral Equations and Applications = euclid.jiea.select

4. Journal of the Mathematical Society of Japan = euclid.jmsj.select.xml

5. Rocky Mountain Journal of Mathematics = euclid.rmjm.select

6. Tohoku Mathematical Journal = euclid.tmj.select.xml

  Euclid Prime/Select XML Records - An Overview

Note: For a full account of XML access records, refer to "Preparing Access Data for DPubS," written by the Director of ePublishing Technologies

Every Prime/Select XML record has the root element "dubs_access_data". Every Prime/Select XML record consists of a few basic parts (sub-elements nesting within the root element): "header", "PPV_cost_data", "open_access", "institutional_subscribers", and individual_subscribers". See this document for more details on the syntax for Prime/Select metadata.

Examples

1. "header" is the topmost part of the record and includes information about the particular Euclid journal and the date of the last record modification. Note that this element has both an opening tag ("header") and a closing tag ("/header"). Nesting inside this element are several other elements, also with opening and closing tags. This element set (the names in angular brackets) is always the same regardless of the journal. The content (the information between opening and closing tags) will, of course, vary (the only constant will be the information for "contact"). 

One important thing to note about the file_modification_date element is that the Euclid parser will not allow you to submit an .xml file with the same date.  Therefore, if you receive update requests multiple times throughout the same day, you must change the file_modification_date each time.  Since we want to be sure we know the history of the file, the best way to work with this issue is to add the time of day to the date element.  If in the example below, one were to have to resubmit the same file later in the same day, one would simply add the time of submission, i.e. 2007-06-04-08:15 for 8:15AM on 6/4/07.  "Military time" must be used, so 1pm would be rendered as 13:00, etc.

 

2. "PPV_cost_data" contains data indicating pay-per-view cost. This data stays fairly constant. Change it only when instructed to do so by someone on the Project Euclid staff.

 

3. "open_access" indicates what year range of the journal is accessible to non-subscribers. This data also stays fairly constant. Change it only when instructed to do so by someone on the Project Euclid staff. (This element is not present in all Prime/Select XML records.)

 

4. "institutional_subscribers" contains data for institutional subscribers. Note that the sub-element "subscription" remains fairly constant. Change it only when instructed to do so by someone on the Project Euclid staff.

 

5. "individual_subscribers" contains data for individual subscribers. Note that not every Prime/Select XML record contains this element. Change it only when instructed to do so by someone on the Project Euclid staff.

A Note on the <ip> Element for Prime/Select Records

This element is repeatable. There should be as many occurrences of it as there are IP addresses (including continuous ranges). Each IP address consists of four octets. In non-technical terms, an octet can be defined as any part of the IP address separated from an adjacent part by a period. Each octet has a number ranging from 0 to 255. (or 1-254, which for our purposes amounts to the same thing). Thus 130.179.9.9 is a valid IP address, 130.179.9 is not. 205.113.134.255 is a valid IP address, 205.113.134.256 is not.

Often an IP address will actually be a range of IP addresses. If any octet is represented as a range of numbers, then the IP address of which it is a part is an IP range. Thus 204.112.133.77-78 is a range. Sometimes an IP address will contain an octet with the broadest range possible. 136.145.56.0-255 is just such an IP address (note that this range can, albeit less elegantly, rendered this way: 136.145.56.0-136.145.56.255). The wildcard character * (asterisk) is often used in lieu of the 0-255/1/254 range. Thus the example just given can be rendered thus: 136.145.56.* The * wildcard can stand for either one octet or two. Thus the IP address 138.37.* signifies that that both the third and fourth octets cover the range 0-255/1-254. Note, however, that this IP address is usually represented as 138.37.*.* In fact, all such IP addresses with one wildcard must be changed when transcribed into the XML record. Using the same example, 138.37.* must be changed to 138.37.*.* Failure to do so will result in an invalid XML record.

The schema which validates an XML record can detect incorrect IP addresses of certain types. As note above, any IP address containing an octet out of the 0-255 (1-254) range will be detected. Any character other than a numerical digit or recognized wildcard will also be detected. Some incorrect IP addresses are only detected at the point of submitting them to Project Euclid.

There is at least one type of incorrect IP address, however, that will escape the notice of both the schema and Project Euclid. Any IP address containing octets with initial zeroes should be edited. E.g., 192.018.016.* should be changed to 192.18.16.* (note that 192.108.106.* would be all right, as would 192.180.160.*

Editing the XML File(s) for Prime/Select

You have accessed both the new unencoded metadata record(s) (Prime+Select\Data_files_new\...) and the corresponding XML record(s). (Prime+Select\XML_Files\...). It is now time to transcribe the relevant information (unencoded metadata) into the appropriate XML record(s).

First, scan the email, PDF, Word doc or Excel spreadsheet for the data that you will most likely need in the corresponding XML record. If it comes to you as an update to an existing record, chances are you will only have to edit the expiration date of "institution", possibly also some of the IP addresses and contact information. If it comes to you as metadata for a new subscription, you will need to create a new "institution" record. The quickest way to do this is to copy an existing "institution" record, delete all the institution-specific content and fill in the elements in accordance with the new metadata at hand. After any update to a Prime/Select XML record, no matter how minor, change the "file_modification_date" to the current date.

Finally, save your changes by clicking on “Save” in File menu. If the updated XML record is not well-formed (as judged against a universal XML standard) or not valid (as judged against a local schema located in the same directory as the XML records), the XML editor will indicate as much and show, with greater or lesser precision, what you need to do to fix it.

International Mathematical Statistics

Open all the records previously copied into \Euclid_DPUBS\Euclid_access_data\IMS\Data_files_new on Library21.

Generally, if you receive an email with a spreadsheet, all the metadata you need will be contained there. Otherwise, look for the metadata you need in the email. In either case, the untagged metadata should be transparent enough to identify and match up with the corresponding XML element.

(Note: As detailed below, metadata supplied in spreadsheets from IMS can usually be automatically encoded into XML, then copied and pasted.)

 First find out which IMS journal(s) is being modified (information supplied in the email, the accompanying spreadsheet, or both). Next open an XML editor (Oxygen is currently our editor of choice) and access the directory with the path name \Euclid_DPUBS\Euclid_access_data\IMS\XML_Files on Library21. Finally, open the corresponding XML record (recognizable by the file name, which simply consists of the publication ID together with the file name extension).

IMS Journals

1. Annals of Applied Probability (euclid.aoap)

2. Annals of Probability (euclid.aop)

3. Annals of Statistics (euclid.aos)

4. Bernoulli (euclid.bj.select)

5. Statistical Science (euclid.ss)

Euclid IMS XML records—an Overview

Note: For a full account of XML access records, refer to "Preparing Access Data for DPubS," written by the Director of ePublishing Technologies.

Every IMS XML record has the root element "dubs_access_data". Every IMS XML record consists of a few basic parts (sub-elements nesting within the root element): "header", "PPV_cost_data", "open_access", "institutional_subscribers", and "individual_subscribers". See this document for more details on the syntax for IMS metadata.

Examples

1. "header" is the topmost part of the record and includes information about the particular Euclid journal and the date of the last record modification. Note that this element has both an opening tag ("header") and a closing tag ("/header").* Nesting inside this element are several other elements, also with opening and closing tags. This element set (the names in angular brackets) is always the same regardless of the journal. The content (the information between opening and closing tags) will, of course, vary (the only constant will be the information for "contact"). 

One important thing to note about the file_modification_date element is that the Euclid parser will not allow you to submit an .xml file with the same date.  Therefore, if you receive update requests multiple times throughout the same day, you must change the file_modification_date each time.  Since we want to be sure we know the history of the file, the best way to work with this issue is to add the time of day to the date element.  If in the example below, one were to have to resubmit the same file later in the same day, one would simply add the time of submission, i.e. 2007-06-04-08:15 for 8:15AM on 6/4/07.  "Military time" must be used, so 1pm would be rendered as 13:00, etc.

 

2. "PPV_cost_data" contains data indicating pay-per-view cost. This data stays fairly constant. Change it only when instructed to do so by Elyse Gustafson at IMS.

 

3. "open_access" indicates what year range of the journal is accessible to non-subscribers. This data also stays fairly constant. Change it only when instructed to do so by Elyse Gustafson at IMS. (This element is not present in all IMS XML records.)

 

4. "institutional_subscribers" contains data for institutional subscribers. Note that the sub-element "subscription" remains fairly constant. Change it only when instructed to do so by Elyse Gustafson at IMS.

 

5. "individual_subscribers" contains data for individual subscribers. Note that every IMS XML record but Bernoulli (euclid.bj.select) contains this element. Furthermore, for each XML record that does contain it there are "individual" subscribers in the hundreds. However, this part is updated infrequently, and updates come by way of the Project Euclid staff as ready-made XML.

 

A Note on the <ip> Element for IMS Records

This element is repeatable. There should be as many occurrences of it as there are IP addresses (including continuous ranges). Each IP address consists of four octets. In non-technical terms, an octet can be defined as any part of the IP address separated from an adjacent part by a period. Each octet has a number ranging from 0 to 255. (or 1-254, which for our purposes amounts to the same thing). Thus 130.179.9.9 is a valid IP address, 130.179.9 is not. 205.113.134.255 is a valid IP address, 205.113.134.256 is not.

Often an IP address will actually be a range of IP addresses. If any octet is represented as a range of numbers, then the IP address of which it is a part is an IP range. Thus 204.112.133.77-78 is a range. Sometimes an IP address will contain an octet with the broadest range possible. 136.145.56.0-255 is just such an IP address (note that this range can, albeit less elegantly, rendered this way: 136.145.56.0-136.145.56.255). The wildcard character * (asterisk) is often used in lieu of the 0-255/1/254 range. Thus the example just given can be rendered thus: 136.145.56.* The * wildcard can stand for either one octet or two. Thus the IP address 138.37.* signifies that that both the third and fourth octets cover the range 0-255/1-254. Note, however, that this IP address is usually represented as 138.37.*.* In fact, all such IP addresses with one wildcard must be changed when transcribed into the XML record. Using the same example, 138.37.* must be changed to 138.37.*.* Failure to do so will result in an invalid XML record.

The schema which validates an XML record can detect incorrect IP addresses of certain types. As note above, any IP address containing an octet out of the 0-255 (1-254) range will be detected. Any character other than a numerical digit or recognized wildcard will also be detected. Thus '192.108.106.* will be recognized as invalid because of the character “’” immediately preceding the first octet (a recurring problem with IMS spreadsheet data). Some incorrect IP addresses are only detected at the point of submitting them to Project Euclid.

There is at least one type of incorrect IP address, however, that will escape the notice of both the schema and Project Euclid. Any IP address containing octets with initial zeroes should be edited. E.g., 192.018.016.* should be changed to 192.18.16.* (note that 192.108.106.* would be all right, as would 192.180.160

 

Editing the XML File(s) for IMS

You have accessed both the new unencoded metadata record(s) (IMS\Data_files_new\...) and the corresponding XML record(s) (IMS\XML_Files\...). It is now time to transcribe the relevant information (unencoded metadata) into the appropriate XML record(s).

If all you have is an email, scan it for the data that you will most likely need to end up in the corresponding XML record. If it comes to you as an update to an existing record, chances are you will only have to edit the expiration date of "institution", possibly also some of the IP addresses and contact information. If it comes to you as metadata for a new subscription, you will need to create a new "institution" record. The quickest way to do this is to copy an existing "institution" record, delete all the institution-specific content and fill in the elements in accordance with the new metadata at hand.

If you have an accompanying Excel spreadsheet, you can speed up the process by using an in-house web-based XML converter (called “Convert to XML 2.0”) First save the spreadsheet data as tab-delimited text. Next access the converter. Near the bottom of the screen you will find a series of radio buttons. The only one we use currently is the second one from the top—IMS_1. Select it. Now browse for the tab-delimited text file just created. Select it. Click “Process File”. The text is instantly converted into XML-encoded records and these are displayed on the same screen. Copy the records, then paste them into the appropriate XML record. Be mindful of the fact that the average spreadsheet will consist of metadata for several IMS journals (usually all of them), in this order: aoap, aop, aos, ss and bj.select. So don’t copy and paste everything you see, indiscriminately, in one fell swoop. First do the records for aoap, then those for aop, and so on. Also, some editing of the IP addresses or other information may be required.

After any update to an IMS XML record, no matter how minor, remember to change the "file_modification_date" to the current date. 

Finally, save your changes by clicking on “Save” in File menu. If the updated XML record is not well-formed (as judged against a universal XML standard) or not valid (as judged against a local schema located in the same directory as the XML records), the the XML editor will indicate as much and show, with greater or lesser precision, what you need to do to fix it.

  Some notable differences in convention between Prime/Select and IMS

IMS—Journal abbreviations in raw data vs. XML records

Note that the abbreviation for Bernoulli—euclid.bj.select—has a markedly different publication ID from the other IMS journals. This journal used to be a Select journal with a different publisher, hence the counterintuitive abbreviation.

Different conventions for Prime/Select and IMS subscription records

Filenames: Filenames for Prime/Select XML records are static (updates to records only indicated by changing "file_modification_date"). Filenames for IMS records are dynamic (updates to records indicated both by changing "file_modification_date" and by changing date component of filename)

Deletes: Deletes for Prime/Select XML records are not made as a rule. Instead, records for institutions with lapsed subscriptions are kept; access for these is controlled by value for “expiration” attribute in "institution" element.

  Submitting Metadata

1. Access home page of Project Euclid

2. Log in (Click “Log in”; at next screen, click “Log in” action button)

3. After login, click on “My profile” appearing in the upper right-hand corner – this takes you to profile for Metadata Services

4. At the top of your profile (this is really the profile for Metadata Services), click on link labeled “Editor Workspace”—you will now enter the DPubS Administration area

5. Under the label “Select a publication to work on”: select the Euclid abbreviation of the journal whose subscription data you are submitting. Click “select”

6. When the left-hand menu changes, select “Submit Subscription Data”

7. Click “Browse”, and select the XML-encoded subscription metadata file on your system. Click “Upload Subscription Data to DPubS”. Wait until a successful upload message is displayed

8. Repeat steps 5-7 for every other file you wish to load

9. When you are finished uploading file(s), click “Log out” in the left-hand navigation bar

  Notifying Concerned Party that Metadata Record has been Revised Per Request

Prime/Select

Upon successful submission of a Prime/Select journal, email party from whom you received the update request that the job is done. Usually this will be the DLIT Account Representative (currently Jinhee Roper) or Terry Ehling. Somewhat less often it will be David Ruddy. Select “Reply to All” to ensure that all concerned parties are informed. Make sure that the DLIT Account Representative is apprised of the update as he/she will send out “Welcome to Prime/Select” emails, as appropriate).

International Mathematical Statistics

Upon successful submission of an IMS journal, email Elyse Gustafson at IMS or the FASEB representative (whomever send the request, in other words) that job is done. Send as attachments all the updated IMS XML records (this extra step only necessary for IMS updates).

  Recording Time Worked on Update(s)

Go to appropriate subdirectory in \TimeSheets\EuclidSubscriptionTimeSheets on Library21.

Open current timesheet (timesheets are organized according to year and quarter, so that Q1 = Jan-Mar, Q2 = April-June, Q3 = July-Sept, Q4 = Oct-Dec).

Go to next available line and fill in all fields. Record time separately for Prime/Select and IMS journals. Using decimals, record time in increments of quarter hours and using decimals (0.25, 0.5, 0.75, 1 …).