{"id":388,"date":"2013-04-12T20:15:57","date_gmt":"2013-04-12T15:15:57","guid":{"rendered":"http:\/\/blogs.nd.edu\/emorgan\/?p=388"},"modified":"2014-04-15T20:46:48","modified_gmt":"2014-04-15T15:46:48","slug":"workflow","status":"publish","type":"post","link":"https:\/\/sites.nd.edu\/emorgan\/2013\/04\/workflow\/","title":{"rendered":"Catholic pamphlets workflow"},"content":{"rendered":"

\n

\"Gratuitous<\/a>

Gratuitous eye candy by Matisse<\/p><\/div>This is an outline of how we here at Notre Dame have been making digitized versions of our Catholic pamphlets available on the Web — a workflow:\n<\/p>\n

    \n
  1. Save PDF files to a common file system – This can be as simple as a shared hard disk or removable media.<\/li>\n
  2. Ingest PDF files into Fedora to generate URLs – The PDF files are saved in Fedora for the long haul.<\/li>\n
  3. Create persistent URLs and return a list of system numbers and… URLs – Each PDF file is given a PURL for the long haul. Output a delimited file containing system numbers in one column and PURLs in another. Create persistent URLs and return a list of system numbers and… URLs – Each PDF file is given a PURL for the long haul. Output a delimited file containing system numbers in one column and PURLs in another. (Steps #2 and #3 are implemented with a number of Ruby scripts: batch_ingester.rb<\/a>, book.rb<\/a>, mint_purl.rb<\/a>, purl_config.rb<\/a>, purl.rb<\/a>, repo_object.rb<\/a>.)<\/li>\n
  4. Update Filemaker database with URLs for quality assurance purposes – Use the PURLs from the previous step and update the local database so we can check the digitization process.<\/li>\n
  5. Start quality assurance process and cook until done – Look at each PDF file making sure it has been digitized correctly and thoroughly. Return poorly digitized items back to the digitization process.<\/li>\n
  6. Use system numbers to extract MARC records from Aleph – The file names of each original PDF document should be an Aleph system number. Use the list of numbers to get the associated bibliographic data from the integrated library system.<\/li>\n
  7. Edit MARC records to include copyright information and URLs to PDF file – Update the bibliographic records using scripts called list-copyright.pl<\/a> and update-marc.pl<\/a>. The first script outputs a list of copyright information that is used as input for the second script which includes the copyright information as well as simply pointers to the PDF documents.<\/li>\n
  8. Duplicate MARC records and edit them to create electronic resource records – Much of this work is done using MARCEdit<\/li>\n
  9. Put newly edited records into Aleph test – Ingest the newly created records into a staging area.<\/li>\n
  10. Check records for correctness – Given enough eyes, all bugs are shallow.<\/li>\n
  11. Put newly edited records into Aleph production – Make the newly created records available to the public.<\/li>\n
  12. Extract newly created MARC records with new system numbers – These numbers are needed for the concordance program — a way to link back from the concordance to the full bibliographic record.<\/li>\n
  13. Update concordance database and texts – Use something like pdftotext to extract the OCR from the scanned PDF documents. Save the text files in a place where the concordance program can find them. Update the concordance’s database linking keys to bibliographic information as well as locations of the text files. All of this is done with a script called extract.pl<\/a>.<\/li>\n
  14. Create Aleph Sequential File to add concordance links – This script (marc2aleph.pl<\/a>) will output something that can be used to update the bibliographic records with concordance URLs — an Aleph Sequential File.<\/li>\n
  15. Run Sequential File to update MARC records with concordance link – This updates the bibliographic information accordingly.<\/li>\n<\/ol>\n

    \nDone, but I’m sure your milage will vary.<\/p>\n","protected":false},"excerpt":{"rendered":"

    This is an outline of how we here at Notre Dame have been making digitized versions of our Catholic pamphlets available on the Web — a workflow: Save PDF files to a common file system – This can be as simple as a shared hard disk or removable media. Ingest PDF files into Fedora to […]<\/p>\n","protected":false},"author":92,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-388","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/posts\/388","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/users\/92"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/comments?post=388"}],"version-history":[{"count":8,"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/posts\/388\/revisions"}],"predecessor-version":[{"id":399,"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/posts\/388\/revisions\/399"}],"wp:attachment":[{"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/media?parent=388"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/categories?post=388"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.nd.edu\/emorgan\/wp-json\/wp\/v2\/tags?post=388"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}