In any long-running project at Hesburgh Libraries<\/a>, our developer teams make countless decisions every day. Some decisions are big and some are small. \u2014 some affect a few people while others have an impact on the entire organization.<\/span><\/p>\n Inevitably, these decisions evolve over time. <\/span><\/p>\n Sometimes we have to adjust or even reverse a decision after gathering more information or gaining experience with a tool or software. We don\u2019t sweat the ebb and flow \u2014 we welcome it. Embracing decision-making as an evolutionary process is one of our guiding principles for a healthy team culture.<\/span><\/p>\n We also realize that decisions are only as good as the documentation and communication processes that underpin them.<\/span><\/p>\n Documenting our decisions from every angle helps us understand where we’re going and why. Image: Eadweard Muybridge, “Eagle” Walking, Free, plate 576 from Animal Locomotion, 1845-1904, albumen silver print. The Janos Scholz collection of 19th century photography, Snite Museum of Art, University of Notre Dame, 1981.031.543.<\/p><\/div>\n To this end, we use consistent documentation and transparent communication to serve as a two-way roadmap for new challenges, team discussions, and retrospectives in the midst of a rapidly changing landscape. <\/span><\/p>\n These decision documents also help to facilitate conversations with stakeholders and build enduring relationships with project partners.<\/span><\/p>\n The cases below illustrate how decision documents and transparent communications during the MARBLE project have contributed to team success and project impact.<\/span><\/p>\n One of the goals of the MARBLE software development project funded by the <\/span>Andrew W. Mellon Foundation<\/span><\/a> is to create a unified discovery for digitized cultural heritage collections held by the <\/span>Snite Museum of Art<\/span><\/a> and <\/span>Hesburgh Libraries<\/span><\/a> at the <\/span>University of Notre Dame<\/span><\/a>. <\/span><\/p>\n Our immediate aim is to make these objects discoverable in the context of a collaborative digital collections platform. We also surmised that we may want to someday make the digitized objects discoverable through our general library catalog. <\/span><\/p>\n Given these aspirations, we decided to leverage the library-wide discovery system as our search and discovery interface. library-wide discovery system is a vendor-supplied search index software used by many libraries around the world as their primary catalog. <\/span><\/p>\n On the surface, this decision ran contrary to another goal of our project: to develop and release open-source software.<\/span><\/p>\n To reconcile these apparent contradictions and keep our road map intact, I wrote a decision document supporting the use of our library-wide discovery system and how we would proceed with delivering open-source software. We clarified that we would use an API to interact with our search index. (An API, or Application Programming Interface, is a protocol or specification that allows information to transfer from one system to another. Using an API is a simple and common practice in developing new software.) <\/span><\/p>\n In this case, we viewed the decision to interact with an API as a way to support other institutions or potential adopters that don\u2019t use our<\/span> discovery system<\/span>. In other words, our solution would be built in such a way that another institution could connect with their search index of choice.<\/span><\/p>\n We shared our draft decision with project leadership, stakeholders, and developer teams to solicit feedback. From the feedback, we amended the draft document to reflect any new considerations, questions, and challenges.<\/span><\/p>\n With a decision document firm in hand, we <\/span>began working on implementing our solution<\/span>. We gathered help from other library-wide discovery system adopters. (Thank you, <\/span>Northwestern Libraries<\/span><\/a>!). We dove deeper into our usage of library-wide discovery system, expanding our expertise and understanding of a technology we have long used.<\/span><\/p>\n Then we hit a wall. <\/span><\/p>\n Our user interviews identified full-text search as a key desired feature. According to library-wide discovery system documentation, this functionality should have worked. But, it didn’t, and we entered into a “waiting on vendor response” holding pattern.<\/span><\/p>\n While waiting, one of our developers explored ElasticSearch as another option.. After only a few afternoons of work and testing, ElasticSearch proved to be a viable alternative. Within a week, we referenced our documents. We reassessed our prior decision to leverage our library-wide discovery system and chose to pivot towards ElasticSearch. <\/span><\/p>\n Pivoting on a decision takes balance and flexibility. Image: Edgar Degas, Study of a Ballet Dancer, ca. 1880-1885, brown conte crayon and pink chalk on paper. Gift of John D. Reilly ND’63, ’64 B.S., Snite Museum of Art, University of Notre Dame, 2004.053.004.<\/p><\/div>\n Again, I wrote up a decision document outlining the rationale, process, and lessons learned. For example:<\/span><\/p>\n Another problem we encountered during the development of the MARBLE project was choosing an <\/span>International Image Interoperability Framework<\/span><\/a> (IIIF) compliant image server. <\/span><\/p>\n Early in the project, we chose to implement Cantaloupe from the list of <\/span>known server options<\/span><\/a>. With that decision documented and shared, we built blueprints to deploy our Cantaloupe instance into Amazon Web Services (AWS) as a <\/span>Fargate container<\/span><\/a>.<\/span><\/p>\n This worked to get us started.<\/span><\/p>\n However, as we added more and more images to Cantaloupe, we encountered problems such as spikes in response times, incidents of high error rates, numerous restarts. We soon discovered the root cause: Cantaloupe’s architecture conflicts with AWS’s Fargate container implementation.<\/span><\/p>\n Our options were to move to a more expensive AWS service or look for something else and a possible contender emerged.<\/span><\/p>\n Our colleagues at Northwestern University, David Schober and Michael Klein, presented “Building node-iiif: A performant, standards-compliant IIIF service in < 500 lines of code” at <\/span>Open Repositories 2019<\/span><\/a>. After a quick conversation, they pointed us to <\/span>their implementation<\/span><\/a>, a serverless service.<\/span><\/p>\n Learning from our community is crucial to the development process. As has become our practice, we documented a plan to experiment with the serverless implementation. <\/span><\/p>\n You can see our archived <\/span>image-server repository<\/span><\/a> and a snapshot of the <\/span>blueprints to build this out in AWS<\/span><\/a>. Here is <\/span>the code commit that moved our blueprints from Cantaloupe to Serverless<\/span><\/a>. You can also look at <\/span>our documentation<\/span><\/a> evaluating migrating from Cantaloupe to serverless.<\/span><\/p>\n The key takeaway is that it\u2019s worth taking the time to document decisions and have consistent communications. <\/span><\/p>\n It\u2019s true that not every decision necessitates thorough documentation. However, the decisions that require widespread buy-in, impact a key tool or process, or re-orient project goals deserve an organization-wide commitment to this evolving decision-making process. <\/span><\/p>\n For me, decision documents should identify the problem that needs to be solved and includes context, considerations, and constraints. Teams should build decision documents by seeking the input of those with a significant stake in this problem.<\/span><\/p>\n Because we have taken the time to document milestones and decisions, our project is modeling how to have a more robust memory of a particular problem and attempted solutions. We are able to be visionary and more agile as we create solutions to meet stakeholder needs.<\/span><\/p>\n Simply said, decision documents make all the difference.<\/span><\/p>\n And, as a bonus, it was much easier to write this blog post. So, go forth and document! <\/span><\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" by Jeremy Friesen Introduction In any long-running project at Hesburgh Libraries, our developer teams make countless decisions every day. Some decisions are big and some are small. \u2014 some affect a few people while others have an impact on the … Continue reading ![\"Photograph](\"http:\/\/sites.nd.edu\/marble\/files\/2020\/05\/1981_031_543-v0001-scaled.jpg\")
A tale of two documented decisions<\/strong><\/h2>\n
![\"Drawing](\"http:\/\/sites.nd.edu\/marble\/files\/2020\/05\/2004_053_004-v0001-scaled.jpg\")
\n
Amazon Web Services: Are you being serverless?<\/strong><\/h2>\n
![\"\"](\"http:\/\/sites.nd.edu\/marble\/files\/2020\/05\/1954_005-v0001.jpg\")
Image: Flemish, The Lawyer’s Office, after Marinus van Reymerswaele, 1535-1590, oil on cradled panel. Gift of Dr. and Mrs. Dudley B. Kean, Snite Museum of Art, University of Notre Dame, 1954.005.<\/p><\/div>\n\n
Conclusion<\/strong><\/h2>\n