Read our blog

HydraDAM2

The National Endowment for the Humanities recently awarded the Indiana University Libraries and WGBH Boston a grant to support the development of HydraDAM2. This preservation-oriented digital asset management system for time-based media will improve upon WGBH’s existing HydraDAM system and work seamlessly with the Avalon Media System for user access, among other features. 

 

Both HydraDAM and the Avalon Media System grew from the Hydra community. Hydra is an open source technology framework that supports the creation of preservation and access applications for digital assets based on the Fedora repository system. A community of institutions known as the Hydra Partners works together to maintain the framework and create applications for local or shared use by libraries, archives, and cultural institutions. Both Indiana University and WGBH Boston are among the 25 Hydra Partner institutions. Indiana University is collaborating with Northwestern University on the development of the Avalon Media System and WGBH developed the original HydraDAM system with help from the Data Curation Experts group.

 

HydraDAM

The current version of HydraDAM is designed to help libraries, archives, and cultural institutions preserve audio and video files. HydraDAM accepts batches of files via FTP or a browser upload, extracts technical metadata, and moves the files and metadata into a Fedora 3 based preservation repository and a hierarchical storage management system. HydraDAM accepts multiple file formats. Via the user interface, users may also edit metadata and set access controls.

 

Watch a demo of HydraDAM during an Open Source Digital Preservation and Access session at the Association of Moving Image Archivists Conference 2014 (start at 35 minutes):

 

The Avalon Media System

The Avalon Media System is focused on the needs of these same types of institutions in providing access to digital audio and video. The software is designed with particular metadata and authentication capabilities, along with many other features that support its usefulness for students, researchers, and teachers. Avalon is currently in production or in production planning at several institutions.

 

HydraDAM2

In the HydraDAM2 project, Indiana University and WGBH will:

 

1) Extend HydraDAM to work on Fedora 4

 

2) Develop Fedora 4 content models for audio and video preservation objects based on current standards and best practices and utilizing new features in Fedora 4 for storage and indexing of RDF

 

3) Implement support in HydraDAM for two different storage models, appropriate to different types of institutions:

a. direct management of media files stored on spinning disk or on tape in a hierarchical storage management (HSM) system; and

b. indirect management and tracking of media files stored offline on LTO tapes

 

4) Integrate HydraDAM into preservation workflows that feed access systems including Avalon and WGBH’s Open Vault and conduct testing of large files and high-throughput workflows

 

5) Document and disseminate information about implementation and experience to the library, archive, digital repository, and audiovisual preservation communities

 

HydraDAM2 builds upon the philosophy, purpose, and experience of the HydraDAM, Avalon Media System, Open Vault, and Fedora projects. It will support digital preservation of time-based media files in a way that can easily be tied in with access. The Avalon team is very excited to see this project develop and be shared with the community.

 

 

Blog Categories:

2014 in Review

2014 was a year filled with development milestones and community engagement for the Avalon Media System team. We achieved implementation at both development partner institutions, Northwestern University and Indiana University this year. We also focused on spreading the word about the Avalon Media System across the library and archives community and connected with several institutions planning to implement the Avalon Media System or in the evaluation process. Some of the adopters even started work on new features that will be included in the Avalon core code in 2015.

 

The Avalon development team also continued their progress throughout the year, embarking upon our first series of minor releases. Continue reading for development, implementation, and presentation highlights from each month of 2014.

 

Fleming, Declan. 2014. Hydra Connect, La Jolla, CA. Flickr. Web. 20 Jan. 2015. <https://www.flickr.com/photos/bigdpix/12097356495/>.

 

In January, Jon, Mark, Julie R., Michael, Chris, and Claire attended the first Hydra Connect meeting on the University of California, San Diego campus. Over the course of four days, the developers and managers of our team joined with other Hydra project participants to synchronize development plans involving the Hydra framework and share their experiences.

 

In February, the Avalon Media System became compatible with different Learning Management Systems (LMS) that use the Learning Tools Interoperability (LTI) standard, such as Blackboard, Canvas and Sakai (Sprint 45 – February 21, 2014). Students and faculty members associated with classes gained the ability to view Avalon media items through their institution’s LMS. This feature was an important goal for Avalon since it supports the use of media collections for teaching and learning.

 

In March, Michael and Julie R. presented “Avalon Media System: Building for Ourselves and Others” at Code4Lib in Raleigh, North Carolina. Their presentation discussed how community engagement has shaped the requirements of the Avalon Media System and emphasized its importance in developing future goals and Avalon’s continued success.  Their 16 foot long graph of Avalon’s dependencies made quite a splash!

 

In April, the team released Avalon 3.0.

 

In May, the team began development of Avalon’s first minor release, 3.1. We also began ingesting the Indiana University Libraries Moving Image Archive’s digital collections into Indiana University’s test instance of the Avalon Media System.

 

Photo of Open Repositories conference via Flickr user todd.visionShared without alterations under CC BY-SA 2.0<https://www.flickr.com/photos/16894927@N08/14228456869/>

 

In June, Jon, Claire, and Julie H. attended the Open Repositories conference in Helsinki, Finland. Jon and Claire had the opportunity to discuss and demonstrate Avalon 3.0 at the poster reception.

 

In July, we celebrated the release of Avalon 3.1 and Northwestern University’s new production instance of the Avalon Media System.

 

In August, we conducted our first community survey and made connections with six new institutions working on production versions of the Avalon Media System. Around a dozen institutions reported work on pilot implementations and about two dozen institutions reported current or future investigations of Avalon.

 

Photo via Flickr user MarkNotess. Shared without alterations under CC BY-SA 2.0. <https://www.flickr.com/photos/61919416@N00/15254158978/

 

In September, Jon, Claire, Mark, Chris, Julie H., Julie R., Brian, Leah, and Michael attended Hydra Connect 2 on the Case Western University campus in Cleveland, OH. Michael demonstrated Avalon during a panel of production systems and Jon participated in the Opening Plenary (see photo above.)

 

In October, we achieved production implementation at Indiana University. At the Association of Association of Moving Image Archivists Annual Conference in Savannah, GA, Stefan discussed the way the Hydra framework is used at Northwestern University and Jon discussed the Avalon Media System in further depth. At the Digital Library Federation Forum in Atlanta, GA, Julie H. presented a poster on metadata in the Avalon Media System and Jon and Julie R. led a panel discussing experiences with Avalon at five different universities. The Avalon team also began coordinating with the University of Virginia to develop several new features.

   

In November, Jon, Mark, and Julie R. visited the Yale University Library to demonstrate Avalon and explain its technical architecture and future development plans.

 

In December, The National Endowment for the Humanities announced the award of a two-year Preservation and Access Research and Development grant to Indiana University and Hydra partner WGBH/Boston to createa Hydra-based digital preservation repository environment for audio and video media that builds on WGBH’s previous work on the HydraDAM system and will integrate with Avalon for access. We ended the year with the release of Avalon 3.2.

 

Over the course of 2014, we presented at 5 different conferences, completed 22 sprints, and shared one major and two minor releases of the Avalon Media System. We’re looking forward to another productive year, filled with new partnerships and exciting new features. Avalon 4.0 is slated for this spring and will include media processing queue management and more robust metadata features such as media section navigation based on structural metadata and bulk metadata updates for media items. Migration to Fedora 4 is also planned, along with accessibility improvements, playlist and clip creation features, and much more. 

 

Blog Categories:

Sprint 66 - January 23, 2015

Back from the holiday break, we began work on our next release, 3.3. We added the capability of importing descriptive metadata from an OPAC by specifying a catalog ID on the Resource Description page. We also added the capability of ingesting multiple quality levels of pre-transcoded derivatives via the batch ingest process. Previously, batch ingest could only add one quality level of pre-transcoded derivative. We also fixed two 3.2 release bugs we discovered. A patch for one is available and the second will shortly be available.
 

Sprint 66 - January 23, 2015 - Demo recording
 

Patched Bugs:

  • VOV-3344 - When you edit a collection name, the items disappear. (Patched in release 3.2.1.)
  • VOV-3347 - Bulk actions only operate on 10 items at a time. (Will be patched in release 3.2.2.)
     

Completed Stories:

  • VOV-1639 - When adding one item, I want to pull in bibliographic info for the new item from our OPAC
  • VOV-2949 - I want to ingest pre-transcoded derivatives with multiple quality levels via batch
  • VOV-3317 - I want automated MARC import records to look good in my system

 

Blog Categories:

Sprint 65 – January 9, 2015

We had another four-week sprint due to the holiday season. In this sprint we squashed a few last minute bugs and we spent some time upgrading our demo server and our local systems to the latest release 3.2, which was announced during the last sprint. We also started work on new and exciting features for our next release including importing bibliographic information from an external system. 

 

Though we did not record this past demo, you can see the stories we completed below: 

  • VOV-3169 – I want to upgrade to 3.2
  • VOV-3262 – I want to see 3.2 on pawpaw
  • VOV-3266 – [IU] I want to make sure IU's Avalon branding looks good on 3.2
  • VOV-3269 – [IU] I want the IJCCR server updated to 3.2
  • VOV-1639 – When adding one item, I want to pull in bibliographic info for the new item from our OPAC

Blog Categories:

Avalon 3.2 Now Available

We are delighted to announce Avalon Media System 3.2, completed and released on December 19, 2014. As part of a series of minor 3.0 releases, Avalon 3.2 provides support for important content management efficiencies and other improvements.

 

Release 3.2 adds the following capabilities:

  • New metadata fields, including Bibliographic ID, Language, Physical Description, Related Item, and Terms of Use
  • Bulk item management actions, including publish, un-publish, change collection, delete, and assign access
  • Avalon dropbox subdirectories are accessible to collection managers using the web interface
  • Upgrade to Hydra 7
  • Numerous interface improvements and bug fixes

For more details on each of these new features, visit the What’s New in Avalon 3.2 wiki page

 

We welcome you to try out Avalon 3.2 on our public test server (https://pawpaw.dlib.indiana.edu/) before installation. Installation options include virtual machine image, manual installation, and source code installation. More information on all available options is found the Download page.

 

Please share your feedback on Release 3.2 in the comments or on the Avalon discussion list, avalon-discuss-l@list.indiana.edu. Join the discussion list via our Connect page.

 

Sprint 64 – December 12, 2014

This was a four-week sprint due to the Thanksgiving holidays. In addition to fixing many bugs in preparation for the 3.2 release, we also added support for bulk change of access control settings. Beyond those activities, the majority of the work went into troubleshooting the 3.2 upgrade process.

 

The 3.2 release is imminently expected toward the beginning of our next sprint, which will also be four weeks in length. Look for a 3.2 announcement shortly!

 

Learn the details of each story and view our Sprint Demo recording below:

 

Sprint 64 - December 12, 2014 Demo Recording

  • VOV-2759 – I want to see and connect to basic info about the item I'm viewing 
  • VOV-2763 – I want to bulk change access control settings 
  • VOV-3123 – I want to know what bugs are out there so I can prioritize which need fixing 
  • VOV-3169 – I want to upgrade to 3.2 
  • VOV-3170 – I want to know how to use the new 3.2 functionality 

 

Blog Categories:

Sprint 63 – November 14, 2014

During this sprint we added additional fields to our resource description form based on feedback from our users and we hope they are useful to you as well. These fields include:

  • Language
  • Original Physical Description
  • Related Item
  • Terms of Use

 

We also began working on two new features - importing multiple pre-transcoded derivatives and importing metadata from an ILS.  Though we got a good start on that work, we decided to refocus our efforts toward getting release 3.2 out the door. So you can look for those two features in next the next release, 3.2.1. 

 

We worked on quite a few bugs which were mostly related to indexing and sorting on certain fields. Next sprint we will focus on testing and fixing bugs in anticipation for our upcoming release.

 

For this sprint demo, the we experienced recording issues, so we aren't posting a video. However, if you'd like to see any of the functionality, please contact us. Learn the details of each story below:

 
  • VOV-3005 – I want to build a process for migrating my content to different bitrates
  • VOV-2949 – I want to ingest pre-transcoded derivatives with multiple quality levels via batch
  • VOV-2759 – I want to see and connect to basic info about the item I'm viewing
  • VOV-3130 – I want to know how to map MARC fields to the current (as of VOV-2759) Avalon fields
  • VOV-3155 – I want scripts for manual testing of bulk change operations new in 3.2

Blog Categories:

Evaluation evolution

Tricia Patterson, the National Digital Stewardship Resident at MIT Libraries is helping evaluate Avalon for MIT's Lewis Music Library. Learn about the team's process and the requirements they have established for media management systems.

 


 

Originally posted on SIPs, DIPs, & bytes: NDSR Boston's Digital Preservation Test Kitchen:

 

As I referenced in my post about World Day for Audiovisual Heritage, MIT’s Lewis Music Library is really looking forward to not only preserving the audio content that we are digitizing, but also finally expanding access to it and awareness of it. For this, we need to identify a suitable access platform – no small feat, I have discovered! Large feat, in fact. Size 11s, at least. Since hearing about the myriad of considerations that inform a software evaluation during one of my MSLIS courses, I’ve actually been harboring a desire to be a part of one – and I haven’t been disappointed! So I thought I would share some of our evaluation process so far.

 

View the original (450 more words)

 

Recent Developments at the University of Virginia

We are pleased to share some news coming from the University of Virginia (UVA). UVA is currently developing several new features for Avalon as part of their plan to implement the software at their institution. The new functionality will be shared with the entire Avalon community since UVA’s contributions will be integrated into the Avalon core code.

 

UVA developers and the Avalon team members are currently working together to determine goals and coordinate their efforts. Mike Durbin, Senior Software & Systems Engineer at the University of Virginia Libraries, recently visited Indiana University to join forces with the Avalon developers. The productive week at IU was also a welcome opportunity to catch up with his former IU Libraries Technologies co-workers.

 

Brian (IU), Chris (IU), and Mike (UVA) discussing Avalon at Indiana University

 

I asked Mike to share some details about UVA's plans for the Avalon Media System.

 

What types of collections does Avalon help provide access to at UVA? What types of collections will be made available in the future? 

 

Currently Avalon provides access to several collections falling into three broad categories.  One consists of licensed streaming content which is accessible both to UVA and a Virginia consortium of libraries, VIVA.  Another category is local digitized rare and unique materials from our Special Collections library.  And finally, locally produced content such as concert recordings and lecture series are made available to the widest audience allowed by the materials rights encumbrance.  Besides expanding the use of content in these categories, we are considering adding electronic reserves material to support classroom instruction in the future.

 

How did the relationship between the University of Virginia and the Avalon team begin? What will their collaboration look like?

 

Initially conversations between UVA and the Avalon team about Avalon began on the Avalon IRC channel (#projectvov).  As it became clear that we at UVA would have the time and the need to make modifications to the Avalon codebase, it seemed best to coordinate any efforts with the project team.  In-person conversations and the focus of taking time to work exclusively on this effort made changes and the planning of future work easier.

 

What are some of the new features that the UVA team will share?

 

We have improved support for iPhones for the default streaming server. We did this by implementing a fix to the generation routines for IOS-specific streaming assets for users of Red5.  While it is not definite that other institutions use the open source Red5 streaming server in their implementations, these changes ensure broader compatibility.

We also developed a working example that allows users from multiple institutions to sign in to Avalon using their familiar institutional single-sign-on page.  To do this, we generalized our Omniauth-Shibboleth Ruby gem that allows easy integration of Shibboleth authentication with Avalon.  This allows us to add permissions to resources based on a user’s affiliation (the institution from which they authenticate) to support our current use-case of providing access just to "Virtual Library of Virginia" (VIVA) member institutions.   This will also allow us to grant access on a per-item basis to scholars at any of nearly 400 institutions that make up the InCommon federation.  That work is available for reuse at: https://github.com/uvalib/omniauth-shibboleth.

 

 

Soon after visiting Indiana University, Mike presented “The Avalon Media System at the University of Virginia” as part of the “Avalon Media System: Implementation and Community” session at the 2014 Digital Library Federation (DLF) Forum on October 28, 2014. Check out the slides from his presentation below:

 

 

Sprint 62 – October 31, 2014

At the very beginning of the sprint, IU's general Avalon instance successfully moved to production. We have started enhancing the metadata by providing a repeatable field to hold other system identifiers such as bib numbers, and we provided a way to facet and display using creation date (if available) not just publication date. We fixed a problem with ID specification for LTI access. We designed how bulk access control editing will work, and we reviewed the Hydra Works proposal. We participated in the Digital Library Federation (DLF) Forum, having discussions with many institutions about Avalon.

 

For this sprint demo, the audio didn't work, so we aren't posting a video of the demo. However, if you'd like to see any of the functionality, please contact us. Learn the details of each story below:

  • VOV-2754 – I want to be able to add classes using friendly ID even if no one has logged in via LTI yet
  • VOV-2946 – I want a place (for both batch and interactive) to put an ID from another system (BIB, OCLC, etc)
  • VOV-3034 – I want to see the creation date
  • VOV-3098 – I want to know if the proposed Hydra Works model might work well for Avalon Story
  • VOV-3104 – I want to know how bulk editing Access Control settings should look and behave

 

Blog Categories:

Pages