Read our blog

Northwestern University Goes Live with the Avalon Media System

Live from Northwestern University, please join us in welcoming the new Repository | audio + video! Northwestern’s media repository currently features two publicly accessible collections: Northwestern University Football Films and the Robert Marcellus Master Class Audio Archives. The team at Northwestern University tailored the product to meet their unique needs. While the NU branding bars and colors may be the most striking change, the team also made some behind-the-scenes changes, including the integration of their Handle System for item identification and additional permissions for collection editors. 

 

Check out Northwestern University’s Repository | audio + video or watch the Sprint 55 recording for a tour of Repository | audio + video from the perspective of a Northwestern University collection manager (see section [Demo] VOV-2769 - NU production go live by July 3).

 

Release 3.1 Now Available

We’re pleased to announce the release of Avalon 3.1. The major goal for our first dot release is to pave the way for production implementations of the Avalon Media System at Northwestern University and Indiana University, the Avalon software development partners. We’ve achieved production at Northwestern University and are looking forward to taking this next step at Indiana University.

 

Release 3.1 adds the following capabilities:

  • Use LDAP groups to restrict item accessibility
  • Change a thumbnail after the master file is gone
  • Add a previously existing PURL or Handle to an Avalon object, in place of the Avalon-generated permalink
  • Ingest previously transcoded files attached from the dropbox via “skip transcoding”
  • Public Avalon items produce more meaningful results in search engines such as Google
  • Improvements to error messages when batch ingest fails due to manifest file problems
  • Numerous bug fixes involving thumbnails, dropbox, batch manifest files, and more

 

For more details on each of these new features, visit the What’s New in Avalon 3.1 wiki page

 

We welcome you to try out Avalon 3.1 on our public test server before installation. Installation options include virtual machine image, manual installation, and source code installation. More information on all available options is found the Download page.

 

Please share your feedback on release 3.1 via the Avalon discussion list, which may be joined through the Connect page.

Tags:

Blog Categories:

Sprint 54 – June 27, 2014

The Avalon team has been busy balancing improvements, testing the Avalon 3.1 source code, and eliminating bugs in preparation for our next release. Avalon html pages have been updated with search engine optimization metadata and microdata to increase the clarity and usefulness of information provided in Google search results about Avalon media items. We’ve also updated the Avalon Collections Guide wiki page to reflect the new features of our upcoming release. Our major tasks for the next sprint involve testing release candidate 3.1 and fixing any bugs we find. 

 

Sprint 54 – June 27, 2014 Demo Recording

  • VOV-2755 - I want my public Avalon items to show up cheerfully in search engines such as Google
  • VOV-2758 - I want to deploy a high-quality 3.1 release
  • VOV-2761 - I want to understand what's in 3.1 and how to configure it
  • VOV-2820 - I want to have up-to-date collection management documentation for 3.1
  • VOV-2810 - When viewing embedded Avalon content, I want to link back to the item in Avalon

Blog Categories:

Sprint 53 – June 13, 2014

In Sprint 53, the Avalon team completed work on LDAP-based access control and made significant improvements to the error messages generated by the batch ingest process. We’re working to strengthen the information provided about Avalon media items in Google search results with Schema.org and Google Webmaster Tools serving as our main resources in this process. 

 

To learn about all of Avalon 3.1’s upcoming features in advance of the release, check out the What’s New in Avalon 3.1 wiki page. 

 

Sprint 53 – June 13, 2014 Demo Recording

  • VOV-1717 - I want to restrict item accessibility to members of one or more dynamic LDAP groups
  • VOV-2757 - I want helpful error emails when my batch fails due to manifest file problems
  • VOV-2755 - I want my public Avalon items to show up cheerfully in search engines such as Google
  • VOV-2758 - I want to deploy a high-quality 3.1 release
  • VOV-2761 - I want to understand what's in 3.1 and how to configure it

Blog Categories:

Metadata in the Avalon Media System, present and future

Julie Hardesty at Indiana University (IU) and Karen Miller at Northwestern University (NU) serve as metadata specialists on the Avalon team, working to develop requirements and standards for metadata in the Avalon Media System. In this interview Julie shares their insight into the current status of metadata in Avalon along with future considerations.

 

What types of metadata may be created for items in the Avalon Media System?

In Avalon it is possible to convey descriptive, structural, and content information about items. Descriptive metadata lets you know things about an Avalon item such as the title, creator, date when the item was created or issued, publisher, subject or genre information, and whether it’s a moving image or an audio item – information that will help you identify and differentiate that Avalon item from another Avalon item.

 

Structural metadata deals with the digitized files that make up the digitized item (.mp3, .mp4, .wav, .mov, etc.). For example, if there are multiple audio files that combine together to create a single recording of the IU Symphony Orchestra from October 20, 1982, structural metadata is going to tell you which files are part of that Avalon item and how they should be ordered to correctly play back the symphony performance from start to finish.

 

Content metadata reflects table of contents/track-listing/chapter list information, so the order of things is important but it also identifies the content on the digitized files. It’s possible that content metadata overlaps with structural metadata (a single track from a track list could very well be split across 2 or more digitized files) so content and structural metadata have to be able to work together pretty tightly.

 

Why did you decide on MODS for descriptive metadata over other schemas?

We decided on MODS because IU and NU both already have a majority of our collections described at some level in our library catalogs. This means MARC data is abundant. It is possible to crosswalk existing MARC metadata into MODS, we have expertise in MODS (Karen Miller at NU is so the MOD-est of the MODS), and we can take MODS and express that descriptive info in Dublin Core or PBCore or any other metadata standard needed.

PBCore was a standard we considered for descriptive metadata, but with PBCore, there wasn’t always a good fit between our MARC info and the descriptive info that PBCore can track and having PBCore as the base for descriptive metadata also meant more difficult transformation work for us.

 

Which MODS terms are available in Avalon?

So we have MODS and then we have MODS in Avalon. What Avalon currently offers is a small set of what Karen and I want to have available. Right now we are limited to title, creator, and date info (which is required for any Avalon object) and then there is space for contributors, publishers, subjects, genres, a general description/abstract, and the type of resource (video or audio). These were easy to translate into MODS and seemed essential to be able to describe an Avalon item and differentiate it from other Avalon items at a basic level.

 

What new MODS terms would you like to add?

Right now, we don’t have a great way to detail publisher info, just a single field. It would be nice to distinguish between publisher name and publisher place because if you have one, you often have the other. Additionally, identifying roles for creators and contributors would make the descriptive info a lot more helpful, I think.

 

When can we expect new MODS terms to be added?

As we start producing interim releases (“dot” releases, like 3.1 and 3.2, etc.), my hope is to bring in additional descriptive fields with each “dot” release.  Some of this is also being helped along by the needs of real places trying to make use of Avalon at IU and NU. Language and physical description (of the original item that was digitized) are hopefully going to be added with the next “dot” release, for example, because the IU Moving Image Archive needs that information along with our other fields to describe and differentiate their items at what they consider a basic level.

 

A metadata schema isn’t currently being used for structural metadata. How does this work?

We currently encase our structural metadata within Fedora’s system instead of starting with a standard like METS. To make an Avalon item, MODS metadata is stored as a datastream in the Fedora digital repository alongside the digitized files (mods.xml is right there with the audio.mp3, for example). The structure of the Avalon item (for example, 4 files that should be in a certain order for proper playback) is actually being handled by RELS-EXT (Relationships-External) in Fedora, relationship definitions defined for Fedora digital repository objects and expressed in RDF XML. So we’re taking advantage of what Fedora can do. This is related to our Agile development process in that it was quicker to use Fedora’s relationship definitions to get things started and functional for Avalon items structured with multiple files than to implement a METS structure that then required further programming to express this structure within Avalon.

 

How is Avalon currently dealing with content metadata?

Avalon is not dealing with content metadata in the current version (Avalon 3.0). METS currently seems to be the best option for implementing content metadata but I think we are open to hearing from our partners and potential users about what they need and how they’re interested in using this information. My experience with METS is that it can be easily overdone because EVERYTHING can go into a single METS file (structural and content information – even MODS can be wrapped inside METS) – it can turn into a giant beast that becomes hard to manage over time. I have some experience with this and know it can lead to logistical problems for updating METS information, something that I don’t want to see happen with Avalon.

 

So the team is exploring the best way to implement structural and content metadata. What are some of the contenders and considerations?

PBCore is a standard we keep looking at, thinking about, and then backing away from for different reasons at different times. It might be worth it to explore what PBCore can do for us in comparison to METS. As a standard, it allows for descriptive, technical, and structural/content metadata on multiple types of time-based media, but not quite the way we want to use it for our library catalog-sourced data and not quite the way we can build on over time. PBCore is all about the instantiations (all the different versions of a time-based item), which is where the structural/content metadata is located. In our previous releases, we weren’t dealing with any of that data, only descriptive metadata.

 

I think other factors affecting our decisions about PBCore are share-ability (who among our partners and users can easily make use of PBCore and how do they make use of it – please let us know!) and support (does PBCore have a solid structure for long-term support and maintenance as a metadata standard). Again, it would be great to receive feedback and thoughts from partners and others who have tried Avalon or are considering using Avalon to know if a certain direction is going to make or break things for you.

 

Encasing our structural metadata within Fedora’s system is familiar but also requires a lot more programming overhead. Maybe it doesn’t matter what we do for Avalon’s functional purposes as long as we bake in the ability to expose content and structural metadata in a standard format for external uses. It’s a hurdle regardless of the metadata standard we choose and how we implement the functionality in Avalon, but I think if anyone can make it work, it’s the Avalon team!

 

We encourage you to share your thoughts with Julie Hardesty at jlhardes@iu.edu. To discuss these topics with the larger Avalon community, join our discussion list on the connect page.

Tags:

Blog Categories:

Sprint 52 – May 30, 2014

In our last sprint, the Avalon team continued its progress on a major feature of 3.1, allowing Avalon collection managers to use LDAP groups to provide access to content. With Avalon 3.1, content permissions may be assigned to user groups created within Avalon, to LTI courses, and to LDAP groups. Enabling LDAP groups requires a simple configuration. The organization of LDAP groups and use of LDAP active directories vary by institution, so we are exploring the most efficient method for Avalon to search for LDAP groups.

 

We’ve completed work on the option to bypass transcoding of media files attached through the Avalon dropbox. New tooltips were written to clarify the process of assigning pre-existing permalinks, setting thumbnail images, and creating section labels for media content with multiple segments. 

 

To ease the transition to Avalon at Indiana University from the current media service, we’re working closely with IU Libraries Media & Reserves Services and IU Libraries Moving Image Archive. We’re planning an ingest workflow for their current and future content and have translated the current service’s metadata fields to Avalon’s so that records can be transferred.

 

Sprint 52 – May 30, 2014 Demo Recording

  • VOV-2661 - I want to use Skip Transcoding for files attached from the dropbox.
  • VOV-1717 - I want to restrict item accessibility to members of one or more dynamic LDAP groups
  • VOV-2727 - I want contextual help about permalinks in the file management page
  • VOV-2463 - BUG: Dropbox directories with spaces break batch ingest
  • VOV-2723 - [IU Prod] I want to know what an Avalon-based VSS interactive ingest workflow will look like at IU
  • VOV-2696 - [IU Pilot] Implement LTI for Oncourse and Canvas

Blog Categories:

Sprints 49, 50, & 51 – April 18, May 2, & May 16, 2014

 

Since the release of Avalon 3.0, the team has focused on putting Avalon in production at Northwestern University as well as adding features needed for IU’s move to production. We expect to implement and release the latest features by the end of June as a minor release 3.1.

 

In our latest sprint we created the ability to assign a pre-existing persistent URL (PURL or Handle) to media that has been added to Avalon. We also developed a way for collection managers to choose or change a thumbnail image for a video even after the master file has been deleted or moved elsewhere. Using HLS to create thumbnails means that this image is made from the best available derivative rather than from the master file.

 

Now the Avalon team is working on the option to bypass (skip) transcoding for media files uploaded through the Avalon dropbox. This option may be used when a transcoded version of a file already exists and can be provided instead.

 

In 3.1, Avalon will be able to take advantage of LDAP groups to authorize access to users affiliated with campuses or departments or other LDAP-managed entities.

 

We have assessed the impact of upgrading to Hydra 7, Blacklight 4, Bootstrap 3, and Rails 4. The upgrades have been deferred to after our minor release due to the amount of user interface rework required, but the work will commence shortly. In addition to these dependency upgrades, our next major release is likely to include some of the following features:

  • Bulk operations to enabling publishing, access changes, or deletion of multiple items simultaneously
  • Ingest queue management, so high priority items can be processed quickly even if there is a long queue
  • Automated metadata ingest from OPACs
  • Additional descriptive and administrative metadata fields
  • Time limits on access permissions

What are your thoughts on our progress, ideas, and goals? We’re looking forward to your comments on our mailing list, avalon-discuss-l@list.indiana.edu. Our mailing list is also the place for our partners to share ideas and issues with existing Avalon pilots and implementations. 

 

Sprints for Avalon 3.0:

 

Sprint 49 - April 18, 2014 Demo Recording

  • VOV-2396 - I want to install R3 
  • VOV-2427 - I want have technical documentation for R3
  • VOV-2625 - I want to tryout R3 on the demo server
  • VOV-2629 - [design only] I want to be able to set time limits on sharing Avalon items with individuals, groups and courses.
  • VOV-2630 - I want manual instructions for how to set up Avalon with AMS
  • VOV-2631 - I want to have a technology training refresh after R3

Sprint 50 – May 2, 2014 Demo Recording

  • VOV-2625 - I want to try out R3 on the demo server 
  • VOV-2630 - I want manual instructions for how to set up Avalon with AMS 
  • VOV-2663 - I want to configure R3 features when I set up or upgrade Avalon 
  • VOV-2664 - I want to use R3
  • VOV-2660 - I want to add an previously existing PURL or Handle to an avalon object
  • VOV-2633 - I want to configure dropbox privacy and access 

First Sprint for Avalon 3.1:

 

Sprint 51- May 16, 2014 Demo Recording 

  • VOV-2660 - Add a previously existing PURL or Handle to an Avalon object 
  • VOV-2659 - Change a thumbnail after the master file is gone 
  • VOV-2661 - Skip Transcoding for files attached from the dropbox. 
  • VOV-2690 - Upgrade to the latest Hydra components 
  • VOV-2631 - I want to have a technology training refresh after R3 
  • VOV-2695 - [IU Pilot] Implement PURLs
  • VOV-2626 - [NU Prod] I want to test R3 on NU Production and identify problems 
  • VOV-2628 - [NU Prod] I want to use Avalon with Blackboard and Canvas 

 

Blog Categories:

Release 3.0 Now Available

 

Release 3.0 of the Avalon Media System is now available. Both Indiana University and Northwestern University will be using this release to set up production environments and we hope other institutions will join us.

 

Release 3.0 adds the following capabilities:

  • provide authorization for item access from your learning management system by integrating Avalon as an LTI tool (Learning Tools Interoperability is a standard used by many course management systems)

  • embed Avalon's media player in other web page using embed codes

  • configure Avalon to delete, move, or do nothing to your master files after they are transcoded

  • track location of master files and derivatives in Fedora

  • configure Avalon to assign a permanent URL (e.g., PURL or Handle) when an item is published

  • quickly import previously transcoded derivatives

  • maintain privacy of dropboxes between collections

  • numerous media player bug fixes and improvements

 

There are several options for exploring release 3.0. You can try it out on our public test server. You can also download and install a preconfigured Avalon virtual machine image, perform a step-by-step installation, or find out how to download our source code from Github. More information on all of these options is available on our software page.

 

We look forward to feedback on this release via the avalon-discuss-l list. If you haven't already signed up for the Avalon discussion list, you can find out how to do so on our connect page.

Tags:

Blog Categories:

Coming Out of Our Ears: An Abundance of Media to Preserve and Share at Stanford Libraries

 

This interview is with Hannah Frost, Services Manager for Stanford Media Preservation Lab and Stanford Digital Repository. Hannah has been leading media preservation efforts at Stanford Libraries since 2001 after earning a MLIS in archives and preservation at the University of Texas at Austin. Her first introduction to the challenges of media preservation arose while working as a studio assistant for documentary photographer and media artist Jim Goldberg, who had some finicky Hi8 video tapes with recorded content to be included in a single channel video piece for his "Raised by Wolves" exhibition.

 

What audio and video collections are you considering putting into Avalon?

Stanford envisions a number of ways to use Avalon. We need the ability to gather media content produced by students, faculty, research groups, and departments. Our expanding Stanford Digital Repository service is attracting a lot of interest around campus, and an amazing variety of media material is coming out of the woodwork: podcasts like Generation Anthropocene, public radio programs like Philosophy Talk, and teaching and learning resources like Entrepreneurship Corner. We also imagine using it for the deposit of event recordings like Stanford commencement speeches (see Steve Jobs in 2005), and creative works, like this chamber opera by Stanford professor, composer, and researcher Jonathan Berger. Stanford is brimming with media content produced in the interest of education, research, creativity, and the well-being of society. Much of the content is of high quality, broad appeal, and long-term value. Avalon will greatly facilitate the Libraries’ ability to collect, describe, preserve, and deliver it.

 

Stanford Libraries also plans to use Avalon for bringing in bulk collections of digital video and audio materials from our partners. We are increasingly collaborating with other cultural heritage institutions that have rich digital collections, including media content, to preserve and serve up, but without their own means to do so.

 

The list of use cases continues to grow!

 

"Zooming In on the Mandelbrot Set", a 16mm motion picture film illustrating the work of Benoit B. Mandelbrot, the "father of fractals". Courtesy Stanford University Libraries, Department of Special Collections.

 

Who will be able to access these collections in Avalon?

In many cases, not everyone, and frankly that is part of Avalon’s appeal. Most of the media content in Stanford’s collections is encumbered by copyright, privacy, or licensing terms.  Without an integrated media delivery solution that supports access limitations to the Stanford community, or specific locations or groups within Stanford, it has not been possible for us to make our digital media collections available in a controlled, systematic way.  Avalon provides a full-featured system that meets the array of Stanford’s access requirements.

 

What are some unique or interesting items in your media collections?

We have lots! Just to list a few:

Original 1-inch video footage from the Stanford Prison Experiment, preserved in the archive of Philip G. Zimbardo, Professor of Psychology. Courtesy Stanford University Libraries, University Archives.

 

How will Avalon help you achieve your preservation and access goals?

For over a decade, Stanford Libraries has been actively engaged in media preservation and digital preservation – two major challenges facing research libraries today. Avalon is a natural choice for us: Stanford is a founding member and devotee of Project Hydra and home of the Stanford Media Preservation Lab.  We have invested heavily in establishing a program with supporting facilities and expert staff for reformatting aging, unique media and stewarding the digital copies in a robust preservation environment. However, we have struggled to bring up an access environment that is compatible with both access policies and the repository’s technical infrastructure … until Avalon.  Furthermore, as more and more media is born-digital, we need to expand our capabilities for efficient deposit to our repository.  We believe strongly in flexible, robust, open source software solutions in the service of information preservation and access, and Avalon fits the bill!

 

Tags:

Sprint 48 - April 4, 2014

 

We have an Avalon release candidate! We've tested and tagged the code and completed technical documentation about how to configure permalinks integration. We had new eyes look at our manual installation process. Since our production environments are going to look different from your production environments, standalone instructions for the various components such as Fedora, MySQL, Solr, Red5, Matterhorn and others have been provided to help get Avalon users up and running without too much trouble.  

 

Sprint 48 - April 4, 2014 Demo Recording

  • VoV-2388 – Getting to code freeze – Chris

  • VoV-2545 – R3 Manual Install – Jim

  • VoV-2427 – Technical Documentation – Adam

  • VOV-2396 – R3 Installers – Team (Story not completed)

 

Now the team is working on migration and installers. The work done earlier in the sprint to create a pushbutton automated release process has really helped to speed up this process and we are happy to say our efforts have paid off; instead of taking 6 hours to build an .ova, the process is less than 40 minutes!

 

We should have some exciting news by the end of this next sprint so keep watching this blog, Facebook and Twitter!

Blog Categories:

Pages