#mipops

[mipops]

In case you missed the session at AMIA 2020, here is the full recorded DVRescue presentation. Presented by Ashley Blewer, Libby Hopfauf, Dave Rice and Andrew Weaver.

[mipops]

DVRescue

DV Rescue: Innovations in the Preservation of Digital Videotape Libby Hopfauf & Dave Rice LIBBY Hello everyone! Thanks for joining us. My name is Libby Hopfauf. I am the program manager and audiovisual archivist for Moving Image Preservation of Puget Sound (or MIPoPS) and the Audiovisual Archivis...

Today, Dave Rice and Libby Hopfauf presented on DVRescue at the Library of Congress Labs Informal A/V Summit. You can view the slide deck here. LoC Labs will post the full recording of the summit here in the next few months. 

[mipops]

Using DV Capture in vrecord

For those of you who haven’t tried it out yet, here’s a run-down on capturing DV tapes in vrecord!

Basic Instructions

To capture DV formats in vrecord (when installed in macOS), you will need to have your DV deck connected directly to your computer via FireWire input. Make sure your deck is in “Local” mode. At this time, vrecord is not equipped with tape control, so you will need to manually start and stop the tape just like with analog capture in vrecord.

To select the deck you would like to use, go into the configuration mode either by:

run vrecord -e

clicking on the "Edit Settings" button in the GUI (run vrecord -g)

Switch from the "Decklink" tab to the "DV" tab under "Input Options" at the top of the configuration window.

Select the name of the DV device you want to use from the list.

Specify the playback, sidecar, file naming, recording event and directory options as you normally would for vrecord. For details see Editing Settings.

Click "OK" to save.

You can then run passthrough and record modes the same as you would with analog videotape. Please note that if the timecode does not start at the very beginning of the tape, the record mode viewer will not open until the timecode is detected (as soon as the counter starts moving on your deck, the viewer window should pop-up).

To end the capture, exit out of the ffmplay viewer window by clicking the red “x” in the upper left hand corner. Please note that if your video ends on a freeze frame of video, you will need to either start rewinding the tape or terminate the terminal window in order to exit capture (otherwise, you will end up with the spinning rainbow wheel of doom). 

Tips & Tricks

Currently, some users are experiencing issues with capturing that results in dropped frames or presentation time stamp (PTS) discontinuities. Here are a few variables to test if you encounter this issue:

[mipops]

dvrescue: dvpackager

Background

Software design for tools that transfer DV from tape to file take one of two approaches:

Receives all the DV frames and writes them to disk as a DV stream.

The software receives a series of DV frames, analyzes them to understand their significant properties, and wraps those frames within a container format, such as mov.

Tools such as DVHSCap, Live Capture Plus, and dvgrab use the first method. The resulting frames are stored as a stream of DV frames. The result is very authentic to what was transmitted; however, there can be significant challenges to using this data. DV supports a variety of options within several characteristics. 

Context

Consider this scenario. A cameraperson recording an interview with a musician on MiniDV tape. The interviewee is frameds in a 4/3 aspect ratio and the audio is recorded onto two channels of audio with a 4/3 aspect ratio. During the recording, the cameraperson remembers that the interview is for a production with a 16/9 production, so adjusts the camera's settings and does a second take at 16/9. Following the interview, the cameraperson records a performance by the musician's band and adjusts the camera to record 4 channels of audio rather than 2 in order to capture more of the microphones. This DV tape could be transferred over FireWire to a single, continuous DV stream; however, the resulting DV file would have a variable aspect ratio, a variable sampling rate for the audio, and variable audio channel count. Some players could support some of these changes, for example VLC will adjust the framing as aspect ratio changes though QuickTime won't. The presentation will be inconsistent since the extent of this sort of variability is rarely supported.

Tools such as Final Cut Pro 7 use the second approach. The resulting QuickTime file contains the same DV frames that first method would result in, but stores them in a convenient container. However, when the incoming DV frames contain incoherency in the characteristics, either because the camera settings changed or the data is damaged, the software will generally stop the data transfer at those events. Software that was written to expect perfect DV frames may then start to break up a DV transmission into dozens of individual files with intermittent gaps or simply stop and give error messages.

dvrescue

In order to effectively preserve DV materials in a way that scales, we need to ensure that we can transfer all of the DV data in a manner that is verifiable and also produce audiovisual files that are well-described and interoperable. This requires a mix of the two approaches above. In the design of dvrescue, we use the first approach and initially just write all incoming DVdv streams into a file. This file is intended to be a digital copy of the data stream that the DV tape player produces and contains audio, video, camera metadata, metadata about frame characteristics and information on how well the reading of the tape went. After writing those DV frames to a file, we then analyze it with the dvrescue utility and produce a report on every incoherency in the file: including jumps in timecode, jumps in recording timestamps, or changes in properties like aspect ratio or audio channel count. With that resulting data, a subsequent script, called dvpackager, then uses that report and the original DV file, to selectively encapsulate every frame into an audiovisual container such as QuickTime or Matroska. There are options here, so a user could make one file for every time the camera had the start button hit (since all the recording metadata is preserved within the DV frames) or the DV frames could be divided only by changes in properties like aspect ratio or channel count.

The need for this sort of packaging is demonstrated by a recent example file shared with us by Morgan Morel at BAVC. Morgan transferred the DV data from a tape to a file using DVHSCap, which follows the most approach described above, writing the received DV frames into a file and that's about it. In QuickTime X, the file starts with about a second of grey frames and then cuts to a recording of an art installation. Some fan noise is heard in the background, but the audio is very choppy, like a mixture of noise and silence and the result is jarring to listen to. VLC creates a similar jarring presentation, where the video looks right, but the audio is painful.

This is a presentation of how QuickTime X decodes the audio of this DV stream. Wanna listen?

With dvrescue, we can figure out what is happening. The dvrescue utility will read the DV file and document events and changes that happen throughout the file.

Here we can see that at the beginning of the file, there are 19 frames that use a sampling rate of 48000 Hz, but then the subsequent frames use a sampling rate of 32000. The 19 frames at the beginning are the gray frames mentioned earlier, whereas the rest of the tape is a recording of an art installation. So QuickTime X is looking at those first frames and presuming that it's all at 48000 Hz, but then when it is decoding a DV frame that only has 32000 Hz of audio per second, that only fills ⅔ of a second, thus these large seconds of silence are interleave since the decode expects a certain number of audio samples per second, but isn't getting it.

For cases like this, we are working on a utility called dvpackager. It analyzes the stream to createget XML like (shown above) and then decides if and how to break that DV file into pieces to avoid the presentation issues that Morgan found.

After dvpackager does it's thing, we have two video files. One is very brief, just those gray frames and 48000 Hz audio. And then the recording of the art installation at 32000 Hz. When we unsplice these very different recordings from one another and handle them individually they both play well with their own unique characteristics. This approach uses the best of the two described above. We know that we got all the data, but by analyzing it and encapsulating all frames in the right way, we have content that is accessible, well-described, and ready to be sustained as digital files.

[mipops]

dvrescue: dvplay

Exciting new developments are underway for the DV Rescue project. Most recently, the quality control analyzer component “dvplay” is in the beta stages of testing. dvplay is a quality control player that supports visualization of uncorrected DV data, including categorizing DV transfer errors, illuminating when selective re-transfer may be most likely to offer improvements, and providing a method to easily distinguish authentic DV data from error concealment techniques and unconcealed damage. Currently, dvplay consists of three modes:

dvplay

A frame-by-frame playback of the video file created from a DV tape transfer. Artifacts are highlighted in yellow.

dvplay -x

Screenshot from QuickTime of same frame

JPEG created in dvplay -x mode

This mode runs a check on the video for frames containing artifacts. Each frame where an artifact was detected is saved (in the same directory as the video) with the damaged pixels highlighted in yellow as a JPEG with the timecode printed on the side of the image. The images can then be reviewed to determine if recapture is necessary and possible issues in the files (such as dropouts, headclog, transfer artifacts, camcorder recording artifacts, etc.)

dvplay -m

This mode alternatively highlights all normal video images in yellow and leaves the artifacts unmasked for review.

[mipops]

MIPoPS x Anne Frantilla

Continuing our series of interviews with some of the most important people in the MIPoPS universe, we’re excited to profile Anne Frantilla this week. Anne is the City Archivist for the City of Seattle. She was one of the original founders of MIPoPS, and today serves on our Steering Committee and has an active presence in planning for MIPoPS future. She is a talented writer and a ferocious advocate for the issues and people she cares about - a powerful combination which makes her help invaluable to MIPoPS.

Ari Lavigne: Hi Anne! Could you tell us who you are and what you do for work?

Anne Frantilla: I am the City Archivist for the City of Seattle. I have worked at the City for 20 years but for most of that time I was reference archivist, connecting people with records. I am responsible for supporting the mission of the Office of the City Clerk by fostering civic education, participation, and openness in Seattle city government through transparent, accountable stewardship of public information and official records. The Archives team does great work and I’m pleased to be able to keep us all on track and push us to provide even more in terms of access and awareness of the records in the custody of the Archives. Archives are important institutions because they enable others to understand the perspectives and actions of people in the past. The more records we can make accessible and the more ways we can create awareness and access, the better we will understand our past.

Seattle Municipal Archives maintains an incredible collection of material that lets us look back on the city’s history. (5233 Ballard Avenue NW, 1975. Courtesy of the Seattle Municipal Archives, Item No. 179239.)

Ari: How did you end up in archives?

Anne: I originally went to library school to become a children’s librarian. I took an introductory archives class and we were required to visit three different types of archives. When I started reading the Grimke sisters’ letters at the Clements Library I knew I wanted to be an archivist. These sisters were abolitionists and their passionate letters to each other gave me a new perspective on the anti-slavery movement. Before that class I didn’t know archives existed.

Ari: How did you come to be involved with MIPoPS?

Anne: I met Hannah Palin at the University of Washington Special Collections through my work at SMA. She coordinated a grant involving film that SMA was a part of and that was my first introduction to working with film. Rachel Price volunteered at SMA with film and Carol Shenk was a colleague before she became King County Archivist. We would talk frequently about how wonderful it would be to have a place to digitize film and videotape and the available expertise to instruct us on not only the digitizing process but best practices regarding formats, codecs, etc. We didn’t get the big building we dreamed of but we did get machines and equipment to digitize magnetic media. The expertise came slowly. I’m thrilled to have been part of its beginning.

The MIPoPS office located inside the Seattle Municipal Archives, on the third floor of Seattle's City Hall.

Ari: Could you explain the relationship between MIPoPS and the Seattle Municipal Archives?

Anne: The agreement between MIPoPS and the Office of the City Clerk specified that MIPoPS lends the City it equipment, including the decks for playing various types of tapes, and the Office of the City Clerk, where SMA is housed, lends them space for the equipment. The MIPoPS space is small but mighty. I love having MIPoPS in the Office of the City Clerk near SMA because I get to meet people from all the institutions coming to digitize their tapes.

SMA material (in this case, 3/4" Umatic tapes) ready to be digitized by MIPoPs.

Ari: There are always volunteers, interns, and work study students working on interesting projects at SMA. What kind of advice would you give to those interested in getting into archival work, both in the Seattle area and farther afield?

Anne: We see a lot of volunteers at SMA. Archives work is very specialized and I think to find work you need to be somewhat flexible. I would advise people to volunteer different places to find the work you like, seeking paid internships where possible. There is a need for people with experience in digital assets, both born digital and digitized, so I would encourage people to gain skills in this arena. Archivists are great people! They are your biggest asset. Everyone brings something different to the profession. Take every opportunity to meet and get to know your future colleagues!

Ari: Outside the office, I know you are a fan of poetry and creative writing. Can you share something that has moved you recently?

Anne: I was with a group of writing friends this summer and one of them wrote a poem and found out it was accepted in Rattle all in the same week. She recorded it on my phone and the audio recording of her reading it is there as well. It is by Susan Vespoli. It moved me deeply.

Susan Vespoli

MY SON NO LONGER MISSING

I like to think he graduated from the methadone clinic, now does yoga, gave up smoking. I like to think he grew a new set of bright teeth to replace the ones that rotted. I like to think he rents a studio with a patio near the canal filled with crappies and sunfish not nodding off with homeless junkies. I like to think he leans back in an Adirondack, after loading the dishwasher with cupcake pans from birthday muffins like the ones he baked for me topped with candles that he brought to the Mex place where he hired a trio of sequined mariachis to serenade us as we dined on cheese enchiladas. I like to think he is waiting for just the right minute of the right hour of the right day to reappear to tell me he is living free of pills and booze and meth and smack and at the end of each long hot Phoenix day, he drops himself into the cool blue complex pool, then emerges shiny, dripping.

[mipops]

Hannah Palin x MIPoPS!

Next in our series of profiles on the people who make MIPoPS tick, we’re talking to Hannah Palin, Moving Image Curator at the University of Washington Libraries’ Special Collections. Hannah is the life of the party. She has the je ne sais quoi usually associated with actors and celebrities - always two seconds away from putting on her next character, her next accent, her next story that’ll have you rolling on the floor with tears streaming down your cheeks. She is magnetic and magnanimus. She’s also on MIPoPS Board of Directors, AND directly responsible for much of the moving image preservation infrastructure here in Washington! Hannah has been with MIPoPS from the very beginning, and we are exceptionally lucky to have her.

Hannah at one of MIPoPS Moving History screenings at the Northwest Film Forum. Ari Lavigne (AL): HIYA! Who are you? What’s your av backstory? What do you do for work? Hannah Palin (HP): Hi back atcha! At one point in my evolution, I thought I decided to explore documentary filmmaking. I had some stories to tell and wanted to delve into film and audio production as a way to get my ideas out into the world. I needed a day job and found myself at a small company that transferred film to videotape. I spent five years sitting in the dark, watching home movies, educating our customers on what kind of film they had in their shoebox and how best to take care of it. In 2001, at the urging of a local filmmaker, I attended the Association of Moving Image Archivists conference in Portland and discovered that there was whole profession devoted to sitting in the dark, watching movies, and educating people on how to preserve their moving image materials. I met Nicolette Bromberg, Visual Materials Curator at the UW Libraries Special Collections, and she gave me her elevator pitch for starting a regional film archive. I was hooked. I started volunteering, then working on grant projects, then they finally gave me a job because I just wouldn’t go away. I’ve been working on building the moving image collections at the UW ever since. AL: How did you come to be involved with MIPoPS? HP: Rachel, MIPoPS Executive Director, and I met during the Washington Film Preservation Project in 2006. The UW was awarded a preservation grant from the Washington Secretary of State’s office to work with nine other institutions in the region to help them work on their film collections, including MOHAI, the Museum of Flight, the Yakama Nation, and Seattle Municipal Archives, among others. Rachel was working with Seattle Municipal Archives at the time, so we got to know each other a bit then. A few years later, we spent time at an Association of Moving Image Archivist conference in Savannah, which led to grabbing coffee back home, which led to conversations about creating a regional film archive, which eventually led to the birth of MIPoPS and the rest is Moving Image History. Because of my responsibilities at the UW, I’ve stepped into the role of Board Member, but I’d love to come back and play more soon! AL: You are a tremendously creative person. You’ve worked on both video and audio documentaries and even been featured on RadioLab - twice! If you had all the money and all the free time in the world, what creative projects might we find you tinkering away at?

Click here to listen to "The Day My Mother's Head Exploded", a 2004 piece by Hannah. HP: Good question! At this point, I feel like a lot of my creative juices are funneled into my job at the UW, but I’ve got another project that’s beginning to take shape. My biological father was a painter and I’ve become the caretaker of his collection, which includes 400 paintings and works on paper (https://www.kennethshowell.com/). I’d like to use my curatorial skills to get his work out into the public sphere, but I think there’s also a story in there somewhere that is just waiting to take shape. One of these days! AL: You’ve personally had a hand in training or otherwise fostering the development of most of the MIPoPS staff (Libby, Andrew Weaver, and I all worked with you at UW). How important is sharing your skill set with newbies in the field? Do you have a philosophy of mentorship / teaching / sharing? HP: I definitely feel an obligation to help the next generation of folks coming up because there just aren’t a lot of opportunities to learn about moving image preservation or archives in the Pacific Northwest. So, if you can’t get to NYU, UCLA, or Eastman House (which I couldn’t do either) come hang with me and I’ll tell you what I know. I learn as much from working with students as I hope they do from me. Every collection and project is so different that there’s always something new to figure out and to do. It’s a collaboration and I’m always thrilled when folks pick up the baton and move the profession forward. My mission is to preach the gospel of moving image preservation so that we can preserve the visual heritage of this region. When I started I could say that I was the only person in the state of Washington paid to work full-time on moving image collections. Now I’m thrilled to say that that isn’t the case. FIlm and videotape are being taken more seriously by cultural institutions across the state and there’s definitely a shift in thinking about them as primary source materials that deserve care and attention. AL: What areas do you think up and coming AV archivists should be exploring?

Hannah Palin and Libby Burke working on the winds at UW. Photo: Meryl Schenker, Seattle Post-Intelligencer, 2004. HP: Even though the digital revolution has pushed the field towards bits and bytes, I think that there is a need for everyone to be a jack-of-all-trades. Maybe it’s just me, but most of us are going to be Lone Arrangers, the sole voice of expertise in any organization, and you’ve got to be able to do it all. On any given day, I might meet with a donor, work with a student on their capstone project, repair a Umatic tape or clean a deck, meet with Advancement to discuss a grant opportunity, accession a new collection using ArchiveSpace, and use bagit to prep files for storage. You’re a specialist and a generalist all the time. It can be overwhelming, but it’s imperative to know it all. AL: What material have you seen lately that really moved you? HP: We recently acquired the KIRO-TV Channel 7 News library with 15,000 tapes from 1975-2006. Last month I discovered a 14 minute film from a program produced for KIRO’s first anniversary that aired in February 1959. It’s basically a day-in-the-life of a television news station and it shares with the audience how the broadcasters spend their days getting stories on the air. Like many TV stations across the county, KIRO discarded their 16mm film library when they switched to videotape in 1975, so this film is like finding a time capsule of what local news was like at the dawn of television. I’m thrilled that it turned up, I’m happy that we had the funds to be able to get a 2K scan at Light Press, and I’m ecstatic that I get to share it with students, researchers, and producers.

AL: Is there anything in your orbit that you’d like to share? It doesn’t have to be av related! HP: It is incredibly satisfying to see how far MIPoPS has come from its beginnings, how much it has grown because of the hard work that everyone has put into the organization (you, Libby, Andrew, and Rachel deserve kudos for all that you’ve done), and, most of all, how deep an impact it is having on our region. Amazing!!! Can’t wait to see what’s next!

[talesofdrunkennessandcruelty]

A nonprofit battles the ‘magnetic media crisis’ by digitizing aging movies before they vanish