#dvrescue

[mipops]

In case you missed the session at AMIA 2020, here is the full recorded DVRescue presentation. Presented by Ashley Blewer, Libby Hopfauf, Dave Rice and Andrew Weaver.

[mipops]

DVRescue

DV Rescue: Innovations in the Preservation of Digital Videotape Libby Hopfauf & Dave Rice LIBBY Hello everyone! Thanks for joining us. My name is Libby Hopfauf. I am the program manager and audiovisual archivist for Moving Image Preservation of Puget Sound (or MIPoPS) and the Audiovisual Archivis...

Today, Dave Rice and Libby Hopfauf presented on DVRescue at the Library of Congress Labs Informal A/V Summit. You can view the slide deck here. LoC Labs will post the full recording of the summit here in the next few months. 

[mipops]

Using DV Capture in vrecord

For those of you who haven’t tried it out yet, here’s a run-down on capturing DV tapes in vrecord!

Basic Instructions

To capture DV formats in vrecord (when installed in macOS), you will need to have your DV deck connected directly to your computer via FireWire input. Make sure your deck is in “Local” mode. At this time, vrecord is not equipped with tape control, so you will need to manually start and stop the tape just like with analog capture in vrecord.

To select the deck you would like to use, go into the configuration mode either by:

run vrecord -e

clicking on the "Edit Settings" button in the GUI (run vrecord -g)

Switch from the "Decklink" tab to the "DV" tab under "Input Options" at the top of the configuration window.

Select the name of the DV device you want to use from the list.

Specify the playback, sidecar, file naming, recording event and directory options as you normally would for vrecord. For details see Editing Settings.

Click "OK" to save.

You can then run passthrough and record modes the same as you would with analog videotape. Please note that if the timecode does not start at the very beginning of the tape, the record mode viewer will not open until the timecode is detected (as soon as the counter starts moving on your deck, the viewer window should pop-up).

To end the capture, exit out of the ffmplay viewer window by clicking the red “x” in the upper left hand corner. Please note that if your video ends on a freeze frame of video, you will need to either start rewinding the tape or terminate the terminal window in order to exit capture (otherwise, you will end up with the spinning rainbow wheel of doom). 

Tips & Tricks

Currently, some users are experiencing issues with capturing that results in dropped frames or presentation time stamp (PTS) discontinuities. Here are a few variables to test if you encounter this issue:

[mipops]

dvrescue: dvpackager

Background

Software design for tools that transfer DV from tape to file take one of two approaches:

Receives all the DV frames and writes them to disk as a DV stream.

The software receives a series of DV frames, analyzes them to understand their significant properties, and wraps those frames within a container format, such as mov.

Tools such as DVHSCap, Live Capture Plus, and dvgrab use the first method. The resulting frames are stored as a stream of DV frames. The result is very authentic to what was transmitted; however, there can be significant challenges to using this data. DV supports a variety of options within several characteristics. 

Context

Consider this scenario. A cameraperson recording an interview with a musician on MiniDV tape. The interviewee is frameds in a 4/3 aspect ratio and the audio is recorded onto two channels of audio with a 4/3 aspect ratio. During the recording, the cameraperson remembers that the interview is for a production with a 16/9 production, so adjusts the camera's settings and does a second take at 16/9. Following the interview, the cameraperson records a performance by the musician's band and adjusts the camera to record 4 channels of audio rather than 2 in order to capture more of the microphones. This DV tape could be transferred over FireWire to a single, continuous DV stream; however, the resulting DV file would have a variable aspect ratio, a variable sampling rate for the audio, and variable audio channel count. Some players could support some of these changes, for example VLC will adjust the framing as aspect ratio changes though QuickTime won't. The presentation will be inconsistent since the extent of this sort of variability is rarely supported.

Tools such as Final Cut Pro 7 use the second approach. The resulting QuickTime file contains the same DV frames that first method would result in, but stores them in a convenient container. However, when the incoming DV frames contain incoherency in the characteristics, either because the camera settings changed or the data is damaged, the software will generally stop the data transfer at those events. Software that was written to expect perfect DV frames may then start to break up a DV transmission into dozens of individual files with intermittent gaps or simply stop and give error messages.

dvrescue

In order to effectively preserve DV materials in a way that scales, we need to ensure that we can transfer all of the DV data in a manner that is verifiable and also produce audiovisual files that are well-described and interoperable. This requires a mix of the two approaches above. In the design of dvrescue, we use the first approach and initially just write all incoming DVdv streams into a file. This file is intended to be a digital copy of the data stream that the DV tape player produces and contains audio, video, camera metadata, metadata about frame characteristics and information on how well the reading of the tape went. After writing those DV frames to a file, we then analyze it with the dvrescue utility and produce a report on every incoherency in the file: including jumps in timecode, jumps in recording timestamps, or changes in properties like aspect ratio or audio channel count. With that resulting data, a subsequent script, called dvpackager, then uses that report and the original DV file, to selectively encapsulate every frame into an audiovisual container such as QuickTime or Matroska. There are options here, so a user could make one file for every time the camera had the start button hit (since all the recording metadata is preserved within the DV frames) or the DV frames could be divided only by changes in properties like aspect ratio or channel count.

The need for this sort of packaging is demonstrated by a recent example file shared with us by Morgan Morel at BAVC. Morgan transferred the DV data from a tape to a file using DVHSCap, which follows the most approach described above, writing the received DV frames into a file and that's about it. In QuickTime X, the file starts with about a second of grey frames and then cuts to a recording of an art installation. Some fan noise is heard in the background, but the audio is very choppy, like a mixture of noise and silence and the result is jarring to listen to. VLC creates a similar jarring presentation, where the video looks right, but the audio is painful.

This is a presentation of how QuickTime X decodes the audio of this DV stream. Wanna listen?

With dvrescue, we can figure out what is happening. The dvrescue utility will read the DV file and document events and changes that happen throughout the file.

Here we can see that at the beginning of the file, there are 19 frames that use a sampling rate of 48000 Hz, but then the subsequent frames use a sampling rate of 32000. The 19 frames at the beginning are the gray frames mentioned earlier, whereas the rest of the tape is a recording of an art installation. So QuickTime X is looking at those first frames and presuming that it's all at 48000 Hz, but then when it is decoding a DV frame that only has 32000 Hz of audio per second, that only fills ⅔ of a second, thus these large seconds of silence are interleave since the decode expects a certain number of audio samples per second, but isn't getting it.

For cases like this, we are working on a utility called dvpackager. It analyzes the stream to createget XML like (shown above) and then decides if and how to break that DV file into pieces to avoid the presentation issues that Morgan found.

After dvpackager does it's thing, we have two video files. One is very brief, just those gray frames and 48000 Hz audio. And then the recording of the art installation at 32000 Hz. When we unsplice these very different recordings from one another and handle them individually they both play well with their own unique characteristics. This approach uses the best of the two described above. We know that we got all the data, but by analyzing it and encapsulating all frames in the right way, we have content that is accessible, well-described, and ready to be sustained as digital files.

[mipops]

dvrescue: dvplay

Exciting new developments are underway for the DV Rescue project. Most recently, the quality control analyzer component “dvplay” is in the beta stages of testing. dvplay is a quality control player that supports visualization of uncorrected DV data, including categorizing DV transfer errors, illuminating when selective re-transfer may be most likely to offer improvements, and providing a method to easily distinguish authentic DV data from error concealment techniques and unconcealed damage. Currently, dvplay consists of three modes:

dvplay

A frame-by-frame playback of the video file created from a DV tape transfer. Artifacts are highlighted in yellow.

dvplay -x

Screenshot from QuickTime of same frame

JPEG created in dvplay -x mode

This mode runs a check on the video for frames containing artifacts. Each frame where an artifact was detected is saved (in the same directory as the video) with the damaged pixels highlighted in yellow as a JPEG with the timecode printed on the side of the image. The images can then be reviewed to determine if recapture is necessary and possible issues in the files (such as dropouts, headclog, transfer artifacts, camcorder recording artifacts, etc.)

dvplay -m

This mode alternatively highlights all normal video images in yellow and leaves the artifacts unmasked for review.

[mipops]

DV RESCUE at AMIA 2019!

Come hear about the exciting progress we've been making on DV RESCUE! MIPoPS' Program Manager Libby Hopfauf and CUNY TV Moving Image Archivist Dave Rice will be presenting at the Association of Moving Image Archivists  conference in Baltimore! Check them out on FRIDAY 11/15 from 3:30-4:30!

[mipops]

DV Rescue - Update #2

Notes from the field! This update comes to us from Dave Rice:

One of the early dilemmas to solve in this project is a simple: how do we do this? We have to decide what tools are going to be part of the stack of technologies that we use to build software to save DV and at the bottom of that stack are the tools we'd need to simply facilitate reading an incoming DV stream to a file. Obviously building tools to transfer and wrangle DV video from tapes into files is going to rely on some obsolete technology; however, the more we rely on abandoned technology the more difficult this project will be to sustain.

On Linux there’s the libiec61883 and libraw1394 software libraries which are what older dv transfer tools like dvgrab (R.I.P.) were based upon. FFmpeg, a tool for working with multimedia files, supports these libraries to be used with a DV deck as an input device, but these libraries are Linux only and investigations at adjusting them to work in other environments has been a series of headaches. Possibly we could move forward with making DV Rescue a Linux-only application; however, such a decision would likely isolate our tools from a majority of our target audience.

In examining how to get data from a DV deck into FFmpeg running on a Mac, Dan Dennedy suggested looking at FFmpeg's support of AVFoundation. AVFoundation is an audiovisual framework recently developed by Apple and is the basis of Final Cut X, QuickTime X and modern Apple audiovisual tools. FFmpeg supports AVFoundation but although it could recognize some types of attached cameras as inputs and screen recording, it wouldn't recognize a DV deck. However, one day we opened Apple's older QuickTime 7. In QuickTime 7, it could access the DV deck by going under the Preferences and selecting an input. Over in QuickTime X we saw that the behavior is similar, that QuickTime X which is built upon the newer AVFoundation could access the DV deck and use it for a recording. This helped us realize that perhaps AVFoundation was supporting DV. Finding this article on tape-based camera support in Final Cut X and this technical note about Apple's AVFoundation API helped confirm that it was Apple's intention to include support for DV tape formats in their newer AVFoundation framework; however FFmpeg's support of AVFoundation was not recognizing the DV input.

From here, we reached out to FFmpeg's maintainer for its AVFoundation support, Thilo Borgmann, and sent him a DV deck for testing. FFmpeg's support for AVFoundation focused on what AVFoundation refers to as AVMediaTypeVideo and AVMediaTypeAudio; however DV is not well-defined as either of these. DV is a single format that mixes together video, audio, camera metadata, captions, timecode, information about the tape reading, and other details. In a test by changing AVMediaTypeVideo to AVMediaTypeMuxed (referring to a data stream of various types of data multiplexed together) we could recognize but not copy or decode the DV. From here, Thilo updated FFmpeg to get video from a DV deck into FFmpeg; however the video coming through wasn't DV but it was already decoded, so while the audio and video were there other types of valuable data about the recording and transfer were lost. Digging deeper in Apple's documentation, we found that AVFoundation has certain options to access a "device native" data stream rather an already decoded version of that video. Thilo then expanded FFmpeg's AVFoundation support with this patch to add the necessary context and handling for FFmpeg to recognize, accept, and transfer along DV inputs.

Historical First Transfer of data from DV camera over Firewire to a Mac as an input to FFmpeg's avfoundation input device!

These experiments helped us find a suitable foundation that we can build DV Rescue upon. We're using FFmpeg throughout, but on Linux we're going to use libiec61883 to get DV data into the process while on Mac we can now use avfoundation to get DV data in. This is a positive outcome as rather than relying on Apple's older audiovisual technology, we can use AVFoundation which is what Apple's modern audiovisual tools are built upon, as we look ahead to sustainability of our work, this will enable us to keep DV Rescue running well longer into the future.

Bonus: Currently (7/8/19) if you build ffmpeg from the current source, then you can capture dv with ffmpeg on a mac using:

ffmpeg -f avfoundation -capture_raw_data 1 -i "DV-VCR" -c copy -f rawvideo out.dv

📼😎📼