Using Open Source Tools to Capture Closed Captions and Timecode

Today’s guest post is from Morgan Morel of the National Audio-Visual Conservation Center at the Library of Congress.

Vrecord is open source video capture software used by video preservation labs across the nation. The idea for vrecord was born from an Association of Moving Image Archivists (AMIA) Hack Day project to see if a Bash script could be written to use media company Blackmagic’s open source Software Development Kit to capture a digitized video signal. In the 12 years since the first version in July 2014, vrecord has expanded its feature set significantly to meet the needs of video preservationists. The latest version of vrecord can be found in the vrecord release tracker. One of the most notable features is that it can natively capture and encode FFV1 Matroska files, a format which was added to the list of “preferred formats” in the Library of Congress’ Recommended Formats Statement in 2023 (for more information on that addition, see our blog post from December 2023). Since then, the Video Lab at the Library’s National Audio-Visual Conservation Center (NAVCC) in Culpeper, Virginia has created FFV1 Matroska preservation files from digitized video content.

Until recently, NAVCC has not been able to capture all tapes into FFV1 Matroska due to some of vrecord’s limitations. The Federal Agencies Digital Guidelines Initiative (FADGI) Audio-Visual Working Group funded a number of updates to vrecord in order to close these gaps so that NAVCC, as well as other federal institutions and other archival organizations nationwide, could deploy vrecord in their digitization workflows. These updates were performed by developers Dave Rice and Jérôme Martinez. As the NAVCC Video Lab Supervisor, I was brought on board during this development project to oversee and test the updates being made to vrecord. All these updates were tested in the NAVCC Video Lab using various video tape formats from our collection. This blog post describes the challenges faced during this process, as well as the results of these updates.

Capturing More Than Just Video

For years now, video capture systems software, including vrecord, have been capable of digitizing and capturing the video and audio essences contained on video tapes. While some nuance is required when dealing with certain details like Pixel Aspect Ratio (PAR) or the exact formats being captured, the conversation around capture of archival video in a 10-bit YUV format has been generally regarded as a closed topic among preservationists. For a deep dive into the topic of Display Aspect Ratio, see A Quick Guide to Digital Video Resolution and Aspect Ratio Conversions.

However, video tapes contain more than just audio and video. In fact, there can be a wide range of additional information and metadata streams contained on video tapes that may not be immediately apparent without the right tools. The NAVCC Video Lab has been focused on adopting tools to more reliably and consistently capture two of these streams: closed captions and timecode.

Closed Captions

Closed captions (CC) are text captions that describe the video content on screen, often the dialog or descriptions of sound effects, which can be turned on and off by the viewer at will. CC was invented in the early 1970s and started to be included on regularly-scheduled broadcasts by the 1980s. In the mid-1990s, the Telecommunications Act made CC required on the vast majority of distributed and broadcast video programming. There were some exceptions, including public service announcements shorter than 10 minutes, but for the most part all materials seen on television or purchased on video tape were required to have CC. This means that nearly all commercial tapes produced after 1996 should have CC data on them, which in most cases amounts to transcriptions of the spoken content of the tapes. The ability to turn this CC data into a transcript gives archives the ability to have written transcripts of media content, increasing content discoverability at scale. An in-depth discussion of Closed Captions can be found in FADGI’s Definitions for Key Accessibility Features for Digital Audiovisual Collections Content.

CC data is, thankfully, quite easy to capture. When CC information is created, it’s encoded as sequential digital hex values that a computer can read with a high degree of efficiency and accuracy. This information is captured digitally, then encoded into the analog video signal. This appears visually as blinking white dots and lines above the main video content. While broadcasted content usually crops out much of this information visually, best practices for archival capture of video signals necessitate including 486 lines of the 525 scan lines in National Television System Committee (NTSC) video, which includes the line 21, or EIA-608, CC data. This means that most archives have been capturing any CC data on their tapes, at least as part of the video signal.

However, simply capturing this data is not enough. It needs to be decoded back into the original timed text values in order to be useful. There are a few tools available, including some open source tools, that can decode this visual information into a Scenarist Closed Caption (SCC) file. A SCC file contains the “raw” information: timestamps and computer-readable hexadecimal values of the text information. The last step is to transform this data into the closed captioning text a viewer might see and read, by converting the SCC files into human-readable subtitle formats, such as SRT and WebVTT. These formats can be used by web players and TVs to display subtitles, or they make for handy transcript data that assists in accessibility and discovery of captured content. CC is one example of non-programmatic data embedded within analog video signals that can provide valuable contextual information about a recording.

Timecode

Timecode is another important category of embedded information found on many video tapes. Timecode refers to a method by which individual frames of an audio or video program are labeled with a timestamp. Timecode is typically used in editing, production and broadcast context, and its implementation was standardized by Society of Motion Picture and Television Engineers (SMPTE) as the SMPTE-12M standard. The purpose or meaning of timecode for a specific program can vary. The timecode on a tape can refer to many types of timing information: recording, editing, and broadcast time are all common use cases. Specific clips within the program can even contain their own timing information, discontinuous from adjacent clips.

Additionally, timecode can be stored in different parts of the program information that video engineers capture, which creates a complex process of discovery when choosing information to store in a preservation file or note during media capture. It’s possible to store the timecode as a linear audio stream using an audio channel on the tape. It’s very common for SMPTE 1” Type C tapes to have the third audio channel contain linear time code (LTC), and also not uncommon to find LTC on the second audio channel of U-matic tapes. It’s also possible for timecode to be stored in the vertical ancillary (VANC) portion of analog video signals. Timecode in the VANC is referred to as vertical interval timecode (VITC). Additionally, both LTC and VITC timecodes can be embedded into a digitized version of a video and audio signal being transmitted using a serial digital interface (SDI). Tapes can contain multiple LTC and VITC streams, and they can all have different information on them.

There are a number of good reasons for a tape or video stream to contain multiple timecode streams. The various streams may all contain the same timecode as a matter of redundancy, in the case that one of the streams is damaged or unreadable. However, it’s also possible for the different streams to represent different information. For example, a video stream made up of various on-location camera shots may contain one discontinuous timecode stream representing the original time each shot was captured, as well as another continuous timecode stream representing when the final edit of the program was printed to the tape. Or, if an international program like the Olympics is being broadcast, that program may contain a different timecode stream representing the broadcast time in each time zone the program is available in.

Regardless of the reason why any given program may contain various timecode streams, archivists need to be able to capture and store any and all timecode streams recorded onto the tapes being digitized. While some readily available capture software claims to be able to capture timecode, there are many cases in which this software mimics timecode behavior rather than correctly capturing timecode information. In these cases, the software captures the timestamp of the first frame of timecode, then simply counts upwards contiguously as the video plays, incrementing the timecode by a single frame for each frame of video captured. In some cases the resulting timecode may be correct, but if for any reason the timecode was recorded to the source tape discontinuously, this would not be reflected in the captured timecode. The prevalence of these sorts of issues made it clear that robust and transparent tools for timecode capture are needed in the archival ecosystem.

Updates to vrecord

Over the past two years FADGI has funded a number of updates to FFmpeg and vrecord that, along with many other useful features, made the capture of CC and timecode data possible. As is the case with many open source projects, these updates were built upon many previous advancements and iterations. In fact, the updates to both CC and Timecode capture in vrecord are indebted to work that was previously funded by The New York Public Library. You can read more about open source work that the NYPL sponsored, on their site Sponsoring Open Source AV Preservation Tools at NYPL | The New York Public Library.

Closed Captions Capture and Decoding

The latest version of vrecord has added a number of features related to capturing and processing the CC data held on line 21 of video signals. A simple and important improvement is that line 21 data can now be captured and stored as an SCC file. As mentioned previously, this file contains the CC data as a series of timestamps and machine-readable hex data.

Once the capture is complete, vrecord will then decode this SCC into an SRT and/or WebVTT file, depending on the user-defined options.

SRT and WebVTT are both commonly used formats for storing subtitles in a human-readable format. The advantage of using WebVTT in an archival context is that FADGI released Guidelines for Embedded Metadata in WebVTT Files. Vrecord will automatically add a few lines of metadata to the beginning of every WebVTT file it creates. This metadata includes information about the creation of the file, and its relation to the original source tape and the file with which the captions are associated. The user can also add their own lines, as well as a Media Identifier Type in order to contextualize the Media Identifier field using the vrecord interface. Below is an image of fields that users can fill to manipulate the WebVTT metadata, and an image of that metadata inside of a WebVTT file. The WebVTT file pictured contains the same caption data as the SCC file shown earlier.

Vrecord also added support for viewing decoded captions in real time before capture. Vrecord has always allowed the user to view the video coming into the computer using “passthru” mode. In passthru mode the video is not being recorded, but the user can use any number of passthru tools to analyze and adjust the incoming video signal before starting the recording. The latest version includes a Caption view that shows a historical readout of the video’s line 21 data, along with the decoded caption data in real time. The real-time decoding turns line 21 data directly into readable text on the screen. This is a useful tool to determine whether a video tape contains captions, and whether those captions are being properly captured and decoded by vrecord. Real-time decoding is particularly processor intensive so, in an attempt to reduce potential capture errors, the real-time decoding is only available in passthru mode.

There is one other update related to CC, which will be discussed in the segment about access files.

Timecode Embedding in Matroska

Support for timecode capture in vrecord required making updates to both FFmpeg and the Matroska specification, which made the update more complex than initially imagined. FFmpeg is open-source video encoding software. The FFV1 format is natively supported by FFmpeg, and thus FFmpeg is the tool of choice for creating, editing, and transcoding FFV1 files. Most of the video recording and encoding processes performed by vrecord are actually performed by FFmpeg under the hood.

At the start of this development cycle the user was able to select a single timecode source (VITC, LTC, or others) for capture. If present, this timecode source would then be captured as a sidecar text file, which would contain the timecode timestamp and map it to the appropriate timestamp in the file. While somewhat primitive, this proved that vrecord could capture timecode, even discontinuous timecode, accurately from a number of source streams.

The developer initially planned to embed the timecode captured as the sidecar text file as a post-capture process. This meant that media capture would happen normally, then vrecord would ask the user to wait patiently while the preservation Matroska file would be rewritten to include the captured timecode. However, the developer came up with a more elegant solution, in which updates to FFmpeg allow vrecord to capture and embed the timecode stream as the tape is being captured. This did create a new wrinkle, however: FFmpeg is a very large open-source project; updates to the code-base are subject to lengthy review and discussion. To facilitate a more expedited workflow in a large project, the developer deployed a fork, or local instance, of the FFmpeg code-base called FFmpeg-MA. This fork, maintained by Media Area, includes a number of updates to FFmpeg that support archival workflows. This update allowed for not only the concurrent capture of timecode, but also the capture of multiple timecode streams, meaning that every timecode stream contained on a tape could be preserved during the digitization process.

In order for these updates in FFmpeg to work, the Matroska specification also had to be updated. In 2023, the Matroska spec had been updated to handle complex timecode; in fact, this update was critical in the inclusion of FFV1 Matroska as a “preferred format” for Moving Image works in the Library’s Recommended Format Statement. However, the nuances of the implementation still hadn’t been fully worked out. The vrecord developer worked with the Matroska team to suggest a naming convention for tagging timecode streams that covers any potential timecode stream. These tags include all possible sources for timecode on video, as well as timecode coming from Digital Audio Tape (DAT) to support the capture of DAT audio to Matroska wrapped Free Lossless Audio Codec (FLAC) files.

With these updates in place, vrecord will now automatically capture all timecode streams it receives from a video tape automatically while creating Matroska preservation files.

Access File Updates: Concurrent Creation and Embedded Captions

Until recently vrecord only allowed users to create one video file at a time, which would generally be the preservation file. The latest version of vrecord allows users to create MP4 Access files concurrently with the recording of the preservation file. An additional post-processing step can be turned on to embed any decoded CC data directly into the Access MP4 as a subtitle stream. This simplifies the accessibility of caption data, obviating the need to ensure that the sidecar caption file and MP4 are located side-by-side. We hope that this feature will give archival institutions a simple solution for providing their patrons with increased accessibility options for video content containing CC data.

HD Capture

Vrecord was originally designed to only capture Standard Definition (SD) video content due to the fact that the vast majority of video tapes contain only SD content. There are only a handful of tape formats that are capable of containing High Definition (HD) content. The most common of these formats are DVCPRO HD, HDCAM and HDCAM SR, though there are some other very rare formats that are HD compatible. These HD video tape formats only existed for a short time before video tape technology became obsolete, and as a result there is only a small amount of archival material contained on these formats. Vrecord development has historically focused almost entirely on handling SD content in order to focus on formats that make up the majority of archival collections. However, the march of time continues, even tape formats that were considered more “modern” are being targeted for digitization and migration. As a result, FADGI funded the development to add HD functionality to the software.

The addition of HD support in vrecord added an additional layer of complication due to the complex nature of HD video formats. Within the vrecord interface the user is given the option to capture SD content as either “NTSC” or “PAL.” These choices refer to the two most common broadcast standards historically recorded to video tape. National Television System Committee (NTSC) video is made up of 525 lines at 29.97 frames per second, and Phase Alternating Line (PAL) is made up of 625 lines at 25 frames. HD content is more complicated, there are a number of different frame resolutions, frame rates, and scan types, and a number of permutations therein. The following 11 HD formats have been added to vrecord to support the vast majority of–but not necessarily all–video recordings recorded to tape:

• 1080p23.98 – 1920×1080, progressive, 23.98 frames per second
• 1080p24 – 1920×1080, progressive, 24 frames per second
• 1080p25 – 1920×1080, progressive, 25 frames per second
• 1080p29.97 – 1920×1080, progressive, 29.97 frames per second
• 1080p30 – 1920×1080, progressive, 30 frames per second
• 1080i50 – 1920×1080, interlaced, 50 fields per second
• 1080i59.94 – 1920×1080, interlaced, 59.94 fields per second
• 1080i60 – 1920×1080, interlaced, 60 fields per second
• 720p50 – 1280×720, progressive, 50 frames per second
• 720p59.94 – 1280×720, progressive, 59.94 frames per second
• 720p60 – 1280×720, progressive, 60 frames per second

There was initially concern that listing these additional formats would clutter the interface. To get around this a checkbox was added that makes these HD formats visible. This keeps the interface simple for users planning to only capture SD formats.

Package Installer

Until now vrecord could only be installed using command line tools, either installing from the source or by using the homebrew package installer. Installation via homebrew is a relatively simple process, but the lack of a package installer has been a barrier to entry for some archival organizations. Many government and large institutional organizations have IT security protocols that require the review of installation files to ensure no malware is contained in software. Other organizations keep their digitization computers disconnected from the internet to avoid harm from malicious internet attacks. Without a package installer, vrecord could not be installed and used at institutions with these IT barriers. As of March 2026, major releases of vrecord will include package installers for simple installation, which can be found on the vrecord release tracker on GitHub. The version being used in the NAVCC Video Lab is 2026-03-03, which contains all of the updates discussed in the article.

Conclusion: Expanding the Capabilities of Open Source Video Preservation Tools

The updates described here significantly expand the capabilities of vrecord as a tool for archival video preservation. By enabling the capture of CC and multiple timecode streams, vrecord now allows archivists to preserve forms of information that have historically been difficult to capture reliably during digitization.

These developments reflect the broader collaborative nature of the audiovisual preservation community. Contributions from FADGI, MediaArea, the Matroska development community, and institutions such as the Library of Congress and the New York Public Library have helped move these tools forward. As archives continue to face the challenges of preserving video tape, ongoing collaboration and open source development will remain essential for ensuring that preservation workflows can meet the needs of the archival community.