Encode videos from your browser with Jupyter Notebook

Today I will show you how Jupyter Notebook can make video encoding accessible (right in your browser), interactive (code/script/visualization in one place), and fun!

What is Jupyter Notebook?

The Jupyter Notebook is a powerful web application that provides a sharable work space with live code (Python, Shell, JavaScript, HTML…), data visualization, and documentation in one place.

It’s been widely adopted by data science and machine learning communities. People use it for prototyping, collaboration, project demo, data analysis and processing, and so much more.

What do you need to start using Jupyter Notebook? Just a browser.

No registration, no installation, no fee! Visit https://jupyter.org/ to find out more.

Why Jupyter Notebook + video encoder?

It may not feel natural or necessary to do video encoding in a Jupyer Notebook at first. But it actually totally makes sense and here is why:

1. Launching a Jupyter Notebook via Binder essentially gives you an on-demand Linux Docker container connected to your browser with all regular user rights. This point alone makes me excited.
2. In particular, you can run FFmpeg with arbitrary command line options in a Jupyter Notebook using a static build you downloaded or built with custom configuration. Yes, running FFmpeg from a Web app without writing and hosting your own server or WebAssembly! Isn’t that “dreams come true”?
3. To make it sweeter: display the encoded video via HTML <video>; visualize PSNR/SSIM/VMAF or any metrics you care about with versatile interactive display widgets; image processing with OpenCV; camera video recording with JavaScript/WebRTC… The limit is truly our imagination.

Walk through a notebook for video encoding

Demo time!

First, launch a running instance of the notebook through this link. It may take a couple of minutes to finish setup since the instances are created on demand.

Open the ffmpeg-demo-compression.ipynb file:

Select “Run all” from “Cell” menu:

Wait for all the cells to finish running. A cell is running or pending if “*” is shown on the left side of the cell.

What’s going on here?

The notebook contains a sequence of “cells”. A cell is a unit of execution with Shell/Python/special purpose commands.

In our notebook, the first 3 cells setup the binary, probe input video information, and run FFmpeg to encode the video and create an output video on the server:

The first cell calls another notebook “util/load-ffmpeg.ipynb”, which is like this:

util/load-ffmpeg.ipynb

It downloads a static build of FFmpeg to the server, unzips it, and adds the directory to the environment variable “PATH”. We cannot install FFmpeg through a package repository because “sudo” access is disallowed in the notebook for good security reason.

The second cell runs ffprobe to print media stream information in “input.mp4”:

ffprobe -hide_banner input.mp4

The third cell runs ffmpeg to compress input.mp4:

ffmpeg -hide_banner -i input.mp4 -b:v 200k -y output.mp4

“-hide_banner” tells ffmpeg to not print a banner containing version and build information.

“-i input.mp4” specifies the input media file.

“-b:v 200k” specifies the desired output video bitrate is 200 kb/s.

“-y” dictates ffmpeg to overwrite output.mp4 if it already exists.

This command compresses the video stream with the default encoder “libx264” at 200 kb/s target bitrate and writes the output video to “output.mp4”.

The next cell display a comparison of the source video and the compressed video by calling another helper notebook “util/play-video.ipynb”, which renders the video file using HTML5 <video> elements and adds additions buttons to play/pause the two videos in sync:

Lastly we calculate some video metrics using ffmpeg filters, for PSNR, SSIM, and VMAF respectively:

Customize it!

You can play with the command line options and re-run each cell individually and repeatedly.

To try another encoding setting, e.g. switch to a different rate control method or target bitrate, modify the FFmpeg command in cell 3, re-run cell 3–5, and see the new visual comparison and quality metrics.

For example, the following command will encode the video with HEVC:

!ffmpeg -hide_banner -i input.mp4 -c:v libx265 -b:v 200k -y output-265.mp4

Modify the file names in cell 6 to compare the H.264 output and HEVC output like this:

%run util/play-video.ipynb --ref output-265.mp4 --tar output.mp4

You can see that at the same bitrate, the HEVC video (top) is much sharper and cleaner than the H.264 video (bottom):

To try it on a different input, you can upload a media file to the server, replace “input.mp4” with the new file name through the notebook, and re-run cell 3–5:

Instead of uploading a local file, you can also download an Internet video by calling “curl” on the server, e.g. adding a cell with this line and run:

!curl https://url/to/video.mp4 -o downloaded.mp4

Any file on the server, including any FFmpeg output, can be downloaded locally, which means the notebook can serve as an online video format converter or transcoder:

What else can we do?

This is a very basic demonstration of how to use a Jupyter Notebook for video encoding, or more generally speaking running FFmpeg with arbitrary input and options .

It’s nothing fancy, but unlocks unlimited possibilities for media processing and analysis with simple modification to the FFmpeg command. Next time I’ll show you another demo with more in depth video stream analysis. Combining FFmpeg with OpenCV/WebRTC for more advanced processing and live camera recording would be interesting too.

Lastly, all my notebooks are open sourced on Github and can be launched with one click on the Binder icon. I’ll keep adding new ones as I discover cool demo ideas. Comments/ideas/demo requests are welcome!

About the author

I’m a full stack software engineer and entrepreneur with 10+ years of development and leadership experience in WebRTC, video encoding, mobile and Web technologies, and media centric machine learning algorithms. Previously I served as the CTO of Visionular and spent years at Google and Microsoft developing multiple consumer facing applications and large scale systems that serve billions of users. When I’m not coding, I like writing, reading, and learning random new skills.