Last few months we at Merlin Studio worked on a project that required transparent videos. Here is how we did it.Merlin Studio is a Creative Digital Studio that build magical web experiences.
The Post
How to use transparent video in your website
Converting the assets 🖼️
Our client delivered multiple image sequences at 12FPS.
Since we wanted to use videos for performance we used ffmpeg to convert our sequence into a webm video with vp9 encoding.
We used the following command for this:ffmpeg -framerate 12 -f image2 -i image-%03d.png -c:v libvpx-vp9 -pix_fmt yuva420p -vf scale=1920:-1 output.webm
This does the followingr 12 locks the framerate to 12 FPS (Our animation was generated from an image sequence that was 12 frames per second)f image2 forces the input or output file format. The format is normally auto detected for input files and guessed from the file extension for output files, so this option is not needed in most cases.image-%03d.png iterates the file names using the following pattern: image-000.png -> image-001.png -> image-002.png etc.c:v libvpx-vp9 sets the codec to vp9pix_fmt yuva420p sets the pixelformat to yuva420 which supports alpha channelvf scale=1920:-1 Re-scales the video to 1920 width using the exisiting aspect ratio (16:9)
This command generated a transparent video that is ready for the web.
It works on all browsers… except Safari.
Safari 🤨
As the modern developer knows, Safari is starting to lean towards IE behaviour. Currently, Safari doesn’t support webm with vp9 transparent video.
But luckily for us, there is a solution to our video problem. Apple developed their own codec called AppleProRes4444. Using this we can create transparent videos that work on Safari too, it just requires 2 steps.
First of all, we have to convert our video for Safari.ffmpeg -framerate 12 -f image2 -i image-%03d.png -c:v prores_ks -profile:v 4 -vendor apl0 -bits_per_mb 8000 -pix_fmt yuva444p10le output.mov
This creates a video suitable for Safari, the only problem is, the filesize will be much bigger compared to the webm.
This is what brings us to the next step, we have to compress the video to make it ready for the web.
We can do this in a few different ways:
After the compression, you should have a video thats around the same size (usually a bit bigger) as the other webm video.
Animated AVIF 🧩
Since the modern AVIF image format also supports animated images (like a GIF), we’ve also tried converting our webm to avif to see if the filesize could be smaller.
Because ffmpeg doesn’t support animated avif encoding out of the box we had to work with an CLI called avifenc. Since avifenc doesn’t support webm as an input we’ve had to use ffmpeg to convert the webm file to a .y4m and then pipe it to avifenc.
We did the following:ffmpeg -i input.webm -c:v libvpx-vp9 -pix_fmt yuva444p -strict -1 -f yuv4mpegpipe - | avifenc --stdin output.avif
This outputs transparent animated avif which is suitable for the web. Unfortunately, the file size of the avif was a bit larger so we were better of using webm.
Fallback 🍂
Since not every browser version supports webm with vp9 codec it’s important to check if the support is there.
You can do this with the following code:
If the support is not there, we can use something like an image or another video to fall back on.const isVP9Supported = MediaRecorder.isTypeSupported('video/webm; codecs=vp9')
// Returns vp9 codec support true/false
Conclusion 📚
When making transparent videos for the web it important to check every browser for support and have a potential fallback. Finally, make sure to compress the video to have the smallest file size possible.
Resources 🔗
Yuri Artiukh is an amazing voice in the creative coding community.
This video nicely explains the current possibilities and problems regarding transparent video.
