FFmpeg is a robust open-source framework designed for command-line-based processing of video and audio files, and widely used for format transcoding, basic editing (trimming and concatenation), video scaling, video post-production effects, and standards compliance.
To date, it remains one of the commonly used solutions for processing videos on Ruby on Rails. Frei0r, in turn, is an extremely simple plugin API that combines the most common video effects into simple filters, sources and mixers that can be controlled by parameters.
I’ve prepared an extensive guide on video processing with FFmpeg. In this guide, we’ll start with the basics – how to set up a project, ending up with controlling system resource usage. You can find the project repository on GitHub.
Getting ready
Step 1. Install FFmpeg with all options
If you’re running macOS, you can use Homebrew when installing FFmpeg:
brew install ffmpeg --with-fdk-aac --with-frei0r --with-libvo-aacenc --with-libvorbis --with-libvpx --with-opencore-amr --with-openjpeg --with-opus --with-schroedinger --with-theora --with-toolsAll of the additional parameters I added to the code above might not be needed for your current project. But I’ve included almost all possible parameters to avoid issues when something doesn’t work because some extension isn’t installed.
Step 2.Fix possible issues with the Frei0r plugin
When I installed FFmpeg with Frei0r, the Frei0r effects didn’t work at first. To check if you have such an issue, run FFmpeg and try to add Frei0r effects to a video:
ffmpeg -v debug -i 1.mp4 -vf frei0r=glow:0.5 output.mpgWhen you run that command, you might see an error like this:
[Parsed_frei0r_0 @ 0x7fe7834196a0] Looking for frei0r effect in '/Users/user/.frei0r-1/lib/glow.dylib' [Parsed_frei0r_0 @ 0x7fe7834196a0] Looking for frei0r effect in '/usr/local/lib/frei0r-1/glow.dylib' [Parsed_frei0r_0 @ 0x7fe7834196a0] Looking for frei0r effect in '/usr/lib/frei0r-1/glow.dylib'If you run the ls -l /usr/local/lib/frei0r-1/ command you’ll see that the plugins are installed with the .so extension.
To solve this problem, on my machine (macOS 10.12.5, ffmpeg 3.3.2, frei0r-1.6.1) I copied the .so extensions to .dlib:
for file in /usr/local/lib/frei0r-1/*.so ; do cp $file "${file%.*}.dylib" ; doneYou also have to set this environment variable with the path to the folder where the .dylib files are stored:
export FREI0R_PATH=/usr/local/Cellar/frei0r/1.6.1/lib/frei0r-1This solution looks like some strange hack. Nevertheless, I finally got Frei0r to work well.
Step 3. Install FFmpeg thumbnailer
We’re going to use the ffmpegthumbnailer utility to create video thumbnails. You can install this utility with brew install ffmpegthumbnailer.
Step 4. Install ImageMagick
If you decide to work with generated thumbnail images or with uploaded watermark images, then you’ll need to install imagemagick.
Step 5. Install MongoDB and Redis
You’ll need to install MongoDB (brew install mongodb) for your project’s database and Redis (brew install redis) storage for sidekiq job parameters and data.
Step 6. Make sure you have Ruby and Rails
I assume you already have rvm (ruby version manager), Ruby, and Rails installed, so we can likely skip this part. If you don’t have Ruby and Rails installed, install them now. Keep in mind that I’m using Ruby 2.3.0 and Rails 5.0.1 for this project.
Step 7. Install and configure all gems
Let’s go directly to creating our project. First, create a new Rails project without a default active record, since we’re going to use MongoDB for storage:
rails new video_manipulator --skip-active-recordNext, add all necessary gems to the Gemfile.
Video processing:
gem 'streamio-ffmpeg'
gem 'carrierwave-video', github: 'evgeniy-trebin/carrierwave-video'
gem 'carrierwave-video-thumbnailer',
github: '23shortstop/carrierwave-video-thumbnailer'
gem 'carrierwave_backgrounder', github: '23shortstop/carrierwave_backgrounder'
gem 'kaminari-mongoid', '~> 0.1.0'Data storage
gem 'mongoid', '~> 6.1.0'
gem 'carrierwave-mongoid', require: 'carrierwave/mongoid'Background jobs
gem 'sidekiq'Website styles
gem 'materialize-sass'The following gem will be used for JSON serialization:
gem 'active_model_serializers', '~> 0.10.0'As you can see, some of these gems are installed directly from github without rubygems. In general, it’s not good practice to install gems like this, but these particular gems include some fixes and recent changes, while the originals of these gems have last been updated a few years ago.
The materialize-sass gem is used for styles. You might use some other style bootstrap gem or even create your own HTML markup styling.
Note that due to compatibility with carrierwave_backgrounder, we’re using sidekiq workers, not modern active workers from Rails.
Therefore, inside config/initializers/carrierwave_backgrounder.rb, sidekiq is configured as the backend for video processing and uploading queues:
CarrierWave::Backgrounder.configure do |c|
c.backend :sidekiq, queue: :carrierwave
end
For the mongoid gem we just generate its generic configuration:
rails g mongoid:configThis is enough for our demo research app. Of course, for commercial apps you shouldn’t store a database configuration file in the project’s repository since it might contain some database connection credentials. The same goes for MongoDB, since by default it doesn’t have a password for database connections; in production you have to set up a password for MongoDB too.
Step 8. Generate a basic scaffold for videos
Since we’re just making a demo application, we’re not going to add authorization and users.
Let’s generate a scaffold for videos with this command:
rails generate scaffold Video title:string file:string file_tmp:string file_processing:boolean watermark_image:stringIn order to store and process files in the background, our model requires the following attributes:
file_tmp:string – stores uploaded files temporarily
file_processing:boolean – lets us check when processing has finished; updated by the carrierwave_backgrounder gemStep 9. Generate video uploader
To generate our video uploader, we need the command: rails generate uploader Video
Let’s include all necessary modules from previously installed gems into our newly generated VideoUploader:
# Store and process video in the background
include ::CarrierWave::Backgrounder::Delay
# Use carrierwave-video gem's methods here
include ::CarrierWave::Video
# Use carrierwave-video-thumbnailer gem's methods here
include ::CarrierWave::Video::ThumbnailerThen we’ll define processing parameters for video encoding:
PROCESSED_DEFAULTS = {
resolution: '500x400', # desired video resolution; by default it preserves height ratio preserve_aspect_ratio: :height.
video_codec: 'libx264', # H.264/MPEG-4 AVC video codec
constant_rate_factor: '30', # GOP Size
frame_rate: '25', # frame rate
audio_codec: 'aac', # AAC audio codec
audio_bitrate: '64k', # Audio bitrate
audio_sample_rate: '44100' # Audio sampling frequency
}.freezeYou can also define acceptable video file formats:
def extension_whitelist
%w[mov mp4 3gp mkv webm m4v avi]
endNext, you have to specify where uploaded files should be stored:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
endThe main part of VideoUploader is encoding operation. We don’t need to keep the original file, which is why processing is forced over the original file. So when processing is finished, the processed file will replace the original. This also ensures that thumbnails will be generated from the processed file.
During processing, we use the :watermark option from the carrierwave-video gem to overlay a user-provided image from the watermark_image field in the Video model.
Here’s the main bit of code:
process encode: [:mp4, PROCESSED_DEFAULTS]
def encode(format, opts = {})
encode_video(format, opts) do |_, params|
if model.watermark_image.path.present?
params[:watermark] ||= {}
params[:watermark][:path] = model.watermark_image.path
end
end
endIn our case, this code also uses the carrierwave-video-thumbnailer gem for generating video thumbnail images from the middle of the file. Note that there’s code that ensures the proper content type for generated thumbnails, since by default thevideo file’s content type and extension would be used.
Thumbnails are processed as a different version of the uploaded attachment with the following version definition:
version :thumb do
process thumbnail: [
{ format: 'png', quality: 10, size: 200, seek: '50%', logger: Rails.logger }
]
def full_filename(for_file)
png_name for_file, version_name
end
process :apply_png_content_type
end
def png_name(for_file, version_name)
%(#{version_name}_#{for_file.chomp(File.extname(for_file))}.png)
end
def apply_png_content_type(*)
file.instance_variable_set(:@content_type, 'image/png')
end
end
Inside the Video model we have to define all necessary attributes:
# file_tmp is used for temporarily saving files while
# they’re processed and stored in the background by carrierwave_backgrounder gem
field :file_tmp, type: String
# file_processing attribute is managed by carrierwave_backgrounder
# it contains an indicator of uploaded file state: being processed or not
field :file_processing, type: BooleanWe also have to define mount_uploader for the file attribute that uses VideoUploader:
# mount_on is specified here because without it the gem
# would name a filename attribute as file_filename
# In some cases this is logical, but in our case it’s strange to have
# file_filename attribute inside video table
mount_uploader :file, ::VideoUploader, mount_on: :fileHere we specify that we want our attachment to be processed and stored in the background:
process_in_background :file
store_in_background :file
validates_presence_of :fileThis is especially useful when the file is processed and stored in some external storage like Amazon S3. Uploading and working with it can take a long time, so it’s good to let them run in the background because otherwise users will have to wait until these tasks have finished, which also might exceed the server request timeout.
We also have to add an uploader mount for a watermark image:
# Same as above. We don’t want to have watermark_image_filename attribute
mount_uploader :watermark_image, ::ImageUploader, mount_on: :watermark_imageLet’s add the uploader for our watermark_image. To do so, run the command rails generate uploader Image. Here’s our image uploader:
class ImageUploader < CarrierWave::Uploader::Base
storage :file
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
def extension_whitelist
%w[jpg jpeg gif png]
end
end
Don’t forget to add the sidekiq configuration to config/sidekiq.yml:
:verbose: true
:pidfile: ./tmp/pids/sidekiq.pid
:logfile: ./log/sidekiq.log
:concurrency: 3
:queues:
- carrierwaveThe main part of the code we are going to focus on is the queues key, since carrierwave is the queue where all such tasks are stored.
We’re not going to cover the HTML views part. In most cases, the generated scaffold should be enough. The only thing that you might need to add is the watermark_image attribute in the video creation form.
This is what such from might look like without any additional markup:
<%= form_for(video) do |f| %>
<%= f.text_field :title, placeholder: 'Title' %>
<%= f.file_field :file, placeholder: 'Video File' %>
<%= f.file_field :watermark_image, placeholder: 'Watermark Image' %>
<%= f.submit %>
<% end %>Then in the video details view you can use the video_tag Rails helper method to display an HTML5 video player:
<%= video_tag(video.file.url, controls: true) if video.file.url.present? %>
Now you just have to run a background job:
bundle exec sidekiq -C config/sidekiq.ymland your server: rails s
That’s it! You have a simple video processing server that was set up in 15 minutes.
FFmpeg and Frei0r filter processing examples
FFmpeg is a video and audio converter that can also process live audio and video. Moreover, you can resize video on the fly with high-quality results. You can find more detailed information about FFmpeg in the FFmpeg documentation.
For a description of all available FFmpeg filters, check the filters section of the documentation.
Video and audio effects
FFmpeg with the Frei0r plugin is used for video effects. The options -vf and -af specify simple video and audio filters. The -filter_complex option specifies complex filters.
To avoid excessive audio processing, you should copy the audio stream unchanged from your input source file with the option -c:a copy when using all video filters.
For all audio effects, we can copy the video steam unchanged with the -c:v copy option.
You can find more details about stream copy here.
The -c:v copy option isn’t applied for reverse, slow down, and speed up effects because they alter both audio and video streams.
By default, FFmpeg selects the best input stream of each kind (video, audio, subtitles, etc.) Let’s assume that for our audio effects we want to convert all audio input streams, however (maybe we have the same stream in different languages, for example). You can do this with the -map 0 option.
All Frei0r plugin video filters are applied with the FFmpeg simple filter option -vf frei0r=filtername:param1_value|param2_value|param3_value
The official Frei0r page doesn’t contain any documentation on how this option should be used. That’s why I had to conduct my own research. During my research, I found a helpful resource that describes Frei0r filters along with some other plugins. With the help of this resource, I learned about all available filters and their parameters.
Here’s the original video that I used for my video effects processing demonstration:
And here’s the original video I used for my audio effects demonstration:
#1 Sepia effect
The complex colorchannelmixer filter adjusts input frames by remixing color channels. This filter changes the color temperature to sepia-like.
-filter_complex colorchannelmixer=.393:.769:.189:0:.349:.686:.168:0:.272:.534:.131
#2 Black and white effect
Zero saturation with the hue filter produces a black and white output video.
-vf hue=s=0
#3 Vertigo effect
The vertigo effect performs alpha blending with zoomed and rotated images. For my example, I set the phase increment parameter to 0.2 and kept the zoomrate parameter with its default value to modify the video.
-vf frei0r=vertigo:0.2
#4 Vignette effect
The vignette effect makes your film look like it was shot with a vignetting lens. I used the default settings.
-vf frei0r=vignette
#5 Sobel effect
The Sobel effect is used in image processing and computer vision, particularly within edge detection algorithms where it creates an image emphasizing edges. The Sobel effect doesn’t have any parameters you can change.
-vf frei0r=sobel
#6 Pixelizor effect
The pixelizor effect is also applied with default settings and creates a pixelated picture.
-vf frei0r=pixeliz0r
#7 Invert0r effect
The invert0r effect inverts all colors of a source image.
-vf frei0r=invert0r
#8 RGBnoise effect
The RGBnoise effect adds RGB noise to your video. It has one parameter that defines the amount of noise added. The result looks like a 90s movie played on an old VHS player.
-vf frei0r=rgbnoise:0.2
#9 Distorter effect
The distorter effect distorts image in a weird way. I set two parameters here: amplitude and frequency of the plasma signal.
-vf frei0r=distort0r:0.05|0.0000001
#10 IIRblur effect
IIRblur provides an Infinite Impulse Response Gaussian blur. I used this filter with its default settings.
-vf frei0r=iirblur
#11 Nervous effect
The nervous effect flushes frames in time in a nervous way; it doesn’t have any additional parameters.
-vf frei0r=nervous
#12 Glow effect
The glow effect creates a glamorous glow. I set the Blur of the glow parameter to its maximum value.
-vf frei0r=glow:1
#13 Reverse effect
The reverse effect is achieved with two FFmpeg filters – video reverse and audio reverse – running at the same time.
-vf reverse -af areverse
#14 Slow down effect
The slow down effect uses the setpts filter to double the Presentation Time Stamp (PTS) of the original film. As a result, you get a video that’s half the speed of the original.
The atempo filter is used to halve the speed of audio with the 0.5 (50% of the original tempo) setting.
-filter:v setpts=2.0*PTS -filter:a atempo=0.5
#15 Speed up effect
The speed up filter works with the same approach as slow down effect, only in the opposite direction. It takes half of the original video Presentation Time Stamp and makes it twice as fast.
-filter:v setpts=0.5*PTS -filter:a atempo=2.0
#16 Echo audio effect
The aecho audio filter reflects your audio stream to create an echo effect just as if you were in the mountains.
-af aecho=0.8:0.9:1000|500:0.7|0.5The first parameter is the input gain of the reflected signal (0.8). The second parameter is the output gain of the reflected signal (0.9). Next there’s a list of delays from the original signal in milliseconds, separated by the pipe | symbol. After this parameter goes the list of decays for each delay in the previous list. The colon : separates items in lists.
#17 Tremolo audio effect
The tremolo filter performs sinusoidal amplitude modulation.
-af tremolo=f=10.0:d=0.7The f parameter is for modulation frequency in Hertz. The d parameter shows the depth of modulation as a percentage.
#18 Vibrato Audio effect
The vibrato filter performs sinusoidal phase modulation.
-af vibrato=f=7.0:d=0.5The f and d parameters perform the same functions as in the tremolo filter.
#19 Chorus audio effect
The chorus filter resembles an echo effect with a short delay. The difference is that the delay in the echo filter is constant, whereas in the chorus filter it varies using sinusoidal or triangular modulation.
-af chorus=0.5:0.9:50|60|40:0.4|0.32|0.3:0.25|0.4|0.3:2|2.3|1.3The parameters in the chorus filter are ordered as follows: input gain, output gain, delays list in ms, decays list, speeds list, depths list. Parameters are separated by the pipe and items in lists are separated by the colon.
#20 Watermark
The watermark feature is implemented in the carrierwave-video gem. It uses FFmpeg’s overlay filter.
On the video below, you can see the combination of the vertigo, vignette, RGBnoise, distorter, glow, reverse, echo, tremolo, chorus, and watermark visual effects.
The watermark on this video is a transparent png image with the red text FFmpeg TV:
Generating thumbnails
Now we’re going to add the Thumbnail model, which is embedded in the Video model. Each Thumbnail record will be stored as one thumbnail image:
class Thumbnail
include Mongoid::Document
embedded_in :video
# Here background uploading is not used since this entity is created in
# background already
mount_uploader :file, ::ImageUploader
end
The next step is to turn back to the Video model again. To do so, we need to define the thumbnails association and the callback method to save our thumbnails:
embeds_many :thumbnails
# Config option: generate thumbnails for each second of video
field :needs_thumbnails, type: Boolean, default: false
# Callback method
def save_thumbnail_files(files_list)
files_list.each do |file_path|
::File.open(file_path, 'r') do |f|
thumbnails.create!(file: f)
end
end
endNext, let’s place all thumbnail creation logic in the CarrierWave::Extensions::VideoMultiThumbnailer module. This module will be designed to be compatible with the carrierwave-video gem’s logic.
This file is too large to be fully presented within the article. To see the full file, check out this repository.
Here’s how you can describe such a module:
module CarrierWave
module Extensions
module VideoMultiThumbnailer
def create_thumbnails_for_video(format, opts = {})
prepare_thumbnailing_parameters_by(format, opts)
# Create temporary directory where all created thumbnails will be saved
prepare_tmp_dir
run_thumbnails_transcoding
# Run callback for saving thumbnails
save_thumb_files
# Remove temporary data
remove_tmp_dir
end
private
def run_thumbnails_transcoding
with_trancoding_callbacks do
if @progress
@movie.screenshot(*screenshot_options) do |value|
@progress.call(value)
end
# It’s an ugly hack, but this operation returned this in the end
# 0.8597883597883599 that is not 1.0
@progress.call(1.0)
else
@movie.screenshot(*screenshot_options)
end
end
end
# Some method definitions are skipped here. Look in the repository for more details.
# Thumbnails are sorted by their creation date
# to put them in chronological order.
def thumb_file_paths_list
Dir["#{tmp_dir_path}/*.#{@options.format}"].sort_by do |filename|
File.mtime(filename)
end
end
def save_thumb_files
model.send(@options.raw[:save_thumbnail_files_method], thumb_file_paths_list)
end
def screenshot_options
[
"#{tmp_dir_path}/%d.#{@options.format}",
@options.format_params,
{
preserve_aspect_ratio: :width,
validate: false
}
]
end
end
end
end
The module I’ve described above uses the screenshot method directly from the streamio-ffmpeg
Also, since this operation is local, we have to create, use, and remove temporary directories for our generated thumbnails with the tmp_dir_path. All generated images are sent to the model’s callback, which is specified in the save_thumbnail_files_method parameter.
For some strange reason, its progress value doesn’t always return 1.0 in the end (returning 0.8597, for example), which is why I had to manually set the 1.0 progress value after the operation is finished.
Inside VideoUploader, we include the CarrierWave::Extensions::VideoMultiThumbnailer module and update the encode method using the create_thumbnails_step:
include ::CarrierWave::Extensions::VideoMultiThumbnailer
def encode(format, opts = {})
# ...
read_video_metadata_step(format, opts)
create_thumbnails_step('jpg', opts) if model.needs_thumbnails?
end
private
def create_thumbnails_step(format, _opts)
create_thumbnails_for_video(
format,
progress: :processing_progress,
save_thumbnail_files_method: :save_thumbnail_files,
resolution: '300x300',
vframes: model.file_duration, frame_rate: '1', # create thumb for each second of the video
processing_metadata: { step: 'create_video_thumbnails' }
)
endSystem Resource Usage Control
FFmpeg video processing is a heavy operation that usually takes almost 100% of server CPU resources. This might slow down some more important operations like web servers and databases.
There are few solutions that can help you deal with this overload: you can limit FFmpeg CPU usage with some Linux system tools and FFmpeg configuration flags, run processing inside a docker container with limited resources, or separate all these into some service that runs on a different machine than the primary server.
Also note that in this section we’re using video and audio filters, which will be covered in the second part of this article.
1. Limit CPU usage with system tools
You can alter FFmpeg system priority with the Linux nice command during process startup, or alter the priority of the already running process with renice.
With nice, the highest priority is -20 and the lowest possible priority is 19. Setting a nice priority can help you save CPU cycles for more important processes.
Next, you can use cpulimit when you want to ensure that a process doesn’t use more than a certain portion of the CPU. The disadvantage of nice is that you can’t use all of the available CPU time when the system is idle.
Here’s my CPU load without FFmpeg or any other heavy operations: System Idle
A small 15-second clip with one video and one audio filter can be processed in 28 seconds on my 4-core Macbook Air with 100% CPU load
time ffmpeg -i test_video.mp4 -vf frei0r=vertigo:0.2 -af "chorus=0.5:0.9:50|60|40:0.4|0.32|0.3:0.25|0.4|0.3:2|2.3|1.3" -vcodec h264 -acodec aac -strict -2 video_with_filters.mp493.45s user 0.88s system 333% cpu 28.285 total was processed in 28 seconds with 100% CPU load – this is what my system usage looks like with no limitations.
With 50% CPU limit
time cpulimit --limit 50 ffmpeg -i test_video.mp4 -vf frei0r=vertigo:0.2 -af "chorus=0.5:0.9:50|60|40:0.4|0.32|0.3:0.25|0.4|0.3:2|2.3|1.3" -vcodec h264 -acodec aac -strict -2 video_with_filters_cpulimit_50.mp4103.44s user 1.57s system 50% cpu 3:26.60 total – this is what my system usage looked like with a 50% CPU limitation.
FFmpeg also has a -threads option that limits the number of threads used (CPU cores). The recommended value is the total number of CPU cores minus one or two. The default value is all available CPU Cores. Note that this option always should be added as the last parameter (before output file name).
time ffmpeg -i test_video.mp4 -vf frei0r=vertigo:0.2 -af "chorus=0.5:0.9:50|60|40:0.4|0.32|0.3:0.25|0.4|0.3:2|2.3|1.3" -vcodec h264 -acodec aac -strict -2 -threads 1 video_with_filters_one_thread.mp460.02s user 1.48s system 99% cpu 1:02.04 total
It took me one minute to complete this operation. CPU load was not as bad either: FFmpeg only thread was like this.
Read more about restricting Linux system resources here.
You can also use all these options simultaneously.
If you’re using the same gems as in this example app – streamio-ffmpeg and carrierwave-video – then you can specify the -threads parameter as a custom option the same way it was used for filters in our application.
In order to use the nice and cpulimit commands with the streamio-ffmpeg gem, you have to alter the default FFmpeg command path inside the initializer file.
The original streamio-ffmpeg gem has a verification that verifies if the FFmpeg binary is executable.
This check won’t let you set this parameter directly with this data /usr/local/bin/cpulimit –limit 50 /usr/local/bin/ffmpeg.
So there are two ways to set this parameter directly: redefine the FFMPEG.ffmpeg_binary=(bin) method to remove this check or create an executable bash script. Since monkeypatching a gem’s code is bad practice, let’s create a bash script with our code:
/usr/local/bin/cpulimit --limit 50 /usr/local/bin/ffmpeg "$@"This “$@” parameter passes all .sh script arguments to the FFmpeg command.
Let’s put this script into the project’s root folder. Don’t forget to make it executable:
chmod +x ffmpeg_cpulimit.shNow you can set the path to your script at initialization:
FFMPEG.ffmpeg_binary = "#{::Rails.root}/ffmpeg_cpulimit.sh"The current sidekiq configuration (config/sidekiq.yml) can work on three tasks simultaneously. If you also want to limit system load, you have to set concurrency to 1 to run only one processing task at a time.
:verbose: true
:pidfile: ./tmp/pids/sidekiq.pid
:logfile: ./log/sidekiq.log
:concurrency: 3
:queues:
- carrierwave2. FFmpeg inside Docker
Another option is to run FFmpeg commands inside Docker. Doing this, it’s possible to limit how many resources a single FFmpeg instance may consume. This also this gives you the ability to control not only CPU usage but also memory and IOs.
Let’s try it with a predefined image from GitHub.
I’ve altered the command to avoid using the frei0r filter since this FFmpeg image doesn’t have the frei0r plugin:
time docker run --rm -v `pwd`:/tmp/workdir -w="/tmp/workdir" jrottenberg/ffmpeg -i test_video.mp4 -filter_complex colorchannelmixer=.393:.769:.189:0:.349:.686:.168:0:.272:.534:.131 -af "chorus=0.5:0.9:50|60|40:0.4|0.32|0.3:0.25|0.4|0.3:2|2.3|1.3" -vcodec h264 -acodec aac -strict -2 test_video_docker.mp40.02s user 0.02s system 0% cpu 33.632 total operation has been completed in 33 seconds.
With the default configuration, my CPU load wasn’t 100% for all cores, but also included FFmpeg with Docker.
You can learn in detail how this works in Docker references.
And of course, you have to alter the streamio-ffmpeg gem’s code to use this command with Docker if you want to use it with Ruby gems we mentioned above.
3. Separate video processing service
If you have enough resources to run heavy background operations on a dedicated server, then that might be the best option.
You could create such service with an API and callbacks for different processing events like processing finished or failed, or with websocket processing progress notifications.
This requires more work on the developer’s side to create a custom discrete service and integrate it with the primary application server (which runs some business logic), but it’s much better in terms of performance and reliability than sharing one machine between the business logic the server and jobs that run in the background.
Here’s a very simple example of such a background jobs application video processor app.
It can work on only one task – video trimming – and it doesn’t have any callbacks, but it has a very simple API.
In this guide, we’ve covered the implementation of FFmpeg video and audio filters and Frei0r plugin effects and our media experiments with them. FFmpeg and Frei0r are powerful tools. The filters described here mostly add some additional distortions to video. But FFmpeg can also be used to perform video and audio enhancements such as lens correction, sound level normalization, noise and shake removal, video streaming, and much more.
Ten articles before and after
API Versioning: Which Gem to Pick for API Versioning on Ruby on Rails
When and Why Angular Is a Good Technical Solution for Your Project
How to Develop a Multi-Language Ruby on Rails Application and Take It Global
Testing Web Software: Unit Testing Methods
Real-Time Features: Best Use Cases and Reason To Implement Them In Your App
How to Use the Android NDK When Writing an App in Kotlin
Integrating SiriKit in a Third-Party iOS App
Using Machine Learning Algorithm for Predicting House Valuations
Introducing Horizon, Our Open Source Library for Sound Visualization
How We Created uCrop, Our Own Image Cropping Library for Android