Video processing and publishing

Background and strategy

This description is very specific to our setup and file name and directory layout. So, the scripts used will assume that the naming and directory layout as described below is in place.

Raw video and sound file information

The video format from the cam memory cards are in "MTS" format:

Input #0, mpegts, from 'Sessions/101-RepRapOpenHardware/00001.MTS':
Duration: 00:34:48.96, start: 0.481989, bitrate: 7277 kb/s
Program 1
Stream #0.0[0x1011]: Video: h264, yuv420p, 1440x1080 [PAR 4:3 DAR 16:9], 50 tbr, 90k tbn, 50 tbc
Stream #0.1[0x1100]: Audio: ac3, 48000 Hz, stereo, s16, 256 kb/s

And the external sound is plain WAV, for instance:

Sessions/101-RepRapOpenHardware/
|-- 00001.MTS
|-- 4CH002I.wav
`-- 4CH002M.wav

The sound in the video file is from the camera mic, and the sound on 4CH002I.wav is from the (??) mic, and the sound on 4CH002M.wav is from the (??) mic.

What format and sources we decided on for publishing on Vimeo

First of all, the MTS files are huge and not easy to handle. Each file is between 1 and 4 GB in size. Second, it is timely and tedious to match and align the surrounding sound files with the video (and sync it with the sound from the camera mic). So we decided to go for only the original internal camera sound for publishing on Vimeo.

Vimeo is a bit picky with what formats it accepts for uploads, and the ffmpeg and mencoder software are not so forgiving either. Rikard found that, since we are to prepend each published video with an FFKP crafted credits page (FSCONS and thanks to .SE and Nordisk Kulturfond) and a CC license, a way forward was to make everything videos in an AVI container (using the same codecs for video and sound in both the prepend files and the AVI from the session MTS).

This way, mencoder could concatenate credits->license->session into an AVI with the whole shebang.

This is an example of the format for a Session AVI:

Input #0, avi, from 'Sessions/encoding/encoded_vids/101-RepRapOpenHardware-1.avi':
Duration: 00:34:48.99, start: 0.000000, bitrate: 2283 kb/s
Stream #0.0: Video: h264, yuv420p, 1280x720, PAR 1:1 DAR 16:9, 25 tbr, 25 tbn, 50 tbc
Stream #0.1: Audio: ac3, 48000 Hz, stereo, s16, 256 kb/s

and this is the cc license AVI:

Input #0, avi, from 'Sessions/encoding/prepends/cc-1920.avi':
Duration: 00:00:02.01, start: 0.000000, bitrate: 634 kb/s
Stream #0.0: Video: h264, yuv420p, 1280x720, PAR 1:1 DAR 16:9, 50 tbr, 50 tbn, 100 tbc
Stream #0.1: Audio: ac3, 48000 Hz, stereo, s16, 256 kb/s

In order to create the license AVI you need to add a fake sound track in the ac3, 48000 Hz, stereo, s16, 256 kb/s format for it to match the AVI created from the MTS file. Strategy for the encoding scripts

We decided to create and upload around one session per day. In order to keep the original files safe, Rikard created a working directory on our NAS where the input files to be encoded by the scripts, and the prepend files and the resulting AVI reside:

$ ls -l Sessions/encoding
total 1
drwx------ 1 rikard rikard 0 2010-11-26 14:01 114-ConflictingValues
drwx------ 1 rikard rikard 0 2010-11-26 14:02 118-OneWireSensors
drwx------ 1 rikard rikard 0 2010-11-26 14:13 120-EmbeddingLinux
drwx------ 1 rikard rikard 0 2010-11-26 13:05 130-SocialResponsibleNetworks
-rwx------ 1 rikard rikard 122 2011-03-14 10:14 DoneAndUploadedSessions.txt
drwx------ 1 rikard rikard 0 2011-03-14 10:21 encoded_vids
drwx------ 1 rikard rikard 0 2011-03-08 11:57 prepends

The directories with names starting with a number are the sessions to be encoded by one round of the encoding scripts. The results go into the encoded_vids directory, and the credits and license AVIs are in the prepends directory. The textfile is just a list to keep track of what has been uploaded to Vimeo and that one is manually maintained ;-)

The encoded AVIs end up in the encoding/encoded_vids directory.

Scripts and commands used to start a round of encoding

There is a directory on our NAS with all the scripts needed for a round of encoding:

~$ tree Sessions/bin/
Sessions/bin/
|-- encode_all.sh
`-- encode.sh

The encode_all.sh script must be run from the Sessions directory (or it won't run). It assumes you have put all sessions directories you want to encode, for instance 114-ConflictingValues 118-OneWireSensors 120-EmbeddingLinux and 130-SocialResponsibleNetworks in the encoding working directory (which then works as a queue for encoding). This is best accomplished by logging into the nas (in this case the nas is also the place for the working directory "encoding") via ssh and rsync the directories to be encoded from Sessions to Sessions/encoding:

/home/rikard$ cd Sessions/
Sessions$ rsync -va --progress 114-ConflictingValues 118-OneWireSensors 120-EmbeddingLinux 130-SocialResponsibleNetworks encoding/

Note that it is, of course, imperative to leave out the trailing slashes on the session directory names, or otherwise rsync will only copy the files within the directories into the encoding directory, and not the whole directories themselves.

After that, the encoding directory will have, in this case, four video directories whose names start with a number:

114-ConflictingValues
118-OneWireSensors
120-EmbeddingLinux
130-SocialResponsibleNetworks.

This is how the encode_all.sh finds them (it treats all directories in encoding that have names starting with a number as video directories to be encoded). This is of cource specific to our setup, and that is exploited by the encoding scripts.

The encode.sh script is just used by encode_all.sh as an auxiliary script, wrapping the creation of AVI from MTS (using ffmpeg).

The resulting encoded AVIs end up in encoding/encoded_vids/ and if there are more than one MTS video per session, they are named with a sequence number starting at 1 like so:[SessionName]-1-all.avi, e.g.:

108-FutureTransports-1-all.avi 108-FutureTransports-2-all.avi 108-FutureTransports-3-all.avi

To start an encoding round, these are the steps:

   1. Make sure last run has finished
   2. Move or delete last run's video directories from the encoding directory
   3. Log in to the nas via ssh and rsync the session directories to be encoded so that they
      are copied from Sessions/ into Sessions/encoding/
      (leave out trailing slashes on video direcory names so that the complete directories are copied)
   4. cd into Sessions/
   5. Run: encode_all.sh 2>&1 | tee -a txt-files/encoding.log (-a flag is for "append") Leave this shell
      open (or start it in a Screen session)
   6. If you'd like to watch what's going on, do a tail -200f txt-files/encoding.log (if you want to see
      what it's doing from a different shell)
   7. When the script is finished, verify that the videos are OK with mplayer, e.g.:
      Sessions$ mplayer encoding/encoded_vids/101-RepRapOpenHardware-1-all.avi
   8. Log in on Vimeo and upload the file named SessionName-1-all.avi (the -all part simply means it's got
      the credits and license prepended and included in it).
   9. Look up the session on FSCONS 2010 schedule and name the video SpeakerFirstName SpeakerLastName - Session Name
  10. In the description, start with the title: SpeakerFirstName SpeakerLastName - Session Name [(pt 1/n)]
      (if it's one of many parts), then copy the abstract from FSCONS schedule page (sessions in the schedule
      are links to the abstract page)
  11. Tag the video with appropriate tags, FSCONS, FSCONS 2010, and save the video.
  12. Do the same for each newly encoded file (named with -all.avi at the end) and then delete the
      Session directories from the encoding working directory to save up space if needed.
  13. Mark the session as uploaded in encoding/DoneAndUploadedSessions.txt

How does it work?

The encode_all.sh script must be run from the Sessions directory. It loops over each directory in Sessions/encoding starting with a number. For each such directory, it loops over each .MTS video file and encodes it to AVI naming it SessionDirectoryName-N.avi (where N is the sequence number, starting with 1 - e.g. the three MTS files in 108-FutureTransports/ will be named: 108-FutureTransports-1.avi 108-FutureTransports-2.avi 108-FutureTransports-3.avi). Next, it prepends each one of them with the credits and licens AVI into an AVI named e.g. 108-FutureTransports-1-all.avi . This means that it is the files named SessionDirectoryName-N-all.avi that are the full files, credits and license and session, that are interesting and to be uploaded to Vimeo.

encode_all.sh is available below and is provided "as is" and use at own risk etc.

The encoding into AVI is done with help of the Session/bin/encode.sh script. It uses ffmpeg with the following flags and options:

ffmpeg -i encoding/SessionDirectoryName/RawVideoFromCamera.MTS -acodec copy -vcodec libx264 -s 1280x720 -vpre default -b 2000000 -r 25 \
encoding/encoded_vids/SessionDirectoryName-N.avi

The encode.sh script is available here and is provided "as is" and use at own risk etc.

The concatenation of the credits and license AVIs to the beginning of the end result AVI before the session video is done with mencoder:

mencoder -oac copy -ovc copy -o encoding/encoded_vids/SessionDirectoryName-N-all.avi \
encoding/prepends/1920-title.avi encoding/prepends/cc-1920.avi \
encoding/encoded_vids/SessionDirectoryName-N.avi

Prerequisites

In order to run the encoding scripts, a number of tools, libraries and codecs need to be installed.

The video transformation tools are:

• ffmpeg
• mencoder

The codecs are:

• libavcodec-extra-52
• libaften0 - Not required if you have the license and title AVIs with AC3 sound! (for creating the title and license AVI, we needed it to have fake silent audio as AC3 to match the AVIs created from the Session MTS files, or mencoder would protest)

To test the results:

• mplayer

Apart from this, you need a fairly speedy PC in order to get a timely encoding. And of course, you need to have mounted the NAS directories on that PC.

What about the raw files and surround audio WAVs?

We'll create torrents of the raw material.

Creating title frames with silent audio

If you'd like to create a - say 4 seconds - banner to show first in the video with no audio, you can do that from a PNG image.

This is how I did it:

ffmpeg -loop_input -t 4 -i TitleNeutral.png -ar 44100 -acodec pcm_s16le -f s16le -ac 2 \
 -i /dev/zero -acodec ac3 -ar 48000 -ab 256k -vcodec libx264 -s 1280x720 \
 -vpre default -b 5000000 -r 50 titleNeutral.avi

The PNG was TitleNeutral.png and just a screen saying "FSCONS" more or less. I needed to create an AVI with AC3 sound (silent, but still AC3) so that mencoder could concatenate it with the AVI from a session. mencoder needs the video AND audio streams to be exactly the same, in order to concatenate the AVIs (as far as I've discovered, anyway).

Here's another example with an exisiting sound file, this time in AAC format:

ffmpeg -loop_input -t 4 -i CC720x756.png -i silence.aac -acodec copy -vpre libx264-main \
   -b 5000000 -r 50 CC.avi

And the silent AAC file was created like this:

1. create a WAV:
ffmpeg -loop_input -t 4 -ar 44100 -acodec pcm_s16le -f s16le -ac 2 -i /dev/zero silence.wav
2. Create the AAC (using aacplusenc):
aacplusenc silence.wav silence.aac 32

Short re-cap

The Sessions directories all have names starting with a number, such as 101-RepRapOpenHardware . Then cd into Sessions/ and run bin/encode_all.sh 2>&1 | tee -a txt-files/encoding.log (which will append output to the logfile).

Note that bin/encode_all.sh will encode every MTS file from every directory in Sessions/encoding starting with a number (such as 101-RepRapOpenHardware ) so clean up after you are done if file space is an issue;-)

The resulting file(s) to be uploaded to Vimeo end up in Sessions/encoding/encoded_vids/ and will have the name(s) in this form: SessionDirectoryName-N-all.avi (where N is the sequence number, starting with 1). The -all part of the name, is there to separate it from the AVI without credits and license. It is the "-all" file you upload to Vimeo. You can tag and describe the video on Vimeo according to our standard as described above.

Scripts

encode_all.sh

#!/bin/bash

# Copyright (c) 2011 Rikard Fröberg <rikard@ffkp.se>.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA

NAS_MOUNT="YOUR_NAS_MOUNT_POINT_HERE"
START_TIME=$(date)
STARTSECS=$(date +%s)
ENCODE_DIR="encoding"
OUTPUT_DIR="$ENCODE_DIR/encoded_vids"
PREPEND_DIR="$ENCODE_DIR/prepends"
CREDITS="$PREPEND_DIR/1920-title.avi"
LICENSE="$PREPEND_DIR/cc-1920.avi"
MENCODER_OPTS="-oac copy -ovc copy -o"

if [ ! -e "$ENCODE_DIR" ] ; then
 echo "$ENCODE_DIR not found. Exiting."
 exit 1
fi

echo Starting at $START_TIME
echo Dir to process: $ENCODE_DIR

# Directories starting with a number contains MTS video files in our case
for SESSION_DIR in $ENCODE_DIR/[0-9]*
do
 countMTS=0
 echo ============================
 echo $(date) Processing $SESSION_DIR
 echo

 # Our raw files all end with .MTS
 for SESSION_MOVIE in ${SESSION_DIR}/*.MTS
 do
  ((++countMTS))
  SESSION_SHORT=${SESSION_MOVIE##encoding/}
  AVI="${SESSION_SHORT%%/*}-$countMTS.avi"
  FINAL_AVI="${SESSION_SHORT%%/*}-$countMTS-all.avi"
  echo $(date) Running: "bin/encode.sh $SESSION_MOVIE $OUTPUT_DIR/$AVI"

  # Our nas was mounted over the network, double check that the
  # mount has not been lost due to network problems
  if [ ! -e "$SESSION_MOVIE" ]
  then
    echo "$SESSION_MOVIE" NOT FOUND.
    if [ -d "$NAS_MOUNT" ]
    then
      echo "NAS mounted on $NAS_MOUNT"
    else
      echo "NAS NOT mounted!"
    fi 
    exit 2
  fi
  bin/encode.sh "$SESSION_MOVIE" "$OUTPUT_DIR/$AVI"
  echo $(date) Running: "mencoder $MENCODER_OPTS $OUTPUT_DIR/$FINAL_AVI $CREDITS $LICENSE $OUTPUT_DIR/$AVI"
  /usr/bin/mencoder $MENCODER_OPTS $OUTPUT_DIR/$FINAL_AVI $CREDITS $LICENSE $OUTPUT_DIR/$AVI 
  echo $OUTPUT_DIR/$FINAL_AVI created at $(date)
  echo "++++++++++++++"
 done
done
ENDSECS=$(date +%s)
TOTAL_SECS=$((ENDSECS-STARTSECS))
ENDTIME=$(date)
echo "Report:"
echo "Encoding started at $START_TIME"
echo "Encoding ended at   $ENDTIME"
echo "Encoding took       ${TOTAL_SECS}secs or"
echo "                    $((TOTAL_SECS/60))mins and $((TOTAL_SECS%60))secs or"
echo "                    $((TOTAL_SECS/3600))hrs and $((TOTAL_SECS/60%60))mins and $((TOTAL_SECS%60))secs"

encode.sh

#!/bin/bash

# Copyright (c) 2011 Rikard Fröberg <rikard@ffkp.se>.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA
#
#Prerequisites
#
#In order to run the encoding scripts, a number of tools, libraries and
#codecs need to be installed.
#
#The video transformation tools are:
#
#    * ffmpeg
#    * mencoder 
#
#The codecs are:
#
#    * libavcodec-extra-52
#    * libaften0 - Not required if you have the license and title AVIs! 
# (for creating the title and license AVI, we needed it to have fake
# silent audio as AC3 to match the AVIs created from the Session MTS
# files, or mencoder would protest) 
#
# To test the results:
#
#    * mplayer 
########################################################################

infile=$1
outfile=$2
inputopts="-i $infile"
audioopts="-acodec copy"
videoopts="-vcodec libx264 -s 1280x720 -vpre default -b 2000000 -r 25"


echo "ffmpeg command:"
echo $inputopts $audioopts $videoopts $outfile
/usr/bin/ffmpeg $inputopts $audioopts $videoopts $outfile

Back to 2011 main planning page