Category Archives: Multimedia

Serviio 0.6 DLNA server on Synology NAS

Serviio-Synology

Update – I have released a Synology package for Serviio 0.6

 

Serviio is an excellent free Java DLNA media server by Petr Nejedly which focuses on minimizing the amount of unnecessary media transcoding, and maximizing the use of renderer devices’ supported features. Version 0.6 adds support for streaming content from online RSS feeds. Since the installation process has changed a lot, I decided to create a new post rather than edit the old one. The guide is accurate for Synology DSM 3.2 beta, and the one tricky section with wget-ssl also applies for DSM 3.1. Older DSM versions may vary.

In the Synology DSM go to Control Panel > Terminal > enable SSH.

Read this Synology wiki document about modifying your NAS carefully and install the bootstrap for your model. If you have previously been messing around with your unit and want to set it back to defaults, you can run the bootstrap again. It will prompt you to delete a couple of folders and reboot, whereupon you can start afresh.

Download the PuTTY SSH client if you’re using Windows – other operating systems will already allow you to SSH from the terminal. Give the session a name, set the remote character set to UTF-8 as shown and save it before you connect (to save time in future). This will ensure that all characters display properly. The Mac OS SSH client defaults to UTF-8.
PuTTY remote character set configuration

Connect to your NAS’s IP address using SSH. Use the root account (same password as admin). For Mac OS for instance, use ssh root@x.x.x.x. I suggest enabling a more descriptive shell prompt which should reduce the chance of accidentally being in the wrong directory:

sed -i -r 's/(^PS1=.*$)/#\1\n\1/' ~/.profile
sed -i -e 's/^PS1=.*$/PS1=\"\\w\\\$ \"/g' ~/.profile
source ~/.profile
 

sed (stream editor) is a powerful tool which uses regular expressions. The first command opens the file ~./profile searches for the line beginning with PS1= and duplicates it, commenting out the first copy (so the change can be undone). Then second command sets the value PS1=”\w\$ “ which changes the prompt to be the current directory followed by dollar sign and space characters.

To install the development tools. Type:

ipkg update
ipkg install optware-devel
 

It will halt and complain that package wget-ssl clashes with wget. Continue with:

cp /opt/bin/wget /opt/bin/wget-old
ipkg remove wget
cp /opt/bin/wget-old /opt/bin/wget
ipkg install libidn
ipkg install optware-devel
 

This time it will finish successfully.

Next we need to install Lame MP3 encoder, providing libmp3lame which will be compiled into FFmpeg, and the Nano text editor (much easier to use than vi), and fix up some other issues (thanks to Nicolas Jolet for these):

ipkg install lame
ipkg install nano
#-----coreutils ls has no colour output by default
mv /opt/bin/ls /opt/bin/ls.bak
ln -sf /bin/ls /opt/bin/ls
#-----coreutils uptime is broken
mv /opt/bin/uptime /opt/bin/uptime.bak
ln -sf /usr/bin/uptime /opt/bin/uptime
 

Download Java SE for Embedded 7 from Oracle, selecting the Linux build for the appropriate CPU. The ARM v5 version is required for the Marvell Kirkwood CPU Synology Products. Note that there is a PowerPC e500v2 version – the CPU core in Synology products which use the Freescale mpc85x3). Unfortunately for PowerPC Synology owners, this depends on a higher version of glibc than the Synology DSM provides for this architecture. Until JamVM supports Java 1.6, or Synology update to glibc 2.4 you won’t be able to follow this guide on PowerPC models. This situation may have changed since DSM 3.2 beta was released.

You will need to sign up to receive the Java download link by email. It’s free to use for non-commercial evaluation use. Use your computer to save it into the top level shared folder of your NAS, which will probably be /volume1/public on the NAS filesystem. Then:

cd /volume1/public
mv ejre-7-fcs-b147-linux-arm-sflt-headless-27_jun_2011.tar.gz /volume1/@tmp
cd /volume1/@tmp
tar xvzf ejre-7-fcs-b147-linux-arm-sflt-headless-27_jun_2011.tar.gz
mv ejre1.7.0 /opt/local/java
 

Synology’s Linux build has no localization support built in, though it does use UTF-8 character encoding for the filesystem. That’s no problem for storage, however the Java VM inherits the locale setting of the host OS. Since this is undefined, Serviio and all other Java software will default to US-ASCII which is a big problem if you have filenames with non-US characters. The solution is to obtain the missing files to add locale support from the Synology toolchain, which is distributed under the GPL. Many thanks to IWarez at the Subsonic forum for this fix, though it took me a good while to realise the required files are indeed included in the ARM toolchain:

#-----for ARM CPU
cd /volume1/@tmp
wget http://sourceforge.net/projects/dsgpl/files/DSM%203.1%20Tool%20Chains/Marvell%2088F628x%20Linux%202.6.32/gcc421_glibc25_88f628x.tgz
tar xvfz gcc421_glibc25_88f628x.tgz
cd arm-none-linux-gnueabi/arm-none-linux-gnueabi/libc/usr/bin
cp locale /opt/bin
cp localedef /opt/bin
cp -R arm-none-linux-gnueabi/arm-none-linux-gnueabi/libc/usr/share/i18n /usr/share
#-----keep another copy safe in case DSM is reinstalled later
cp -R arm-none-linux-gnueabi/arm-none-linux-gnueabi/libc/usr/share/i18n /opt/share
mkdir /usr/lib/locale
localedef -c -f UTF-8 -i en_US en_US.utf8

#-----for Intel CPU
cd /volume1/@tmp
wget http://sourceforge.net/projects/dsgpl/files/DSM%203.1%20Tool%20Chains/Intel%20x86%20Linux%202.6.32/gcc420_glibc236_pineview.tgz
tar xvfz gcc420_glibc236_pineview.tgz
cd i686-linux-gnu/i686-linux-gnu/bin
cp locale /opt/bin
cp localedef /opt/bin
cp -R i686-linux-gnu/i686-linux-gnu/share/i18n /usr/share
#-----keep another copy safe in case DSM is reinstalled later
cp -R i686-linux-gnu/i686-linux-gnu/share/i18n /opt/share
mkdir /usr/lib/locale
localedef -c -f UTF-8 -i en_US en_US.utf8

 

Now use nano to edit some configuration files (Ctrl-o saves, and Ctrl-x exits).
First edit the profile for all bash shell users:

nano /opt/etc/profile
 

Make the changes shown in bold:

#
# Bash initialization script
#

PS1=”[\u@\h \W]$ ”
PATH=/opt/sbin:/opt/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/local/java/bin
LD_LIBRARY_PATH=/opt/lib:${LD_LIBRARY_PATH}
JAVA_HOME=/opt/local/java
LANG=en_US.utf8

export PS1 PATH LD_LIBRARY_PATH JAVA_HOME LANG

 

Save and exit. Next edit the profile for all ash shell users (root):

nano /etc/profile
 

Add the changes shown in bold:

#/etc/profile: system-wide .profile file for ash.
PATH=”$PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/syno/bin:/usr/syno/sbin:/usr/local/bin:/usr/local/sbin
PATH=”$PATH:/opt/local/java/bin”
JAVA_HOME=/opt/local/java
LANG=en_US.utf8
export JAVA_HOME LANG

umask 022

 

Save and exit. Now install Serviio 0.6:

cd /volume1/@tmp
wget http://download.serviio.org/releases/serviio-0.6-linux.tar.gz
tar xvzf serviio-0.6-linux.tar.gz
mv serviio-0.6 /opt/local/serviio
 

Next, compile librtmp which FFmpeg will use to retrieve Adobe Flash streaming video, the libssl static libraries (which librtmp depends on), libz and libbz2 static libraries for FFmpeg.

#-----libz static lib (ipkg only provides shared)
wget http://zlib.net/zlib-1.2.5.tar.gz
tar xvfz zlib-1.2.5.tar.gz
cd zlib-1.2.5
./configure --prefix=/opt --static
make
make install
 

At this point ou will see the error “cp: missing destination file operand after `/opt/lib'” because the libz.so shared library wasn’t compiled and the install script tries to copy it nonetheless. libz.a which we need was built ok, so ignore and continue.

cd ..
#-----libbz2 static lib (ipkg only provides shared)
wget http://bzip.org/1.0.6/bzip2-1.0.6.tar.gz
tar xvfz bzip2-1.0.6.tar.gz
cd bzip2-1.0.6
make
make install PREFIX=/opt
cd ..

#-----libssl & libcrypto static libs (ipkg only provides shared)
ipkg install openssl-dev
wget http://www.openssl.org/source/openssl-0.9.8p.tar.gz
tar xvfz openssl-0.9.8p.tar.gz
cd openssl-0.9.8p
./config --prefix=/opt no-shared
make
cp libssl.a /opt/lib
cp libcrypto.a /opt/lib
#-----librtmp won't compile without these pkgconfig definitions
cp libssl.pc /opt/lib/pkgconfig
cp libcrypto.pc /opt/lib/pkgconfig
cd ..

#-----librtmp depends on libpthreads but the optware copy seems to be broken
#-----http://forum.synology.com/enu/viewtopic.php?f=90&t=30132
mkdir /opt/arm-none-linux-gnueabi/lib_disabled
mv /opt/arm-none-linux-gnueabi/lib/libpthread* /opt/arm-none-linux-gnueabi/lib_disabled
cp /lib/libpthread.so.0 /opt/arm-none-linux-gnueabi/lib/
cd /opt/arm-none-linux-gnueabi/lib/
ln -s libpthread.so.0 libpthread.so
ln -s libpthread.so.0 libpthread-2.5.so
cd /volume1/@tmp
wget http://download.serviio.org/opensource/rtmpdump-c58cfb3e9208c6e6bc1aa18f1b1d650d799084e5.tar.gz
tar xvfz rtmpdump-c58cfb3e9208c6e6bc1aa18f1b1d650d799084e5.tar.gz
cd rtmpdump
#-----move all static libs to a separate folder to force compiler to use them
#-----http://www.gossamer-threads.com/lists/apache/dev/265052
mkdir /volume1/@tmp/lib
cp /opt/lib/*.a /volume1/@tmp/lib
#-----fix Makefile (won't compile without libdl linked)
#-----http://forum.luahub.com/index.php?topic=2390.0
sed -i.bak -e '/^LIB_OPENSSL\=/s/lcrypto/lcrypto \-ldl/' Makefile
make SYS=posix prefix=/opt INC=-I/opt/include XLDFLAGS=-L/volume1/@tmp/lib SHARED=
make install prefix=/opt SHARED=
cd ..

#-----gather all static libs again ready for FFmpeg compile
cp /opt/lib/*.a /volume1/@tmp/lib

#-----remove unsupported URL line from /opt/lib/pkgconfig/librtmp.pc
#-----pkg-config --exists --print-errors librtmp
sed -i -e '/^URL/d' /opt/lib/pkgconfig/librtmp.pc

#-----FFmpeg 0.8.2 (slightly newer than the one at Serviio.org)
cd /volume1/@tmp
wget http://dl.dropbox.com/u/1188556/ffmpeg-HEAD-05a2673.tar.gz
tar xvfz ffmpeg-HEAD-05a2673.tar.gz
cd ffmpeg-HEAD-05a2673

#-----for ARM CPU
./configure --arch=arm --enable-armv5te --prefix=/opt --extra-cflags='-I/opt/include' --extra-ldflags='-L/volume1/@tmp/lib' --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-pthreads --enable-libmp3lame --enable-librtmp --extra-version=Serviio

#-----for Intel CPU
ipkg install yasm
./configure --arch=x86_64 --enable-ssse3 --prefix=/opt --extra-cflags='-I/opt/include' --extra-ldflags='-L/volume1/@tmp/lib' --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-pthreads --enable-libmp3lame --enable-librtmp --extra-version=Serviio

#-----for PowerPC CPU
./configure --arch=powerpc --disable-altivec --prefix=/opt --extra-cflags='-I/opt/include' --extra-ldflags='-L/volume1/@tmp/lib' --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-pthreads --enable-libmp3lame --enable-librtmp --extra-version=Serviio

make
make install
 

Notice that the ./configure command is line wrapped – it’s all one command. The make command takes approximately 25 minutes and will show many warnings during compilation, but this is expected.

Try running ffmpeg and check the compile time to make sure the newly compiled one is running. You should see this but with your compilation date and time:

ffmpeg version 0.8.2.git-05a2673-Serviio, Copyright (c) 2000-2011 the FFmpeg developers
built on Aug 22 2011 09:57:40 with gcc 4.2.3
configuration: –arch=arm –enable-armv5te –prefix=/opt –extra-cflags=-I/opt/include –extra-ldflags=’-L/volume1/@tmp/lib’ –enable-static –disable-shared –disable-ffplay –disable-ffserver –enable-pthreads –enable-libmp3lame –enable-librtmp –extra-version=Serviio
libavutil 51. 13. 0 / 51. 13. 0
libavcodec 53. 11. 0 / 53. 11. 0
libavformat 53. 9. 0 / 53. 9. 0
libavdevice 53. 3. 0 / 53. 3. 0
libavfilter 2. 34. 1 / 2. 34. 1
libswscale 2. 0. 0 / 2. 0. 0
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]… {[outfile options] outfile}…

Use -h to get full help or, even better, run ‘man ffmpeg’

In the DSM User Control Panel create a new user called serviio and set a password. Give that user access to the paths that contain the media you want to serve. Click the User Home button and enable the User Home Service. Go back to your SSH session and type:

nano /etc/passwd
 

Be very careful editing this file. A wrong move here could trash your system. Notice that the serviio user has a shell of /sbin/nologin. Change this to /bin/sh like the admin user has. Nano may try to line wrap this line as you type if you added an long account description. If it does, delete the carriage return before the line break and pull it back onto one line. Save and exit.

Now we’ll create the Serviio daemon start and stop script:

nano /volume1/@tmp/S99serviio.sh
 

Paste in the following text (mouseover and use the view source button to copy):

#!/bin/sh

User=serviio

case "$1" in

stop)
        echo "Stop Serviio..."
        su -l $User -c "/opt/local/serviio/bin/serviio.sh -stop" > /dev/null 2>&1 &
        ;;

start)
        # start Serviio in background mode
        su -l $User -c "/opt/local/serviio/bin/serviio.sh" > /dev/null 2>&1 &
        echo "Start Serviio..."
        ;;

restart)
        $0 stop
        sleep 1
        $0 start
        ;;
*)
        echo "usage: $0 { start | stop | restart}" >&2
        exit 1
        ;;

esac
 

Now make it executable and set the serviio user to be the owner of the Serviio folder (so it can create the media database, and log files):

chmod +x /volume1/@tmp/S99serviio.sh
chown -R serviio /opt/local/serviio
 

Test the script manually by running:

/volume1/@tmp/S99serviio.sh start
ps
 

The running process list should show the serviio user running the Serviio launcher shell script and the JVM like so:

 5132 root     16988 S    /usr/syno/sbin/nmbd -D
 5134 admin    34804 S    postgres: admin photo [local] idle
 5187 root     21080 S    /usr/syno/sbin/smbd -D
 5189 root     21080 S    /usr/syno/sbin/smbd -D
 5205 root      7216 S    /usr/syno/sbin/cnid_metad -l log_error
 5207 root     13808 S    /usr/syno/sbin/afpd -g guest -c 256 -n DS111 AFPServ
 5253 root      9208 S N  /usr/syno/sbin/synomkflvd
 5317 root      4112 S    /usr/syno/sbin/sshd
 5332 root      6932 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 5344 root      7068 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 5345 root      7068 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 5369 root      6928 S    /usr/syno/apache/bin/httpd
 5383 nobody    6928 S    /usr/syno/apache/bin/httpd
 5384 nobody    6928 S    /usr/syno/apache/bin/httpd
 5385 nobody    6928 S    /usr/syno/apache/bin/httpd
 5436 root      2408 S    avahi-daemon: running [DS111.local]
 5460 root     11928 S    sshd: root@pts/0
 5466 root      5452 S    -ash
 5582 root         0 SW   [flush-8:0]
 5602 serviio   2988 S    -sh -c /opt/local/serviio/bin/serviio.sh
 5605 serviio   2988 S    /bin/sh /opt/local/serviio/bin/serviio.sh
 5610 serviio   498m S    /opt/local/java/bin/java -Xmx384M -Djava.net.preferI
 5623 root      5452 R    ps
 

Stop the daemon with:

/volume1/@tmp/S99serviio.sh stop
 

Check the web UI or the process list again and make sure it did indeed stop. If it’s all ok, we need to move the daemon launcher script so it starts automatically on boot (it will survive a DSM upgrade):

mv /volume1/@tmp/S99serviio.sh /opt/etc/init.d
 

Shutdown the NAS and restart, checking that Serviio starts by itself.

From now on you can start and stop Serviio manually using this same script (say if you were experimenting with changes to profiles.xml for instance):

/opt/etc/init.d/S99serviio.sh stop
/opt/etc/init.d/S99serviio.sh start
 

Remote console

AcidumIrae’s PHP Web UI is only for Serviio up to version 0.5.2 at present, and at the time of writing the Java Restful Web UI by kairoh does not yet integrate with Serviio 0.6. The only option for now is to use the proper Serviio Console from another computer. Since 0.6 there is no longer a requirement to change anything at the server end, so you just need to pass the parameter -Dserviio.remoteHost=x.x.x.x to the Serviio Console to indicate the IP of your Serviio server. This is covered in the Serviio FAQ. There’s quite a lag for it to start up (at least on Windows), but eventually you see the familiar icon.

 

Serviio settings

Navigate to the Transcoding tab. Set the transcoded files location to /volume1/@tmp. Failure to do this will result in temporary files being written to /tmp which will fill up the partition it’s on the moment you remux a 4GB movie, which would prevent you from logging into the NAS. If your Synology has an Intel CPU set the number of transcoding threads to 2. The dual cores should take advantage of FFmpeg’s pthreads support for a decent performance boost.

I recommend de-selecting Generate thumbnails for local videos in the Metadata tab. Note that you have to click Save on each tab or any change will not take effect. As you’ll see if you have a DLNA renderer that supports thumbnails, Serviio retrieves good ones from the online databases it checks so they’re not really needed. Often FFmpeg will get stuck trying to generate a thumbnail for a video and will lock the CPU at 100% for long periods of time. This issue was raised in this Serviio forum thread.

You can use the Library tab to add the media folders you want to share. Use Add Path not Add Local (we’re using a remote console remember) and express the paths in Linux syntax, e.g. /volume1/public/videos. Be sure that the serviio user has been granted read privileges over the folders you add (User Control Panel in DSM).

 

Performance

The Synology seems perfectly able to transcode DTS audio in a hi-def Matroska file down to 2 channel AC3 in MPEG-TS while copying the H.264 stream. The CPU use leaps up to 100% but I guess that’s because it’s running ahead transcoding down to the end of the file (and is CPU-bound).

If I play something that’s only remuxing the container and copying both audio and video streams then the CPU stays at around 40% (because it’s I/O-bound) then 5 minutes into a film it falls away to pretty much idle – I guess it has finished remuxing to the temporary file.

Memory use even while running Serviio is around 20% at idle, 30% during a remux with audio transcode, though I have noticed that with a larger library this creeps up to around 70%. I have a 2011 product so it has 256MB of RAM. This seems to indicate that the value line of products (with the j suffix) like the DS211j should run Serviio on their 128MB. They have a 1.2GHz CPU though versus 1.6GHz on mine (and the DS110j is only 800MHz), so it would need testing by someone.

All in all it would seem that the Synology products are very capable Serviio appliances!

 

Uninstalling

Log in as root and stop Serviio, then in the DSM User Control Panel delete the user called serviio. Undo the changes that you made to /opt/etc/profile and /etc/profile (the LANG, JAVA_HOME and java path modifications marked in bold earlier in the guide).

Finally, re-download the bootstrap for your model of NAS and run it again. It will tell you to delete a few folders and restart. This will completely trash all optware ipkg packages (i.e. everything in /opt) and undo everything else you did in this guide without affecting your data partitions. If you had any other ipkg packages installed since installing Serviio, these would also be lost.

Advertisements

Batch script for recursive FFmpeg transcoding

I recently had to shrink around 50GB of MP3 audio recordings that were sitting in a nested folder structure on a web server. Having experimented to find more appropriate LAME encoder settings for spoken word content I needed to transcode the files whilst keeping the existing ID3 tags intact. FFmpeg can do this nicely using libmp3lame, whereas LAME by itself cannot. Armed with my own compile of FFmpeg, I created a drag & drop batch script to recursively work through a folder structure transcoding MP3 and WAV files and writing out the resulting MP3 files in the same folder structure with an amended top level folder name. It will accept multiple files or folders being dragged and dropped. You could adapt this script to whatever task you’re using FFmpeg for.

::MP3 transcoding to more sensible default quality settings for spoken word
::32kHz mono VBR @ approx. 64Kbps
::
::pcloadletter.co.uk

@echo off
setlocal ENABLEDELAYEDEXPANSION
::parse multiple command lines (so multiple targets can be dragged and dropped at once)
:commandlineloop
if "%~1"=="" goto :continue
call :mainloop %1
shift
goto :commandlineloop
:continue
echo.
pause
goto :eof

::main loop we run for each command line arg of the script
:mainloop
set folderpath=
if "%~1"=="" (
  echo Drag and drop files or folders onto this script.
  goto :eof
)
set folderpath=%~1
echo "%1" | find /I ".mp3" && (
  goto :fileonly
)
echo "%1" | find /I ".wav" && (
  goto :fileonly
)

::folders
echo.
echo.
echo Processing folder %1
echo ________________________________________________________________________________
echo.

::we need to find the last folder level in the folderpath
::batch scripting is horrible so we need to use a hacky way to get the last token
::(from http://stackoverflow.com/questions/5473840/last-token-in-batch-variable)
set temp_string=%~1
set count=0

::iterate parsing the tokens and trimming the string until it's empty, while counting how many loops
:loopcounter
for /F "tokens=1* delims=\" %%a in ( "%temp_string%" ) do (
  set /A count+=1
  set temp_string=%%b
  goto loopcounter
)
for /F "tokens=%count% delims=\" %%i in ( "%~1" ) do set folder=%%i

::append the top level folder name with "-optimized" so we create a new folder tree
for /R "%folderpath%" %%i in (*.mp3 *.wav) do (
  set sourcepath=%%~di%%~pi
  set destpath=!sourcepath:%folder%=%folder%-optimized!
  md "!destpath!"
  ffmpeg -i "%%~fi" -acodec libmp3lame -aq 8 -ar 32000 -ac 1 "!destpath!%%~ni.mp3"
)
goto :eof

::individual files
:fileonly
for %%i in ("%folderpath%") do (
  ffmpeg -i "%folderpath%" -acodec libmp3lame -aq 8 -ar 32000 -ac 1 "%%~di%%~pi%%~ni-optimized.mp3"
)
 

There are several neat little tricks here. Multiple folders can be dragged onto the script because it uses SHIFT to work through each command line argument. The next hurdle is to get the last part of the folder name(s) being dragged. Though this would be simple in any Unix-like shell, but to do this in batch without relying on any additional tools proved to be quite tricky. Though tokens can be parsed by FOR, finding the last one (which we need) requires us to count how many tokens there are. Only then can the last one be selected. The folder name string substitution is done by the SET command (also used in my file renaming script). Delayed variable expansion means that the variables between the the exclamation marks are evaluated once per loop rather than the default of once per script execution (more info here). I hadn’t realised until today that you can create a folder structure several layers deep with a single MD command. This avoids having to iterate through all subfolders – we can use FOR’s /R switch to handle the recursion in a single line. For more information on some of the variables containing tildes like %%~pi and %%~ni, try running FOR /?. The script also first runs the FFmpeg subroutine with ECHO in front of the commands so you can double-check the syntax before proceeding.

There are of course much more efficient ways of compressing spoken word content than MP3 these days – AAC-HEv2 for instance, but that will rule out all but the latest audio playback devices.
The source MP3 recordings were in 44.1kHz mono @ 128Kbps. This was definitely overkill for speech. Just by opting for 64Kbps I could halve that. However I couldn’t drop the sample rate below 32kHz, as this is the lowest legal limit of MPEG-1 layer III. This info is tricky to find but the LAME encoder reveals it in its extended help:

MPEG-1   layer III sample frequencies (kHz):  32  48  44.1
bitrates (kbps): 32 40 48 56 64 80 96 112 128 160 192 224 256 320
MPEG-2   layer III sample frequencies (kHz):  16  24  22.05
bitrates (kbps):  8 16 24 32 40 48 56 64 80 96 112 128 144 160
MPEG-2.5 layer III sample frequencies (kHz):   8  12  11.025
bitrates (kbps):  8 16 24 32 40 48 56 64

 

I think it’s safe to assume that most MP3 players can handle variable bit rate. As far as I can remember only the first generation ones couldnt from early last decade – the sort of ones which only had around 64MB of storage. I don’t think excluding players this old is much of a concern.

Playing around with VBR quality settings in LAME we can see that with -V 9 we get a small file (12.9MB becomes 3.7MB) however it’s actually an MPEG-2 layer III file which as you can see above supports lower sample rates. I’m not certain of the wider compatibility of this sub-type of MP3 file, and besides it does sound considerably worse than the lowest MPEG-1 layer III settings which is –V 8 (12.9MB becomes 5.2MB). This proved to be the best compromise between size and quality.

Compiling x64/x86 FFmpeg on 64bit Windows with libfaac and libmp3lame

Once I had compiled FFmpeg for 32bit Windows, I discovered that it’s probably ten times more difficult to find information about doing the same with free tools on 64bit editions of Windows. A perfect topic for PCLOADLETTER then!

For building FFmpeg we must use the build environment MinGW-w64 which is a separate version of MinGW. The complication is that even when compiling natively you need to use configuration options as if you’re cross compiling (for host type x86_64-w64-mingw32). It’s not strictly necessary with all packages, but some will fail to build without these parameters. As a result you have to search around trying to find all the necessary configure commands for each package, and they’re not always identical. It’s a slow and frustrating process. What’s even more confusing, is that some builds of MinGW-w64 are targetted at 32bit, some 64bit, and some both. What is more, I had to try three different ones before I found one that could compile LAME without errors (though that could be LAME’s fault).

Most people will want a toolchain with the flexibility to create both x64 and x86 binaries, so that is what I shall describe in this guide.

On the MinGW-w64 downloads site browse to the Multilib Toolchains (Targetting Win32 and Win64) section, and download the latest megasoft78 binary build prefixed with mingw-w64-bin_x86_64-mingw. Extract the contents of this archive to the root of your C: drive (which at time of writing will create a folder named mingw64_4.5.2_multilib).

There is no installer – it’s not as automated as the 32bit distribution of MinGW. I spent a long time trying to compile LAME with the official MSYS build and eventually gave up, chosing instead to use a working one made by a third party. Download MSYS_MinGW_GCC_460_x86-x64_Full.7z from xhmikosr.1f0.de. Extract that archive into C:\mingw64_4.5.2_multilib so you have a subfolder in there named MSYS.

Update – I now realise that this archive also includes the whole GCC toolchain as well, but by then I’d already written the guide :)

Download the latest DirectX SDK. Install it, but only select the following component:
DirectX SDK installation options

This document provides useful background information to the uninitiated. Crucially it explains that when you compile with a particular host type defined, most configure scripts will look for a compiler whose filename is prefixed with that host type. So gcc.exe might be present in the toolchain distro as x86_64-w64-mingw32-gcc.exe. The toolchain I just recommended you download does not contain prefixes, but we shall need them since FFmpeg’s configure script expects them. On Windows Vista or later (which should cover most 64bit Windows systems) we can use mklink.exe to create symbolic links. We’ll also copy some missing DirectX headers into the toolchain and configure MSYS.

Start a Command Prompt as Administrator:

cd \mingw64_4.5.2_multilib\bin\
for %i in (*.exe) do mklink x86_64-w64-mingw32-%i %i
copy /y "%DXSDK_DIR%\Include\*.*" C:\mingw64_4.5.2_multilib\x86_64-w64-mingw32\include\
cd \mingw64_4.5.2_multilib\msys\postinstall
pi
 

This will launch the post-install script which will configure a filesystem mount for /mingw. When asked, enter your mingw installation path as c:/mingw64_4.5.2_multilib

Once that’s complete, launch the MSYS shell from your Administrator command prompt:

C:\mingw64_4.5.2_multilib\MSYS\msys.bat
 

Download the source .tar.gz files of:

Copy those downloaded files to C:\mingw64_4.5.2_multilib\msys\home\administrator (since you’re running in the Administrator user context). In the MSYS shell this is your home directory (cd ~).

In the MSYS shell, run:

tar xvfz faac-1.28.tar.gz
tar xvfz lame-3.98.4.tar.gz
tar xvfz ffmpeg-HEAD-6fd00e9.tar.gz
 

For a 64bit compile of FFmpeg with libfaac and libmp3lame:

cd ~/faac-1.28
./configure --prefix=/usr/local/x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --enable-static --disable-shared --with-mp4v2=no
make clean && make
make install
cd ..
cd lame-3.98.4
./configure --prefix=/usr/local/x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --enable-static --disable-shared --disable-decoder --enable-nasm
make clean && make
make install
cd ..
cd ffmpeg-HEAD-6fd00e9
CPPFLAGS="$CPPFLAGS -I/usr/local/x86_64-w64-mingw32/include" ./configure --extra-ldflags='-L/usr/local/x86_64-w64-mingw32/lib' --prefix=/usr/local/x86_64-w64-mingw32 --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-nonfree --enable-libfaac --arch=x86_64 --enable-runtime-cpudetect --enable-w32threads --cross-prefix=x86_64-w64-mingw32- --target-os=mingw32
make clean && make
make install
 

The compiled ffmpeg.exe will be in C:\mingw64_4.5.2_multilib\MSYS\local\x86_64-w64-mingw32\bin.

In summary what’s happening is we’re compiling FAAC without MP4v2 support (compilation halts if that’s enabled), and we’re making sure the binaries are stored in /usr/local/x86_64-w64-mingw32. Likewise for LAME. FFmpeg needs to be told where to find these new libraries (and headers too). We’re also installing it to /usr/local/x86_64-w64-mingw32, so as to keep x64 and x86 binaries separated.

 

For a 32bit compile of FFmpeg with libfaac and libmp3lame:

cd ~/faac-1.28
CFLAGS="-m32" CXXFLAGS="-m32" ./configure --prefix=/usr/local/i686-pc-mingw32 --host=x86_64-w64-mingw32 --enable-static --disable-shared --with-mp4v2=no
make clean && make
make install
cd ..
cd lame-3.98.4
CFLAGS="-m32" CXXFLAGS="-m32" ./configure --prefix=/usr/local/i686-pc-mingw32 --host=x86_64-w64-mingw32 --enable-static --disable-shared --disable-decoder --enable-nasm
make clean && make
make install
cd ..
cd ffmpeg-HEAD-6fd00e9
CPPFLAGS="$CPPFLAGS -I/usr/local/i686-pc-mingw32/include" ./configure --extra-cflags=-m32 --extra-ldflags='-m32 -L/usr/local/i686-pc-mingw32/lib' --prefix=/usr/local/i686-pc-mingw32 --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-nonfree --enable-libfaac --arch=x86_32 --enable-runtime-cpudetect --enable-w32threads --cross-prefix=x86_64-w64-mingw32- --target-os=mingw32
make clean && make
make install
 

The compiled ffmpeg.exe will be in C:\mingw64_4.5.2_multilib\MSYS\local\i686-pc-mingw32\bin.

This is pretty much the same as before, but CFLAGS=”-m32″ and CXXFLAGS=”-m32″ force the 64bit-native C and C++ compilers to produce 32bit binaries. I found that useful option at the bottom of this FFmpeg wiki page. As usual, FFmpeg has its own way of specifying most of these settings (except CPPFLAGS). It is also instructed to use 32bit libraries via –extra-ldflags=’-m32′.

 

Footnote

Until I gave up and used the MSYS distro from xhmikosr.1f0.de, LAME persistently failed to compile with the error undefined reference to `init_xrpow_core_sse’. I tried a newer version of NASM than 2.09.08, replacing nasm.exe with yasm.exe, using LAME 3.98.2 instead of 3.98.4, disabling NASM in the configure, using the Unix makefile, all to no avail. Apart from a problem someone had compiling a Mac universal binary (fixed by using compiler options I could not use with MinGW), the only other link I could find about this is here, and that’s also unsolved. Even taking the nasm.exe (which I notice is 32bit) from the working MSYS bundle and trying it with the SourceForge ‘official’ MSYS bundle, I cannot get LAME to compile. I had no such problem when using the normal MinGW on my 32bit PC from my previous post on this subject, despite compiling my own nasm.exe. If you have any insight into why this might be, please leave a comment!

 

Compiling FFmpeg for Windows with libfaac and libmp3lame

Having realised how simple it is to compile stuff on my Synology NAS I decided to finally get something built on Windows using free tools. It’s not really much different, but I had to do a lot of searching to finally gather all the necessary configure parameters.

First we need to set up the build environment. MinGW stands for ‘Minimalist GNU for Windows’. Download the MinGW-get installer, which will install everything you’ll need including MSYS which is a Unix shell complete with all the essential binaries. Run the MinGW-get installer and select Download latest repository catalogs.
In the options add the C++ compiler (needed for FAAC) and select MSYS Basic System and MinGW Developer Toolkit.

Now download the source .tar.gz files of:

Launch MinGW Shell from the Start Menu.
Copy those downloaded files to C:\MinGW\msys\1.0\home\yourusername
For each one run:

tar xvfz packagename.tar.gz
 

Then in order, cd into each extracted directory and…

For NASM and yasm:

./configure
make
make install
 

For FAAC (we need to disable the mp4v2 option to avoid a nasty compiler error – took me a long time to discover):

./bootstrap CC='gcc -mno-cygwin'
./configure --prefix=/mingw --exec-prefix=/mingw --enable-static --disable-shared --with-mp4v2=no
make
make install
 

For LAME:

./configure --prefix=/mingw --exec-prefix=/mingw --enable-static --disable-shared --disable-decoder --enable-nasm
make
make install
 

For FFmpeg:

./configure --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-nonfree --enable-libfaac --arch=x86 --enable-runtime-cpudetect --enable-w32threads
make
make install
 

The compiled ffmpeg.exe will be in C:\MinGW\msys\1.0\local\bin

Update – I tried to set this up on my work PC which runs 64 bit Windows 7 and I discovered that it’s a world of pain, so I’ll have to amend this post once I solve it.

5.1 channel audio through ordinary headphones using MPC-HC

Blown-Away Man

Though I have owned a surround sound system before, I don’t currently have the space for one. I was recently considering buying gaming headphones with support for Dolby Digital 5.1 channel audio which I could also use for watching films. With some more reading I discovered that these hardware solutions will only work with a Dolby Digital source (AC-3). The problem is that I have a lot of content with DTS audio which the chips in these products don’t support, not to mention MP4 files with multichannel AAC audio.

Reading the specs of these headphones, I discovered that they license a technology called Dolby Headphone – some clever signal processing developed by Lake DSP in Sydney. Early 5.1 channel headphones actually contained multiple tweeters mounted at different positions inside each ‘can’ which apparently weren’t very convincing, but this better approach is more of an emulation – one that relies on the fact that you only have two ears. It’s able to model the audio delays and reflections of a room containing a 5.1 setup, plus it seems to boost up the low frequencies so you’re really aware of the LFE channel. I don’t really understand why there isn’t much awareness of Dolby Headphone. It’s not at all new – look at this 1998 press release!

What’s also not very widely known is that Dolby Headphone can be encoded in software, through the likes of PowerDVD’s audio decoder, regardless of whether your sound chip is Dolby Headphone certified (as some now are). It’s also not particularly processor intensive – my Sony Vaio P can play H.264 movies and process DH with its Intel GMA 500 GPU and meagre 1.33GHz Intel Atom Z520 CPU.

 

What does Dolby Headphone sound like?

Listen for yourself – plug in some headphones and try these DH encoded samples. The choice of movie trailer isn’t exactly showcasing the 360 degree soundstage, but it was quite tricky to encode so I didn’t want the hassle of sampling a scene from an actual film. I can assure you that the action sequences in District 9 for instance sound a whole lot better with Dolby Headphone enabled.


These samples are 192Kbps MP3 files for size reasons. The slight metallic high-end ringing to the deep bass is an encoder artifact from the conversion to MP3 using the LAME encoder, and it was still present when I tried 320Kbps. The channel test clip was taken from http://www.lynnemusic.com/surround.html and the I Am Legend trailer was from http://www.h264info.com/clips.html.

 

How to get it working for AC-3, AAC, and DTS multichannel sources

In a previous post I explained how to setup Media Player Classic Home Cinema to use a third party h.264 decoder with DXVA support. MPC-HC has several key advantages as a media player – it’s open source, it’s lean, it’s extremely configurable, and it plays pretty much any format. I use it for my Sony Vaio P to keep CPU use as low as possible on its rather limited Intel Atom processor. Many people use MPC-HC for their Home Theatre PCs, hence this article.

A guide for configuring Dolby Headphone using PowerDVD’s CyberLink Audio Decoder already exists (with files):
http://www.head-fi.org/forum/thread/405417/guide-dolby-headphone-in-mpc-zoom-player

However that information isn’t sufficient. The complication is that when this decoder is used outside the PowerDVD software, it only works with DTS audio streams even though it should also support AC-3. Reading around, it seems that the AC-3 support works in Windows XP but not in Vista nor in Windows 7.

The next piece of the puzzle can be found here on the forum for Zoom Player, a commercial media player:
http://forum.inmatrix.com/index.php?showtopic=7224

The important point is that the CyberLink decoder can accept a multichannel LPCM input, so you can use another decoder (ffdshow tryouts in this case) to convert the source bitstream into LPCM first, then feed that to the CyberLink decoder. This means that multichannel AAC audio found in MP4 containers could have Dolby Headphone applied too. The above post describes how to do this for Zoom Player, but we’re interested in MPC-HC. The next obstacle is figuring out how to link multiple decoders together in MPC-HC – something I couldn’t find any info about despite spending hours searching online.

The key to understanding this is knowing how the modular Windows DirectShow media layer works. If you’ve already played with MPC-HC you will be familiar with adding and removing filters, as well as the concepts of ones that are built into MPC-HC and those that are external. All these DirectShow filters have ‘pins’ which are their inputs and outputs – their connection points in other words. Each pin has a defined list of media types it will and won’t accept. As long as you comply with this, they can be threaded together to create a ‘graph’. The simplest way to illustrate this is with a screenshot of the aptly named GraphStudio, and this example shows the combination of filters I used to make one of the sample audio clips earlier in this post:

So how can we construct a similar sequence in MPC-HC? In the end I finally found the crucial bit of information in a single sentence of the excellent self-explanatory and unofficial All the things you may want to know about Media Player Classic – HomeCinema:

When a video file is loaded in MPC-HC, MPC-HC will run through this list from top to bottom and load the relevant filter(s), create a graph from them and play the file.

Bingo! All you need to do is explicitly list them in the right order in MPC-HC’s External Filters window and MPC-HC should thread them together, compatible pins permitting. Sure enough, it works.

Would you believe it, there is yet another hitch – the CyberLink Audio Decoder expects LPCM input channels to be in a certain order, one that differs from the default order in ffdshow. This is trivial to fix, though it will mess things up if you rely on ffdshow to decode other media formats.

 

The guide

You’ll need MPC-HC obviously, the CyberLink Audio Decoder will need to be installed (from PowerDVD), and you’ll need ffdshow tryouts installed (I used the SVN release, since the beta is years old).

Once ffdshow is installed, open up its Audio Decoder configuration utility from the Start Menu. In the Output section select 16bit LPCM and apply.

Disable the Mixer and configure the Swap Channels menu as follows. Note the different order:

Open MPC-HC and hit O to bring up the options menu. In the Internal Filters section de-select AAC, AC3 and DTS from Transform Filters on the right.

UPDATE – Contrary to what I had originally written here, do not disable the built-in audio switcher! I hadn’t understood that this is needed for when you have a video file with multiple audio tracks. If it’s not enabled then all audio tracks play simultaneously. You could consider disabling Regain volume if you want to minimize the amount of additional processing.

Audio switcher settings

In the External Filters section make sure that ffdshow is set to prefer, and is listed above CyberLink Audio Decoder (also set to prefer). Use Add Filter… to set Microsoft DTV-DVD Audio Decoder to Block.

Remember that DTS audio bitstreams don’t require the ffdshow treatment. To avoid having to reconfigure this for different movie formats we can customize the recognized media types for the ffdshow filter. As per the screenshot above, select ffdshow Audio Decoder then scroll down until you find MEDIASUBTYPE_DTS and delete it. DTS files will now skip ffdshow and will be entirely handled by CyberLink Audio Decoder. If you ever need to revert this change click Reset List.

Open a movie file in MPC-HC, then select Play -> Filters -> CyberLink Audio Decoder and you will be able to select Dolby Headphone like so:

On the next tab you can choose between three DH modes. I found a nice succinct guide to the three modes here:
http://www.head-fi.org/forum/thread/444681/headphones-with-good-sound-stage-better-with-dolby-headphone-or-no-need-for-it#post_6003214

Sadly it seems that every time you load a new file into MPC-HC you need to re-enable Dolby Headphone (the setting doesn’t stick for some reason). Also, owing to the channel order swapping in ffdshow you’ll find that in ordinary stereo mode you’re only getting the Front Left and Front Center channels. As a result you’ll probably want to de-select the Swap Channels setting in the ffdshow Audio decoder tool when you’re not using headphones. A small price to pay though… Enjoy!

 

Further reading

There is a guide to using Dolby Headphone for listening to two channel music in the audio player foobar2000 here:
http://www.head-fi.org/forum/thread/447089/5-1-headphone-experience-foobar-configuration-for-all-stereo-music-files

I’ve given it a try and I don’t really think it’s worth it. When I had a actual surround system I used to play all my music on Dolby Pro Logic IIx and it sounded very good using all the speakers. It meant you could be sitting anywhere in the room and still get nice separation and so on. However, creating those additional channels only to merge them back down to two channels often doesn’t seem to add much since they were virtual to start with. In side by side listening comparisons I often preferred the original stereo mix, but admittedly it does seem to vary with the way they’ve been mastered (remember, to hear the original stereo you have to remove all selected DSPs, not just DH).

Serviio 0.5.2 DLNA server on Synology NAS

last updated 24/08/2011 – fixed typos in locale section
Serviio-Synology

This guide is now redundant – I have released a Synology package for Serviio 0.6!

 

Serviio is an excellent free Java DLNA media server by Petr Nejedly which focuses on minimizing the amount of unnecessary media transcoding, and maximizing the use of renderer devices’ supported features. Some of the more main-stream servers like Windows Media Player just brute-force everything to MPEG2 video and MP3 audio, which degrades quality and wastes power. Though some servers like Mezzmo are better and will play Matroska files, even they tend to transcode all audio to AC-3 regardless of source type. As a Java app Serviio will run on anything that has a JVM, and the media tool it relies on is the open source and therefore highly portable FFmpeg. All these design priorities make Serviio an ideal choice to run on a NAS device since, when paired with a renderer with good format support like a Sony Bluray Player, the NAS will barely ever be transcoding.

I was about to buy a new large external hard disk, but once I realised that Serviio could probably run on a NAS I started looking at one of these instead. Synology seemed to offer a lot of value and seemed to have the sort of user-community enjoyed by my old Linksys NSLU2, which I promptly sold on eBay for almost what I had paid for it in 2007. I considered the value DS110j model but I decided to go for the more expensive DS111 on the basis that the double CPU speed and RAM would probably be a wise move.

This guide outlines how to get Serviio 0.5.2 running on the Marvell Kirkwood ARM CPUs found in most of the 2011 product line-up, but Synology devices also exist with Freescale PowerPC and Intel Atom processors. The key problem is finding a Java virtual machine, but FFmpeg also needs compiling from source. This is because although there is an FFmpeg binary bundled with DSM 3.0, it’s too old and lacks support for features critical to Serviio. This guide could be used for other CPU architectures, but the compilation options for FFmpeg need adapting.

In the Synology DSM go to Control Panel > Terminal > enable SSH.

Read this Synology wiki document about modifying your NAS carefully and install the bootstrap for your model.

Download the PuTTY SSH client.

Connect to your NAS’s IP address using SSH. Use the root account (same password as admin). I suggest that you perform the mod at the bottom of this post to enable colour directory listings and a more descriptive shell prompt which should reduce the chance of accidentally being in the wrong directory.

We need to install the development tools. Type:

ipkg install optware-devel
 

It will halt and complain that package wget-ssl clashes with wget. Continue with:

ipkg remove wget
cp /usr/syno/bin/wget /opt/bin
ipkg install wget-ssl
ipkg update
ipkg upgrade
ipkg install optware-devel
 

This time it will finish successfully.

Update – It seems that there is a serious problem with running ipkg on a clean install of DSM 3.1. This guide was written before it was released, and though I have since upgraded my Synology I haven’t encountered that issue, but there have been many comments about it. User mayk on the Synology forum seems to have the solution here. Use the extra wget verbosity switch to find out the exact package URL for the following two packages, then manually download them with wget and install:

cd /volume1/@tmp
ipkg install -verbose_wget libidn
wget url1
ipkg install -verbose_wget wget-ssl
wget url2
ipkg install filename1
ipkg install filename2
 

Next we need to install Lame MP3 encoder, providing libmp3lame which FFmpeg will be compiled to depend on, and the Nano text editor (much easier to use than vi):

ipkg install lame
ipkg install nano
 

JamVM is a JVM that gets mentioned a lot in connection with NAS systems, but it’s only Java 1.5, and Serviio needs version 1.6. Download the Java SE Embedded Runtime from Oracle, selecting the ARM v5 Linux version (note that there is a PowerPC e500v2 version – the CPU core in Synology products which use the Freescale mpc85x3). Unfortunately for PowerPC Synology owners, this depends on a higher version of glibc than the Synology DSM provides for this architecture. Until JamVM supports Java 1.6, or Synology update to glibc 2.4 you won’t be able to follow this guide on PowerPC models. This may have changed since DSM 3.1 was released.

You will need to sign up to receive the download link by email. It’s free to use for non-commercial self-educational use. Use your computer to save it into the top level shared folder of your NAS, which will probably be /volume1/public on the NAS filesystem. Then:

cd /volume1/public
mv ejre-1_6_0_21-fcs-b09-linux-arm-sflt-eabi-headless-27_sep_2010.tar.gz /volume1/@tmp
cd /volume1/@tmp
tar xvzf ejre-1_6_0_21-fcs-b09-linux-arm-sflt-eabi-headless-27_sep_2010.tar.gz
mkdir /opt/java
mv ejre1.6.0_21 /opt/java
 

Synology’s Linux build has no localisation support built in, though it does use UTF-8 character encoding for the filesystem. That’s no problem for storage, however the Java VM inherits the locale setting of the host OS. Since this is undefined Serviio, and all other Java software, will default to US-ASCII which is a big problem if you have filenames with non-US characters. The solution is to obtain the missing files to add locale support from the Synology toolchain, which is distributed under the GPL:

#-----for ARM CPU
cd /volume1/@tmp
wget http://sourceforge.net/projects/dsgpl/files/DSM%203.1%20Tool%20Chains/Marvell%2088F628x%20Linux%202.6.32/gcc421_glibc25_88f628x.tgz
tar xvfz gcc421_glibc25_88f628x.tgz
cd arm-none-linux-gnueabi/arm-none-linux-gnueabi/libc/usr/bin
cp locale /opt/bin
cp localedef /opt/bin
cp -R arm-none-linux-gnueabi/arm-none-linux-gnueabi/libc/usr/share/i18n /usr/share
#-----keep another copy safe in case DSM is reinstalled later
cp -R arm-none-linux-gnueabi/arm-none-linux-gnueabi/libc/usr/share/i18n /opt/share
mkdir /usr/lib/locale
localedef -c -f UTF-8 -i en_US en_US.utf8

#-----for Intel CPU
cd /volume1/@tmp
wget http://sourceforge.net/projects/dsgpl/files/DSM%203.1%20Tool%20Chains/Intel%20x86%20Linux%202.6.32/gcc420_glibc236_pineview.tgz
tar xvfz gcc420_glibc236_pineview.tgz
cd i686-linux-gnu/i686-linux-gnu/bin
cp locale /opt/bin
cp localedef /opt/bin
cp -R i686-linux-gnu/i686-linux-gnu/share/i18n /usr/share
#-----keep another copy safe in case DSM is reinstalled later
cp -R i686-linux-gnu/i686-linux-gnu/share/i18n /opt/share
mkdir /usr/lib/locale
localedef -c -f UTF-8 -i en_US en_US.utf8
 

Now use nano to edit some configuration files (Ctrl-o saves, and Ctrl-x exits).
First edit the profile for all bash shell users:

nano /opt/etc/profile
 

Make the changes shown in bold:

#
# Bash initialization script
#

PS1=”[\u@\h \W]$ ”
PATH=/opt/sbin:/opt/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/java/ejre1.6.0_21/bin
LD_LIBRARY_PATH=/opt/lib:${LD_LIBRARY_PATH}
JAVA_HOME=/opt/java/ejre1.6.0_21
LANG=en_US.utf8

export PS1 PATH LD_LIBRARY_PATH JAVA_HOME LANG

 

Save and exit. Next edit the profile for all ash shell users (root):

nano /etc/profile
 

At the last line make the changes in bold:

PATH=/opt/java/ejre1.6.0_21/bin:/opt/bin:/opt/sbin:$PATH
JAVA_HOME=/opt/java/ejre1.6.0_21
LANG=en_US.utf8
export JAVA_HOME LANG

 

Save and exit. Now install the Serviio application:

cd /volume1/@tmp
wget http://download.serviio.org/releases/serviio-0.5.2-linux.tar.gz
tar xvzf serviio-0.5.2-linux.tar.gz
mv serviio-0.5.2 /opt/serviio
 

In the DSM Control Panel got to Web Services > Web Applications tab > Enable Web Station.
Install AcidumIrae’s PHP web UI for Serviio. You will need to have enabled Web Station for the directory /volume1/web to exist.

wget http://labs.softjourn.com/attachments/download/67/serviio-0.5.2.1b.zip
unzip serviio-0.5.2.1b.zip
mv serviio-0.5.2 /volume1/web/serviio
 

You should already be able to browse to http://your_NAS_IP/serviio and see the user interface, though it will complain with a big red X that Serviio is not running.

FFmpeg depends on the libbz2 and zlib libraries, and although both are installed along with the optware-devel package, FFmpeg will only look for them in /lib rather than in their actual location in /opt/lib. Copies of the existing symbolic links will be fine:

cp /opt/lib/libbz2.so.1.0 /lib
cp /opt/lib/libz.so.1 /lib
 

Compile the patched version 26303 of FFmpeg from the Serviio download page. Running cat /proc/cpuinfo it is clear that the DS111 is an ARM 5TE platform so I enabled those specific optimizations:

cd /volume1/@tmp
wget http://download.serviio.org/opensource/ffmpeg-26303.tar.gz
tar xvzf ffmpeg-26303.tar.gz
cd ffmpeg
./configure --arch=arm --enable-armv5te --prefix=/opt --extra-cflags='-I/opt/include' --extra-ldflags='-L/opt/lib' --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-libmp3lame
make
 

Notice that the ./configure command is line wrapped – it’s all one command. The make command takes approximately 25 minutes on the 1.6GHz CPU and will show many warnings during compilation, but this is expected.

Update – Thanks to bakman for pointing out that for Intel Atom CPUs you will need to install the assembler YASM and also use the following ./configure parameters:

ipkg install yasm
./configure --arch=x86_64 --enable-ssse3 --prefix=/opt --extra-cflags='-I/opt/include' --extra-ldflags='-L/opt/lib' --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-libmp3lame
 

Another Update – Thanks to gregorio for parameters for Freescale PowerPC processors:

./configure --arch=powerpc --disable-altivec --prefix=/opt --extra-cflags='-I/opt/include' --extra-ldflags='-L/opt/lib' --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-libmp3lame
 

When the compile is done, install it.

make install
 

Try running ffmpeg and check the compile time to make sure the newly compiled one is running. You should see this but with your compilation date and time:

FFmpeg version UNKNOWN, Copyright (c) 2000-2011 the FFmpeg developers
built on Feb 6 2011 01:14:38 with gcc 4.2.3
configuration: –arch=arm –enable-armv5te –prefix=/opt –extra-cflags=-I/opt/include –extra-ldflags=-L/opt/lib –enable-static –disable-shared –disable-ffplay –disable-ffserver –enable-libmp3lame
libavutil 50.36. 0 / 50.36. 0
libavcore 0.16. 0 / 0.16. 0
libavcodec 52.108. 0 / 52.108. 0
libavformat 52.92. 0 / 52.92. 0
libavdevice 52. 2. 3 / 52. 2. 3
libavfilter 1.72. 0 / 1.72. 0
libswscale 0.12. 0 / 0.12. 0
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]… {[outfile options] outfile}…

Use -h to get full help or, even better, run ‘man ffmpeg’

If you see the text below then something is wrong – this is the version included with DSM 3.0:

FFmpeg version SVN-r20167-snapshot, Copyright (c) 2000-2009 Fabrice Bellard, et al.

In the DSM User Control Panel create a new user called serviio and set a password. Give that user access to the paths that contain the media you want to serve. Click the User Home button and enable the User Home Service. Go back to your SSH session and type:

nano /etc/passwd
 

Be very careful editing this file. A wrong move here could trash your system. Notice that the serviio user has a shell of /sbin/nologin. Change this to /bin/sh like the admin user has. Nano may try to line wrap this line as you type if you added an long account description. If it does, delete the carriage return before the line break and pull it back onto one line. Save and exit.

Now we’ll create the Serviio daemon start and stop script:

nano /volume1/@tmp/S99serviio.sh
 

Paste in the following text (mouseover and use the icon in the top right to copy):

#!/bin/sh

User=serviio

case "$1" in

stop)
        echo "Stop Serviio..."
        su -l $User -c "/opt/serviio/bin/serviio.sh -stop" > /dev/null 2>&1 &
        ;;

start)
        # start Serviio in background mode
        su -l $User -c "/opt/serviio/bin/serviio.sh" > /dev/null 2>&1 &
        echo "Start Serviio..."

        #check libs FFmpeg depends on (in case DSM was upgraded)
        if [ ! -f /lib/libbz2.so.1.0 ]; then
                cp /opt/lib/libbz2.so.1.0 /lib
        fi
        if [ ! -f /lib/libz.so.1 ]; then
                cp /opt/lib/libz.so.1 /lib
        fi
        ;;

restart)
        $0 stop
        sleep 1
        $0 start
        ;;
*)
        echo "usage: $0 { start | stop | restart}" >&2
        exit 1
        ;;

esac
 

Now make it executable and set the serviio user to be the owner of the Serviio folder (so it can create the media database, and log files):

chmod +x /volume1/@tmp/S99serviio.sh
chown -R serviio /opt/serviio
 

Test the script manually by running:

/volume1/@tmp/S99serviio.sh start
ps
 

The running process list should show the serviio user running the Serviio launcher shell script and the JVM like so:

 2542 root      9080 S N  /usr/syno/sbin/synomkflvd
 2589 root      3908 S    /usr/syno/sbin/sshd
 2591 root      5376 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 2599 root      5944 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 2600 root      5936 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 2628 root     58920 S    /usr/syno/apache/bin/httpd
 2687 nobody   58920 S    /usr/syno/apache/bin/httpd
 2688 nobody   58920 S    /usr/syno/apache/bin/httpd
 2689 nobody   59500 S    /usr/syno/apache/bin/httpd
 2695 root     11560 S    /usr/syno/sbin/mDNSResponder -f /tmp/mDNSResponder.c
 2703 root      5568 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 2949 root      5568 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 3025 root      5568 S    /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/
 3087 root      8664 S N  /bin/ntfs-3g -o uid=1024,gid=100 /dev/sdk1 /volumeUS
 3518 root      8852 S    /usr/syno/sbin/cnid_metad
 3524 root     15132 S    /usr/syno/sbin/afpd -c 256 -g guest -n SynologyDS111
 3566 nobody   58920 S    /usr/syno/apache/bin/httpd
 7747 root      6660 S    sshd: root@pts/0
 7826 root      5404 S    -ash
13567 serviio   2940 S    -sh -c /opt/serviio/bin/serviio.sh
13570 serviio   2940 S    /bin/sh /opt/serviio/bin/serviio.sh
13575 serviio   501m S    /opt/java/ejre1.6.0_21/bin/java -Xmx384M -Djava.net.
13917 root      5404 R    ps
 

Stop the daemon with:

/volume1/@tmp/S99serviio.sh stop
 

Check the web UI or the process list again and make sure it did indeed stop. If it’s all ok, we need to move the daemon launcher script so it starts automatically on boot:

mv /volume1/@tmp/S99serviio.sh /opt/etc/init.d
 

Shutdown the NAS and restart.

Update – previously I had used the directory /usr/syno/etc/rc.d, but this is destroyed when the DSM software is updated. /usr/local/etc/rc.d is the official Synology location for 3rd party daemon init scripts but I found that it doesn’t work, so I used /opt/etc/init.d instead which also survives a DSM upgrade. I just tested this by upgrading to DSM 3.1-1605 and Serviio remained intact. I only had to copy the sym links for the libraries FFmpeg needs (“cp /opt/lib/libbz2.so.1.0 /lib” and “cp /opt/lib/libz.so.1 /lib”).

 

Web UI integration with the Synology DSM

I read the official Synology notes for 3rd party developers and decided to make some icons. The result is pretty neat:

Open an SSH session:

cd /usr/syno/synoman/webman/3rdparty
wget http://dl.dropbox.com/u/1188556/blog/serviio_syno_DSM.zip
unzip serviio_syno_DSM.zip
rm serviio_syno_DSM.zip
nano serviio/config
 

The file loaded into nano will look like this, and you need to edit the IP address to match the IP of your Synology (either put your syno on a static IP, or set a reservation in your broadband router’s DHCP server options).

{
 ".url": {
    "org.serviio.serviio": {
      "type": "legacy",
      "allUsers": true,
      "title": "Serviio",
      "desc": "DLNA Media Server",
      "icon": "images/icon_{0}.png",
      "url": "http://192.168.1.202/serviio/index.php"
    }
  }
}
 

I can’t find a way to avoid manually specifying the IP address like this – relative paths can’t be used because the Synology runs two different webservers: the DSM one on port 5000 without PHP, and the PHP-enabled Web Station one on port 80.

Log out of DSM and log back in. You will see the Serviio icon in the pull down menu in the top left, which you can drag to the desktop if you prefer:

 

Serviio settings

Navigate to the Transcoding tab. Set the transcoded files location to /volume1/@tmp. Failure to do this will result in temporary files being written to /tmp which will fill up the partition it’s on the moment you remux a 4GB movie, which would prevent you from logging into the NAS.

I recommend de-selecting Generate thumbnails for local videos in the Metadata tab. Note that you have to click Save on each tab or any change will not take effect. As you’ll see if you have a DLNA renderer that supports thumbnails, Serviio retrieves good ones from the online databases it checks so they’re not really needed. Often FFmpeg will get stuck trying to generate a thumbnail for a video and will lock the CPU at 100% for long periods of time. This issue was raised in this Serviio forum thread.

You can use the Library tab to add the media folders you want to share. Note that the Add Local… button will fail because the web service user does not have access to the root of the filesystem (probably a good thing security-wise). Use Add Path… instead and express the paths as I have done in the first screenshot above. Be sure that the serviio user has been granted read privileges over the folders you add (User Control Panel in DSM).

Update – If you really want to use the Ajax file browser UI, then go to DSM Control Panel > Web Services > PHP Settings tab > open_basedir and append /volume1: to the start of that list. Then on the Web Applications tab, disable Web Station, Ok, enable Web Station. Go back to your SSH session and run:

nano /volume1/web/serviio/afb/config.php
 

Change the value of $path from / to /volume1/public (or the top level shared folder where your media resides). You’ll notice however that when you use Add local button in the Serviio Library tab it’s very buggy. It will only list a maximum of 5 child folders from each node.

 

Performance

The Synology seems perfectly able to transcode DTS audio in a hi-def Matroska file down to 2 channel AC3 in MPEG-TS while copying the H.264 stream. The CPU use leaps up to 100% but I guess that’s because it’s running ahead transcoding down to the end of the file (and is CPU-bound).

If I play something that’s only remuxing the container and copying both audio and video streams then the CPU stays at around 40% (because it’s I/O-bound) then 5 minutes into a film it falls away to pretty much idle – I guess it has finished remuxing to the temporary file.

Memory use even while running Serviio is around 20% at idle, 30% during a remux with audio transcode, though I have noticed that with a larger library this creeps up to around 70%. I have a 2011 product so it has 256MB of RAM. This seems to indicate that the value line of products (with the j suffix) like the DS211j should run Serviio on their 128MB. They have a 1.2GHz CPU though versus 1.6GHz on mine (and the DS110j is only 800MHz), so it would need testing by someone.

All in all it would seem that the Synology products are very capable Serviio appliances!

 

Uninstalling

Log in as root and stop Serviio, then delete a couple of libraries which had been copied to /lib:

/opt/etc/init.d/S99serviio.sh stop
rm /lib/libbz2.so.1.0
rm /lib/libz.so.1
 

In the DSM User Control Panel delete the user called serviio. Undo the changes that you made to /opt/etc/profile and /etc/profile (the JAVA_HOME and java path modifications marked in bold earlier in the guide).

To remove the web UI, in DSM go to Web Services > Web Applications tab > Disable Web Station. Then:

rm -r /volume1/web/serviio
rm -r /usr/syno/synoman/webman/3rdparty/serviio
 

Finally, re-download the bootstrap for your model of NAS and run it again. It will tell you to delete a few folders and restart. This will completely trash all optware ipkg packages (i.e. everything in /opt) and undo everything else you did in this guide without affecting your data partitions. If you had any other ipkg packages installed since installing Serviio, these would also be lost.

Live H.264 streaming from Digital Rapids to Helix and Wowza

Helix Mobile Server 14

I had a few difficulties with getting this set up so it’s probably worth sharing them here. Digital Rapids apparently don’t support live broadcasting from their hardware encoder companion software ‘Stream’ to Helix Mobile Server even though the latter supports RTP and SDP, and the encoder clearly offers these up as possible exports when you’re using the Digital Rapids AVC for Web codec profile (an optional extra purchase):

Stream encoder RTP broadcast SDP settings

What happens is that when you copy the SDP file into your Helix server’s Content/rtpencodersdp folder and try to connect to the stream pointing a media player at rtsp://helixserver/rtpencoder/live_stream.sdp it will fail, with the following error appearing in rmerror.log:

21-Sep-2010 13:47:14.062 tmplgpln(6076): 32: Error retrieving URL `rtpencoder/live_stream.sdp’ (Invalid path)

Following contact with Helix support, it turns out that the SDP file which the encoder creates does not declare the bandwidth of the audio track using a b= line as per RFC 4566 – though apparently this is an optional field. You will need to manually add this. 64Kbps results in b=AS:64 as below:

v=0
o=- 1287599430 1287599438 IN IP4 10.45.0.155
s=Session Streamed by Digital Rapids Corp Stream Server
t=0 0
a=type:broadcast
c=IN IP4 10.45.0.150/15
m=audio 20002 RTP/AVP 97
a=rtpmap:97 MPEG4-GENERIC/90000/2
a=fmtp:97 streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;bitrate=64000;config=1390
a=control:trackID=1
b=AS:64
m=video 20000 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=4D4020;sprop-parameter-sets=J01AIJZWDQ/8mApEAAADAAQAAAMAyhA=,KO8GDMg=
a=cliprect:0, 0, 416, 234
a=control:trackID=2

I also discovered that QuickTime Player is not a reliable player to use for testing as it sometimes crashes, and will not play H.264+AAC streams which VLC has no issue with. This view was echoed by Helix support.

I did have another problem – one of aspect ratios. The Digital Rapids Stream encoder will author files with garbage aspect ratio metadata if the resolution isn’t exactly 16:9 to the nearest even number of pixels. I read somewhere else that the chosen picture width should be a multiple of the x part of the aspect ratio. Here is FFMPEG’s analysis of a non-compliant resolution:

Stream #0.0(eng): Video: h264, yuv420p, 412×232 [PAR 37376:14889 DAR 401453:90053], 399 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc

and a correct 16:9 resolution:

Stream #0.0(eng): Video: h264, yuv420p, 416×234 [PAR 1:1 DAR 16:9], 399 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc

 

Wowza Media Server 2.1.2

Setting this up was also a little bit confusing, given that the encoder offers you so little insight into what the problem is if it doesn’t start. Anyway, there are excellent tutorials on the Wowza forums – this is the one we need.

As you can see from the screenshot I run Wowza Media Server on port 80 rather than 1935 to make sure that clients using restricted and proxied wifi networks (hotels etc.) are not prevented from viewing the streams:

Digital Rapids Stream RTMP media server settings

There is a gotcha though. In the forum tutorial they mention setting up security, but this is only for RTP encoders, not RTMP. I had been setting the credentials in the dialog above, but in fact you need to leave the drop-down on Server with no password.

Fortunately you can secure RTMP stream publishing in Wowza – you will need to make the following additions to live/application.xml to invoke an additional module:

       <Module>
              <Name>ModuleSecureURLParams</Name>
              <Description>ModuleSecureURLParams</Description>
              <Class>com.wowza.wms.plugin.security.ModuleSecureURLParams</Class>
       </Module> 
</Modules>
<!-- Properties defined here will be added to the IApplication.getProperties() and IApplicationInstance.getProperties() collections -->
<Properties>
       <Property>
              <Name>secureurlparams.publish</Name>
              <Value>mysecret4231.doPublish</Value>
       </Property>
</Properties>

You would then append this secret as a query string on the stream name as below:

Security settings for RTMP stream publishing

The mount point for testing with VLC for this example would be still be rtsp://10.45.0.151:80/live/mystream
or from an iPhone http://10.45.0.151/live/mystream/playlist.m3u8 (assuming AVC Level 3.0 or lower and standard AAC audio).