Tag Archives: Synology

Cross-compiling FFmpeg for Serviio 1.8 with shared libraries on Synology NAS (for 7 CPU architectures)

This method will build FFmpeg 3.2.2 for Synology DSM 6.0 with the shared libraries linked with relative paths, required for Serviio DLNA media server, using a Ubuntu/Xubuntu Desktop 14.04 virtual machine. The same method probably works with many other Linux distributions. The target CPU architectures are ARMv5, ARMv7 hard float ABI, ARM hard float ABI with NEON, Intel i686, Intel i686 Evansport, Intel x86-64, and PowerPC e500v2.

The >10MB size of the static executable is kind of getting out of hand especially on embedded systems with very limited RAM. Using shared libs means many concurrent instances of FFmpeg can use broadly the same memory footprint, and package distribution binaries can be smaller – our target systems already have libmp3lame, libm, libz, libbz2, liblzma, libpng16, librt, libgmp, libgnutls, libhogweed, libnettle, libfreetype, libfontconfig, libexpat, and libpthread which add up to several megabytes. I’m also guessing that the OS will decide when to unload them from RAM which could help when FFmpeg is being launched repeatedly in a short period of time (e.g. during library scanning).

The notes below also detail how to compile the Synology fork of FFmpeg 2.7.1 with Intel Evansport SMD (Streaming Media Drivers) hardware transcoding support. This source is itself based upon a fork by Intel which seems to form part of the Evansport SDK. Synology took a very long time (over a year) to partly comply with the GNU Public License and release this source code, despite numerous support requests, forum posts, and reports to the GPL Violations team. Their first DSM 6.0 beta was released in October 2015, and the DSM 6.0 sources were only published in late November 2016, but these remain incomplete since the kernel sources are not included.

Although the unmodified FFmpeg source code will compile successfully for PowerPC CPUs, the FFmpeg binary will core dump when running any command on a video file. I contacted Synology Support for help with this issue back in October 2012 and was given a patch to use at that time which changes the use of a CPU register in the DSP assembly code. The FFmpeg source code has changed a bit recently meaning the patch was no longer valid, but I was able to discover the appropriate location in the code and create a new working patch (code was moved from dsputil_ppc.c to blockdsp.c):

diff -rupN ffmpeg/libavcodec/ppc/blockdsp.c ffmpeg-patched/libavcodec/ppc/blockdsp.c
--- ffmpeg/libavcodec/ppc/blockdsp.c    2015-06-19 21:44:40.000000000 +0100
+++ ffmpeg-patched/libavcodec/ppc/blockdsp.c    2015-07-11 04:06:27.326911100 +0100
@@ -115,8 +115,9 @@ static long check_dcbzl_effect(void)
     memset(fakedata, 0xFF, 1024);
 
     /* Below the constraint "b" seems to mean "address base register"
-     * in gcc-3.3 / RS/6000 speaks. Seems to avoid using r0, so.... */
-    __asm__ volatile ("dcbzl %0, %1" :: "b" (fakedata_middle), "r" (zero));
+     * in gcc-3.3 / RS/6000 speaks. Seems to avoid using r0, so.... 
+    __asm__ volatile ("dcbzl %0, %1" :: "b" (fakedata_middle), "r" (zero)); */
+    __asm__ volatile ("dcbz %0, %1" :: "r" (fakedata_middle), "r" (zero));
 
     for (i = 0; i < 1024; i++)
         if (fakedata[i] == (char) 0)
 
#-----set up Synology toolchain
cd ~/Downloads
export DL_PATH="https://sourceforge.net/projects/dsgpl/files/DSM%206.0.2%20Tool%20Chains"
#-----add 32bit binary compatibility to Ubuntu 14.04 LTS (Syno toolchains are 32bit)
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
sudo apt-get install git
sudo apt-get install chrpath
#----------------------------------------------------
#-----NOW PASTE ONE OF THE FOLLOWING SIX BLOCKS TO THE TERMINAL, DEPENDING ON YOUR TARGET CPU TYPE
#-----Marvell Kirkwood mv6281/mv6282 (ARMv5te)
#-----Marvell Armada 370/375/XP (ARMv7l FPU)
#-----Mindspeed Comcerto 2000/Marvell Armada 375 (ARM Cortex-A9 NEON)
#-----Freescale PowerQUICC III MPC8533E/QorIQ P1022 (PPC e500v2 FPU)
#-----Intel Atom/Bromolow/Cedarview (i686 SSSE3)
#-----Intel Evansport (i686 SSSE3 various HW decoders, HW H.264 encoder)




#-----Marvell Kirkwood mv6281/mv6282 CPU is based on the ARMv5TE core which has DSP and Thumb instruction support
#-----However using Thumb on ARMv5 results in worse performance
#-----http://www.arm.com/products/processors/technologies/dsp-simd.php
#-----http://www.marvell.com/embedded-processors/kirkwood/assets/88f6282-3_pb.pdf
wget "${DL_PATH}/Marvell%2088F628x%20Linux%202.6.32/6281-gcc464_glibc215_88f6281-GPL.txz"
tar xvJf 6281-gcc464_glibc215_88f6281-GPL.txz
wget "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM6.0/ds.6281-6.0.dev.txz"
export DEV_DL="ds.6281-6.0.dev.txz"
export DEV_DL_ROOT="libc"
export CROSS_PREFIX=arm-marvell-linux-gnueabi
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
export TARGET=arm-marvell-linux-gnueabi
export MARCH="-march=armv5te -mtune=marvell-f -mtune=xscale"
#-----libx264:
#-----http://forum.doom9.org/showthread.php?t=160584
#-----https://mailman.videolan.org/pipermail/x264-devel/2010-December/008093.html
#-----x264 builds with NEON by default because x264 is so slow without NEON (and on any non-NEON chip) as to be useless.
#-----You can of course compile with --disable-asm on such chips, but we don't do it by default because
#-----we don't feel it's necessary to actively support chips on which x264 would be basically useless
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-asm --enable-pic --enable-strip --disable-opencl"
#-----FFmpeg:
#-----ARMv5TE CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
#-----something changed in the FFmpeg configure script and now vfp and neon optimizations have to be explicitly disabled for ARMv5TE, which doesn't have those features in any case
export FF_CONFIG="./configure --arch=arm --cpu=armv5te --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-neon --disable-vfp --disable-armv6 --disable-armv6t2 --disable-debug --enable-pthreads --enable-libshine --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"
#export FF_CONFIG="./configure --arch=arm --cpu=armv5te --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-neon --disable-vfp --disable-armv6 --disable-armv6t2 --disable-debug --enable-pthreads --enable-libshine --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-gpl --enable-libx264 --enable-gnutls --enable-nonfree --enable-libfdk_aac --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"
#-----these next two packages are needed to build libshine
sudo apt-get install automake
sudo apt-get install libtool
wget https://github.com/savonet/shine/zipball/master
unzip master
cd toots-shine-*
./bootstrap
cd ..
#-----fetch libfdk-aac fixed point math AAC encoder, binaries linked to it cannot be redistributed under GPL
#-----so it is excluded from the FFmpeg binary I built for the Serviio package
wget http://sourceforge.net/projects/opencore-amr/files/fdk-aac/fdk-aac-0.1.4.tar.gz
tar xvzf fdk-aac-0.1.4.tar.gz
cd fdk-aac-0.1.4
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..
 



#-----Marvell Armada 370/XP CPU is based on a dual issue ARMv7 core with Thumb-2, VFPv3-16, but no NEON vector unit
#-----http://www.marvell.com/embedded-processors/armada-300/assets/Marvell_ARMADA_370_SoC.pdf
#-----http://www.arm.com/products/processors/technologies/vector-floating-point.php
#-----since DSM 6.0 hard float ABI is used
wget "${DL_PATH}/Marvell%20Armada%20370%20Linux%203.2.40/armada370-gcc493_glibc220_hard-GPL.txz"
tar xvJf armada370-gcc493_glibc220_hard-GPL.txz
wget "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM6.0/ds.armada370-6.0.dev.txz"
export DEV_DL="ds.armada370-6.0.dev.txz"
export DEV_DL_ROOT="sysroot"
export CROSS_PREFIX=arm-unknown-linux-gnueabi
export TARGET=arm-unknown-linux-gnueabi
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
#-----Tune for Marvell PJ4 dual issue core (two instructions per clock)
#-----Thumb-2 can be used on ARMv6 or newer with no performance drop
export MARCH="-march=armv7-a -mcpu=marvell-pj4 -mtune=marvell-pj4 -mhard-float -mfpu=vfpv3-d16 -mthumb"
#-----libx264:
#-----http://forum.doom9.org/showthread.php?t=160584
#-----https://mailman.videolan.org/pipermail/x264-devel/2010-December/008093.html
#-----x264 builds with NEON by default because x264 is so slow without NEON (and on any non-NEON chip) as to be useless.
#-----You can of course compile with --disable-asm on such chips, but we don't do it by default because
#-----we don't feel it's necessary to actively support chips on which x264 would be basically useless
#-----http://lists.busybox.net/pipermail/buildroot/2014-October/107797.html
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-asm --enable-pic --enable-strip --disable-opencl"
#-----FFmpeg:
#-----ARMv7 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=arm --cpu=armv7-a --enable-thumb --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-neon --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----Marvell Armada 375, Armada 385, and Mindspeed Comcerto 2000 CPU are based on dual ARM Cortex-A9 cores with NEON vector unit
#-----http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA_375_SoC-01_product_brief.pdf
#-----http://www.marvell.com/embedded-processors/armada-38x/assets/A38x-Functional-Spec-PU0A.pdf
#-----http://www.mindspeed.com/products/cpe-processors/comcertoreg-2000
#-----http://www.arm.com/products/processors/cortex-a/cortex-a9.php
#-----http://www.arm.com/products/processors/technologies/neon.php
#-----Since DSM 6.0 Armada 375 finally has hard float ABI and therefore gains NEON support
wget "${DL_PATH}/Marvell%20Armada%20375%20Linux%203.2.40/armada375-gcc493_glibc220_hard-GPL.txz"
tar xvJf armada375-gcc493_glibc220_hard-GPL.txz
#-----Marvell gave all the ARMv7 toolchains the same name so rename to allow concurrent installations
mv arm-unknown-linux-gnueabi/ arm-cortexa9-linux-gnueabi/
wget "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM6.0/ds.armada375-6.0.dev.txz"
export DEV_DL="ds.armada375-6.0.dev.txz"
export DEV_DL_ROOT="sysroot"
export CROSS_PREFIX=arm-unknown-linux-gnueabi
export TARGET=arm-unknown-linux-gnueabi
export TOOLCHAIN=/usr/local/arm-cortexa9-linux-gnueabi
#-----it seems that in general neon should be used as the fpu when present unless there's a specific reason not to use it
export MARCH="-march=armv7-a -mcpu=cortex-a9 -mfpu=neon -mhard-float -mthumb"
#-----libx264:
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --enable-pic --enable-strip --disable-opencl"
#-----FFmpeg:
#-----ARMv7 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=arm --cpu=cortex-a9 --enable-thumb --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----Freescale PowerQUICC III MPC8533E/QorIQ P1022 CPUs use the PowerPC e500v2 core with Signal Processing Engine (SPE) which is not a classic FPU design, but have no AltiVec vector unit 
#-----Some QorIQ models have e500mc cores with true FPUs but these are not used in 2013 series Synology NAS
#-----http://en.wikipedia.org/wiki/QorIQ
#-----http://cache.freescale.com/files/32bit/doc/fact_sheet/QP1022FS.pdf?fpsp=1
wget "${DL_PATH}/PowerPC%20QorIQ%20Linux%202.6.32/qoriq-gcc493_glibc220_hard_qoriq-GPL.txz"
tar xvJf qoriq-gcc493_glibc220_hard_qoriq-GPL.txz
wget "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM6.0/ds.qoriq-6.0.dev.txz"
export DEV_DL="ds.qoriq-6.0.dev.txz"
export DEV_DL_ROOT="sysroot"
export CROSS_PREFIX=powerpc-e500v2-linux-gnuspe
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
export TARGET=powerpc-e500v2-linux-gnuspe
export MARCH="-mcpu=8548 -mhard-float -mfloat-gprs=double"
#-----asm disabled since QorIQ has no AltiVec
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-asm --enable-pic --enable-strip --disable-opencl"
#-----FFmpeg:#-----PowerPC e500v2 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=ppc --cpu=e500v2 --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----Intel Evansport with support for hardware decoding of VC-1, H.264, MPEG-4, MPEG2, AAC and hardware encoding of H.264
#-----http://www.anandtech.com/show/8020/synology-ds214play-intel-evansport-almost-done-right/9
#-----libx264 requires yasm 1.20
sudo apt-get install yasm
wget "${DL_PATH}/Intel%20x86%20Linux%203.2.40%20%28Evansport%29/evansport-gcc493_glibc220_linaro_i686-GPL.txz"
tar xvJf evansport-gcc493_glibc220_linaro_i686-GPL.txz
#-----Both Intel i686 toolchains the same name so rename to allow concurrent installations
mv i686-pc-linux-gnu evansport-pc-linux-gnu
wget "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM6.0/ds.evansport-6.0.dev.txz"
export DEV_DL="ds.evansport-6.0.dev.txz"
export DEV_DL_ROOT="sys-root"
export CROSS_PREFIX=i686-pc-linux-gnu
export TARGET=i686-pc-linux-gnu
export MARCH="-march=atom"
export TOOLCHAIN=/usr/local/evansport-pc-linux-gnu
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --enable-pic --enable-strip --disable-opencl"
#-----FFmpeg:
#-----Intel CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure.syno --arch=x86 --cpu=atom --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-libx264 --enable-libh264_smd --enable-smd --enable-gpl --enable-gnutls --pkg-config=pkg-config --extra-version=SMD_enabled-compiled_by_patters_for_Serviio"
export FF_CONFIG_I686="./configure --arch=x86 --cpu=atom --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-libx264 --enable-gpl --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"
#-----http://trac.ffmpeg.org/ticket/1794
#-----first ffmpeg compilation error run "make V=1" then add " -lismd_core -lismd_viddec -ljson-c" to the command line
#-----second ffmpeg compilation error run "make V=1" then add " -ljson-c" to the command line




#-----With the exception of Evansport (i686), Intel CPUs used in Synology products are all 64bit and SSSE3 capable
#-----Since DSM 6.0 x86_64 toolchain is used
#-----libx264 requires yasm 1.20
sudo apt-get install yasm
#-----unlike the others this toolchain has an external dependency on libz
#sudo apt-get install libz1:i386
wget "${DL_PATH}/Intel%20x86%20Linux%203.10.77%20%28Pineview%29/x64-gcc493_glibc220_linaro_x86_64-GPL.txz"
tar xvJf x64-gcc493_glibc220_linaro_x86_64-GPL.txz
wget "https://sourceforge.net/projects/dsgpl/files/toolkit/DSM6.0/ds.cedarview-6.0.dev.txz"
export DEV_DL="ds.cedarview-6.0.dev.txz"
export DEV_DL_ROOT="sys-root"
export CROSS_PREFIX=x86_64-pc-linux-gnu
export TARGET=x86_64-pc-linux-gnu
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
#-----We don't really care how lame is built since ffmpeg will use the version in DSM.
#-----The DSM version is built without asm optimizations but these only make a 5% difference, and have been known to introduce instability
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --enable-pic --enable-strip"
#-----FFmpeg:
#-----Intel CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=x86 --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-libvorbis --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#----------------------------------------------------
#-----script continues
sudo mv `echo $TOOLCHAIN | sed -r "s%^.*/(.*$)%\1%"` /usr/local
#-----some of the toolchains have bad permissions
sudo chmod -R 0755 ${TOOLCHAIN}
export PATH=${TOOLCHAIN}/bin:$PATH
export AR=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ar
export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
export CC=${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc
export CXX=${TOOLCHAIN}/bin/${CROSS_PREFIX}-g++
export LD=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ld
export LDSHARED="${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc -shared "
export RANLIB=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ranlib
export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"
export LDFLAGS="-L${TOOLCHAIN}/lib"
export PKG_CONFIG_PATH="${TOOLCHAIN}/lib/pkgconfig"


#-----set up all our dependent libraries from the Synology dev environment
mkdir ${CROSS_PREFIX}-dev
cd ${CROSS_PREFIX}-dev
tar xfJv ../${DEV_DL}

mkdir $TOOLCHAIN/lib/pkgconfig/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/expat.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/expat_external.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libexpat.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/expat.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/expat.pc

cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/fontconfig/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libfontconfig.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/fontconfig.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/fontconfig.pc

cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/freetype2/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libfreetype.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/freetype2.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/freetype2.pc
sed -i "s|^exec_prefix=.*$|exec_prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/freetype2.pc
sed -i "s|^libdir=.*$|libdir=${TOOLCHAIN}/lib|" $TOOLCHAIN/lib/pkgconfig/freetype2.pc
sed -i "s|^includedir=.*$|includedir=${TOOLCHAIN}/include/freetype2|" $TOOLCHAIN/lib/pkgconfig/freetype2.pc

cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/gmp.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libgmp.* $TOOLCHAIN/lib/

cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/gnutls/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libgnutls.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/gnutls.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/gnutls.pc

cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/lame/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libmp3lame.* $TOOLCHAIN/lib/

cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/nettle/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libnettle.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libhogweed.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/nettle.pc $TOOLCHAIN/lib/pkgconfig/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/hogweed.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/nettle.pc
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/hogweed.pc

cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/png.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/pngconf.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/pnglibconf.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libpng.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/libpng.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/libpng.pc

cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/zlib.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/zconf.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libz.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/zlib.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/zlib.pc

cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/json-c/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libjson-c.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/json-c.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/json-c.pc

#-----libbz2 and libxml-2.0 are dependencies of fontconfig
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/bzlib.h $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libbz2.* $TOOLCHAIN/lib/

cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/libxml2/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libxml2.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/libxml-2.0.pc $TOOLCHAIN/lib/pkgconfig/
sed -i "s|^prefix=.*$|prefix=${TOOLCHAIN}|" $TOOLCHAIN/lib/pkgconfig/libxml-2.0.pc

#-----liblzma is a dependency of libass
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/lzma.h $TOOLCHAIN/include/
cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/lzma/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/liblzma.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/liblzma.pc $TOOLCHAIN/lib/pkgconfig/

#-----libpng16 is a dependency of freetype2
cp -R usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/libpng16/ $TOOLCHAIN/include/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/libpng16.* $TOOLCHAIN/lib/
cp usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/pkgconfig/libpng16.pc $TOOLCHAIN/lib/pkgconfig/

#-----End of syno devkit extraction
cd ..

#-----shine shared lib for ARMv5 only (no FPU)
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && cd toots-shine-*
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && make
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && make install
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && cd ..

#-----fribidi shared lib
wget http://fribidi.org/download/fribidi-0.19.6.tar.bz2
tar xvjf fribidi-0.19.6.tar.bz2
cd fribidi-0.19.6
#-----patch fribidi source since it won't compile - some sort of page size function is no longer available
#-----error on Intel: fribidi-run.c:70: error: 'PAGE_SIZE' undeclared (first use in this function)
#-----http://armbedded.eu/node/56
#-----http://lists.debian.org/debian-glibc/2006/10/msg00169.html
#-----http://my.safaribooksonline.com/book/operating-systems-and-server-administration/linux/0596009585/advanced-file-i-o/mapping_files_into_memory
[ "${CROSS_PREFIX}" == "i686-pc-linux-gnu" ] && sed -i "s|include <asm/page.h>|include <unistd.h>\n#   define PAGE_SIZE sysconf(_SC_PAGESIZE)|" lib/common.h
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..

#-----fontconfig shared lib
wget http://www.freedesktop.org/software/fontconfig/release/fontconfig-2.11.1.tar.gz
tar xvzf fontconfig-2.11.1.tar.gz
cd fontconfig-2.11.1
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..

#-----libass shared lib
wget https://github.com/libass/libass/releases/download/0.13.4/libass-0.13.4.tar.xz
tar xvJf libass-0.13.4.tar.xz
cd libass-0.13.4
LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,\'\$\$ORIGIN\' ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..
#-----Although libass would link correctly to older FFmpeg versions, this mod is now necessary:
#-----http://ffmpeg.org/pipermail/ffmpeg-user/2012-November/010906.html
#-----no longer needed, note here in case of regression in future versions 
#sed -i -r "s/(^Libs: .*$)/\1 -lfontconfig -lfribidi -lfreetype -lexpat/" $TOOLCHAIN/lib/pkgconfig/libass.pc

#-----libx264 shared lib
wget https://download.videolan.org/x264/snapshots/last_stable_x264.tar.bz2
tar xvjf last_stable_x264.tar.bz2
cd x264-snapshot-*
#-----NEON ASM needs to be built by GCC assembler not AS
[ "${TOOLCHAIN}" == "/usr/local/arm-cortexa9-linux-gnueabi" ] && export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc
#-----Intel ASM needs to be built by yasm assembler
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && export AS=yasm
[ "${TOOLCHAIN}" == "/usr/local/i686-pc-linux-gnu" ] && export AS=yasm
[ "${TOOLCHAIN}" == "/usr/local/x86_64-pc-linux-gnu" ] && export AS=yasm
$X264_CONFIG
#-----back out assembler changes
AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
make
make install
cd ..

#-----libx265 shared lib
#wget https://bitbucket.org/multicoreware/x265/downloads/x265_2.1.tar.gz
#tar xvzf x265_2.1.tar.gz
#cd x265_2.1

#-----librtmp shared lib
git clone git://git.ffmpeg.org/rtmpdump
cd rtmpdump
make CROSS_COMPILE=${TOOLCHAIN}/bin/${CROSS_PREFIX}- SYS=posix prefix=${TOOLCHAIN} CRYPTO=GNUTLS INC=-I${TOOLCHAIN}/include XLDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,\'\$\$ORIGIN\'
#------librtmp.pc will have the wrong lib dependencies defined unless GNUTLS is specified again here
make install prefix=${TOOLCHAIN} CRYPTO=GNUTLS
cd ..

#-----libspeex shared lib
wget http://downloads.xiph.org/releases/speex/speex-1.2rc2.tar.gz
tar xvzf speex-1.2rc2.tar.gz
cd speex-1.2rc2
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..

#-----libogg shared lib
wget http://downloads.xiph.org/releases/ogg/libogg-1.3.2.tar.xz
tar xvJf libogg-1.3.2.tar.xz
cd libogg-1.3.2
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..

#-----libvorbis shared lib
wget http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.5.tar.xz
tar xvJf libvorbis-1.3.5.tar.xz
cd libvorbis-1.3.5
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..

#-----Evansport only - don't run these two lines until right before compiling FFmpeg since some of the includes will break other stuff
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cp -R ${CROSS_PREFIX}-dev/usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/include/intelce-utilities $TOOLCHAIN/include
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cp -R ${CROSS_PREFIX}-dev/usr/local/$TARGET/$TARGET/${DEV_DL_ROOT}/usr/lib/intelce-utilities $TOOLCHAIN/lib

#-----FFmpeg (Evansport uses a separate Synology-modified source bundle)
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || wget https://www.ffmpeg.org/releases/ffmpeg-3.2.2.tar.xz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || tar xvJf ffmpeg-3.2.2.tar.xz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || cd ffmpeg-3.2.2
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && wget https://sourceforge.net/projects/dsgpl/files/Synology%20NAS%20GPL%20Source/8451branch/evansport-source/ffmpeg-2.7.x.txz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && tar xvJf ffmpeg-2.7.x.txz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd ffmpeg-2.7.x

#-----PowerPC-specific fix originally provided by Synology Support in Oct 2012, adapted after code re-org by ffmpeg
[ "${CROSS_PREFIX}" == "powerpc-e500v2-linux-gnuspe" ] && wget http://dl.dropboxusercontent.com/u/1188556/ffmpeg-syno-ppc.patch
[ "${CROSS_PREFIX}" == "powerpc-e500v2-linux-gnuspe" ] && patch --verbose -p1 < ffmpeg-syno-ppc.patch

#-----Evansport SMD driver scheduler uses inline asm which breaks compilation without -fasm
#-----http://stackoverflow.com/questions/18574943/gcc-inline-assembler-implicit-function-declaration-on-powerpc-arch
#-----http://ffmpeg.org/pipermail/ffmpeg-devel/2008-October/056533.html
#-----http://www.tldp.org/HOWTO/Assembly-HOWTO/gcc.html
#-----we also need to add the intelce-utlities locations of the SMD libraries
#-----libx264 will fail to compile with this added to include path, so only add it just before compiling ffmpeg
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] \
  && export CFLAGS="${CFLAGS} -fasm -I${TOOLCHAIN}/include/json-c -I${TOOLCHAIN}/include/intelce-utilities -I${TOOLCHAIN}/include/intelce-utilities/linux_user"

#-----Evansport SMD functionality is disabled by default
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && sed -i "1i#define SYNO_EVANSPORT_SMD" synoconfig.h
#-----Evansport ffmpeg will by default skip DTS audio tracks for transcoding and will refuse to encode AC3
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && wget https://dl.dropboxusercontent.com/u/1188556/ffmpeg-2.7.1-syno-dts-skip.patch
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && patch --verbose -p1 < ffmpeg-2.7.1-syno-dts-skip.patch

#-----set library linker to use relative library search path (../lib and .)
#-----tricky shell escaping of $ORIGIN is required http://itee.uq.edu.au/~daniel/using_origin/
#-----http://stackoverflow.com/questions/6562403/i-dont-understand-wl-rpath-wl
#-----however although FFmpeg gets a correct rpath, the downstream libraries get a value of "RIGIN/../lib:RIGIN"
#-----owing to some additional variable substitution happening - easier to build with XORIGIN and use chrpath later
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] \
  || LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,XORIGIN/../lib:XORIGIN,-rpath-link,${TOOLCHAIN}/lib ${FF_CONFIG} \
  && LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-L${TOOLCHAIN}/lib/intelce-utilities,-rpath,XORIGIN/../lib:XORIGIN,-rpath-link,${TOOLCHAIN}/lib,-rpath-link,${TOOLCHAIN}/lib/intelce-utilities ${FF_CONFIG}

#-----external libs (librtmp, libass) don't seem to link to ffmpeg (check ffmpeg binary with objdump -x)
#-----this is in fact normal because LDFLAGS as defined in config.mak contains '--as-needed'
#-----https://wiki.gentoo.org/wiki/Project:Quality_Assurance/As-needed
#-----e.g. FFmpeg does not contain any symbols from librtmp, but libavformat does
#-----use ldd on the target system and the full dependency chain will be shown
make
make install

#-----remove intelce-utilities from include path
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && rm -rf ${TOOLCHAIN}/include/intelce-utilities
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && rm -rf ${TOOLCHAIN}/lib/intelce-utilities 

mkdir native-${CROSS_PREFIX}
cd native-${CROSS_PREFIX}

#-----evansport FFmpeg 2.7.1
#-----fix up rpath settings
sudo chrpath -r '$ORIGIN/../lib:$ORIGIN' ${TOOLCHAIN}/bin/ffmpeg
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavcodec.so.56
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavdevice.so.56
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavfilter.so.5
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavformat.so.56
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavutil.so.54
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libpostproc.so.53
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswresample.so.1
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswscale.so.3
cp ${TOOLCHAIN}/lib/libavcodec.so.56 .
cp ${TOOLCHAIN}/lib/libavdevice.so.56 .
cp ${TOOLCHAIN}/lib/libavfilter.so.5 .
cp ${TOOLCHAIN}/lib/libavformat.so.56 .
cp ${TOOLCHAIN}/lib/libavutil.so.54 .
cp ${TOOLCHAIN}/lib/libpostproc.so.53 .
cp ${TOOLCHAIN}/lib/libswresample.so.1 .
cp ${TOOLCHAIN}/lib/libswscale.so.3 .
cp ${TOOLCHAIN}/bin/ffmpeg .
cp ${TOOLCHAIN}/bin/fc-cache .
cp ${TOOLCHAIN}/lib/libass.so.5 .
cp ${TOOLCHAIN}/lib/libfribidi.so.0 .
cp ${TOOLCHAIN}/lib/libogg.so.0 .
cp ${TOOLCHAIN}/lib/librtmp.so.1 .
cp ${TOOLCHAIN}/lib/libspeex.so.1 .
cp ${TOOLCHAIN}/lib/libvorbis.so.0 .
cp ${TOOLCHAIN}/lib/libvorbisenc.so.2 .
cp ${TOOLCHAIN}/lib/libx264.so.148 .


#-----FFmpeg 3.2.2
#-----fix up rpath settings
sudo chrpath -r '$ORIGIN/../lib:$ORIGIN' ${TOOLCHAIN}/bin/ffmpeg
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavcodec.so.57
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavdevice.so.57
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavfilter.so.6
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavformat.so.57
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavutil.so.55
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libpostproc.so.54
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswresample.so.2
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswscale.so.4
cp ${TOOLCHAIN}/lib/libavcodec.so.57 .
cp ${TOOLCHAIN}/lib/libavdevice.so.57 .
cp ${TOOLCHAIN}/lib/libavfilter.so.6 .
cp ${TOOLCHAIN}/lib/libavformat.so.57 .
cp ${TOOLCHAIN}/lib/libavutil.so.55 .
cp ${TOOLCHAIN}/lib/libpostproc.so.54 .
cp ${TOOLCHAIN}/lib/libswresample.so.2 .
cp ${TOOLCHAIN}/lib/libswscale.so.4 .
cp ${TOOLCHAIN}/bin/ffmpeg .
cp ${TOOLCHAIN}/bin/fc-cache .
cp ${TOOLCHAIN}/lib/libass.so.5 .
cp ${TOOLCHAIN}/lib/libfribidi.so.0 .
cp ${TOOLCHAIN}/lib/libogg.so.0 .
cp ${TOOLCHAIN}/lib/librtmp.so.1 .
cp ${TOOLCHAIN}/lib/libspeex.so.1 .
cp ${TOOLCHAIN}/lib/libvorbis.so.0 .
cp ${TOOLCHAIN}/lib/libvorbisenc.so.2 .
cp ${TOOLCHAIN}/lib/libx264.so.148 .
[ ${TOOLCHAIN} == "/usr/local/arm-none-linux-gnueabi" ] && cp ${TOOLCHAIN}/lib/libshine.so.3 .
#cp ${TOOLCHAIN}/lib/libfdk-aac.so.1 .
sudo chmod +x *
sudo chown root:root *
XZ_OPT=-9 tar -cvJf ffmpeg-${CROSS_PREFIX}.tar.xz *
 

Here’s how the library dependencies look. There are no links to any libraries in /opt, and DSM-bundled libraries in /lib are linked to where possible. All the other new libraries are kept within the Serviio lib folder (../lib relative to the ffmpeg binary).

DS214> ldd /var/packages/Serviio/target/bin/ffmpeg
	linux-gate.so.1 (0xffffe000)
	libavdevice.so.57 => /volume1/@appstore/Serviio/bin/../lib/libavdevice.so.57 (0xb7708000)
	libavfilter.so.6 => /volume1/@appstore/Serviio/bin/../lib/libavfilter.so.6 (0xb74fd000)
	libavformat.so.57 => /volume1/@appstore/Serviio/bin/../lib/libavformat.so.57 (0xb72bd000)
	libavcodec.so.57 => /volume1/@appstore/Serviio/bin/../lib/libavcodec.so.57 (0xb5ee8000)
	libpostproc.so.54 => /volume1/@appstore/Serviio/bin/../lib/libpostproc.so.54 (0xb5ec1000)
	libswresample.so.2 => /volume1/@appstore/Serviio/bin/../lib/libswresample.so.2 (0xb5ea2000)
	libswscale.so.4 => /volume1/@appstore/Serviio/bin/../lib/libswscale.so.4 (0xb5e11000)
	libavutil.so.55 => /volume1/@appstore/Serviio/bin/../lib/libavutil.so.55 (0xb5d8b000)
	libm.so.6 => /lib/libm.so.6 (0xb5d42000)
	libpthread.so.0 => /lib/libpthread.so.0 (0xb5d25000)
	libc.so.6 => /lib/libc.so.6 (0xb5b77000)
	libass.so.5 => /volume1/@appstore/Serviio/bin/../lib/libass.so.5 (0xb5b33000)
	librtmp.so.1 => /volume1/@appstore/Serviio/bin/../lib/librtmp.so.1 (0xb5b14000)
	libz.so.1 => /lib/libz.so.1 (0xb5afb000)
	libgnutls.so.28 => /lib/libgnutls.so.28 (0xb59f0000)
	libbz2.so.1.0 => /lib/libbz2.so.1.0 (0xb59df000)
	libdl.so.2 => /lib/libdl.so.2 (0xb59da000)
	libx264.so.148 => /volume1/@appstore/Serviio/bin/../lib/libx264.so.148 (0xb583c000)
	libvorbisenc.so.2 => /volume1/@appstore/Serviio/bin/../lib/libvorbisenc.so.2 (0xb57aa000)
	libvorbis.so.0 => /volume1/@appstore/Serviio/bin/../lib/libvorbis.so.0 (0xb5770000)
	libspeex.so.1 => /volume1/@appstore/Serviio/bin/../lib/libspeex.so.1 (0xb5747000)
	libmp3lame.so.0 => /lib/libmp3lame.so.0 (0xb56cf000)
	liblzma.so.5 => /lib/liblzma.so.5 (0xb56a3000)
	/lib/ld-linux.so.2 (0xb7718000)
	libfribidi.so.0 => /volume1/@appstore/Serviio/bin/../lib/libfribidi.so.0 (0xb5688000)
	libfontconfig.so.1 => /lib/libfontconfig.so.1 (0xb5645000)
	libfreetype.so.6 => /lib/libfreetype.so.6 (0xb5586000)
	libhogweed.so.2 => /lib/libhogweed.so.2 (0xb5555000)
	libnettle.so.4 => /lib/libnettle.so.4 (0xb5521000)
	libgmp.so.10 => /lib/libgmp.so.10 (0xb54ac000)
	libogg.so.0 => /volume1/@appstore/Serviio/bin/../lib/libogg.so.0 (0xb54a2000)
	libxml2.so.2 => /lib/libxml2.so.2 (0xb5321000)
	libpng16.so.16 => /lib/libpng16.so.16 (0xb52e4000)
DS214>
 

Here’s how the even more complex Evansport build with hardware transcoding support looks:

DS214Play> ldd /var/packages/Serviio/target/bin/ffmpeg
	linux-gate.so.1 (0xffffe000)
	libavdevice.so.56 => /volume1/@appstore/Serviio/bin/../lib/libavdevice.so.56 (0xb77c0000)
	libavfilter.so.5 => /volume1/@appstore/Serviio/bin/../lib/libavfilter.so.5 (0xb7686000)
	libavformat.so.56 => /volume1/@appstore/Serviio/bin/../lib/libavformat.so.56 (0xb748d000)
	libavcodec.so.56 => /volume1/@appstore/Serviio/bin/../lib/libavcodec.so.56 (0xb630e000)
	libpostproc.so.53 => /volume1/@appstore/Serviio/bin/../lib/libpostproc.so.53 (0xb62e7000)
	libswresample.so.1 => /volume1/@appstore/Serviio/bin/../lib/libswresample.so.1 (0xb62cb000)
	libswscale.so.3 => /volume1/@appstore/Serviio/bin/../lib/libswscale.so.3 (0xb624a000)
	libavutil.so.54 => /volume1/@appstore/Serviio/bin/../lib/libavutil.so.54 (0xb61d4000)
	libffmpeg_plugin.so => /lib/libffmpeg_plugin.so (0xb61be000)
	libm.so.6 => /lib/libm.so.6 (0xb6175000)
	libismd_core.so => /lib/libismd_core.so (0xb6168000)
	libismd_viddec.so => /lib/libismd_viddec.so (0xb6160000)
	libjson-c.so.2 => /lib/libjson-c.so.2 (0xb6153000)
	libpthread.so.0 => /lib/libpthread.so.0 (0xb6137000)
	libc.so.6 => /lib/libc.so.6 (0xb5f88000)
	libass.so.5 => /volume1/@appstore/Serviio/bin/../lib/libass.so.5 (0xb5f44000)
	librtmp.so.1 => /volume1/@appstore/Serviio/bin/../lib/librtmp.so.1 (0xb5f25000)
	libz.so.1 => /lib/libz.so.1 (0xb5f0c000)
	libgnutls.so.28 => /lib/libgnutls.so.28 (0xb5e02000)
	libbz2.so.1.0 => /lib/libbz2.so.1.0 (0xb5df0000)
	libx264.so.148 => /volume1/@appstore/Serviio/bin/../lib/libx264.so.148 (0xb5c52000)
	libvorbisenc.so.2 => /volume1/@appstore/Serviio/bin/../lib/libvorbisenc.so.2 (0xb5bc0000)
	libvorbis.so.0 => /volume1/@appstore/Serviio/bin/../lib/libvorbis.so.0 (0xb5b87000)
	libspeex.so.1 => /volume1/@appstore/Serviio/bin/../lib/libspeex.so.1 (0xb5b5e000)
	libmp3lame.so.0 => /lib/libmp3lame.so.0 (0xb5ae5000)
	liblzma.so.5 => /lib/liblzma.so.5 (0xb5ab9000)
	libdl.so.2 => /lib/libdl.so.2 (0xb5ab4000)
	libxml2.so.2 => /lib/libxml2.so.2 (0xb5933000)
	libismd_demux.so => /lib/libismd_demux.so (0xb5928000)
	libismd_vidrend.so => /lib/libismd_vidrend.so (0xb591f000)
	libismd_clock.so => /lib/libismd_clock.so (0xb591d000)
	libismd_audio.so => /lib/libismd_audio.so (0xb590e000)
	libismd_videnc.so => /lib/libismd_videnc.so (0xb590a000)
	libismd_vidpproc.so => /lib/libismd_vidpproc.so (0xb5903000)
	libismdmessage.so => /lib/libismdmessage.so (0xb58ff000)
	libsven.so => /lib/libsven.so (0xb57b9000)
	libpal.so => /lib/libpal.so (0xb57b7000)
	libosal.so => /lib/libosal.so (0xb57ae000)
	libplatform_config.so => /lib/libplatform_config.so (0xb57ab000)
	libsystem_utils.so => /lib/libsystem_utils.so (0xb57a8000)
	libismd_clock_recovery.so => /lib/libismd_clock_recovery.so (0xb57a4000)
	/lib/ld-linux.so.2 (0xb77d0000)
	libfribidi.so.0 => /volume1/@appstore/Serviio/bin/../lib/libfribidi.so.0 (0xb5789000)
	libfontconfig.so.1 => /lib/libfontconfig.so.1 (0xb5747000)
	libfreetype.so.6 => /lib/libfreetype.so.6 (0xb5688000)
	libhogweed.so.2 => /lib/libhogweed.so.2 (0xb5657000)
	libnettle.so.4 => /lib/libnettle.so.4 (0xb5623000)
	libgmp.so.10 => /lib/libgmp.so.10 (0xb55ad000)
	libogg.so.0 => /volume1/@appstore/Serviio/bin/../lib/libogg.so.0 (0xb55a4000)
	libpng16.so.16 => /lib/libpng16.so.16 (0xb5567000)
DS214Play> 
 
 

SANE Backends network scanning package for Synology NAS

UPDATE – SANE version 1.0.28, new frontend AirSane package, and updated frontend software info below

SANE-package

Many scanners last considerably longer than they enjoy driver updates for. Manufacturers are seldom willing to update drivers for new operating systems. The transition to 64bit was a similar obstacle. Furthermore Microsoft dropped support for the TWAIN scanning API in favour of the WIA (Windows Image Acquisition) for 64bit versions of its OS, and Apple removed TWAIN in macOS 10.9 Mavericks, superseded by ICA (Image Capture Architecture). These conditions have resulted in a mountain of perfectly good hardware being thrown away unnecessarily. I recently found myself without any modern drivers for my own Canon CanoScan LiDE 20, so I decided to do something about the problem. Now you can use this package to share your obsolete scanner from your Synology NAS.

SANE is an open source scanning API which has a fantastic wealth of device support. SANE’s differentiator is that it separates the drivers (backend) from the presentation and the scanning (frontend). This is ideal for a headless device like a NAS, and it also frees the frontend to be quite generic rather than highly customized to each device.

 

Device Support

HP all-in-one models are not supported by the package since their backend hpaio is not part of a standard SANE build, and it requires significant additional dependencies. Scanner backends which require libieee1284 are not supported by the package (canon_pp, hpsj5s, and mustek_pp), neither is backend dell1600n_net which needed a TIFF library and in fact only supports one device. You can verify which backend is required for your model of scanner using the SANE Project search engine. Expand below for the full list of backends which this Synology package supports:

Backend support in Synology package as of version 1.0.28-003
------------------------------------------------------------

Note that some backends default to disabled in /var/packages/sane-backends/target/etc/sane.d/dll.conf
This can be because they may slow down detection routines
This is a default SANE config unrelated to the Synology pacakge
Detection is faster if unused backends are commented out in this file

abaton
agfafocus
apple
artec
artec_eplus48u
as6e
avision
bh
canon
canon630u
canon_dr
cardscan
coolscan
coolscan2
coolscan3
dc25
dc210
dc240
dmc
epjitsu
epson
epson2
epsonds
fujitsu
genesys (except on armv5tel which lacks C++11 toolchain support)
gt68xx
hp
hp3500
hp3900
hp4200
hp5400
hp5590
hpljm1005
hs2p
ibm
kodak
kodakaio
kvs1025
kvs20xx
kvs40xx
leo
lexmark
ma1509
magicolor
matsushita
microtek
microtek2
mustek
mustek_usb
mustek_usb2
nec
net
niash
p5
pie
pieusb
pixma
plustek
plustek_pp
qcam
ricoh
ricoh2
rts8891
s9036
sceptre
sharp
sm3600
sm3840
snapscan
sp15c
st400
stv680
tamarack
teco1
teco2
teco3
test
u12
umax
umax_pp
umax1220u
xerox_mfp
 

Synology Package Installation

  • In Synology DSM’s Package Center, click Settings and add my package repository:
    Add Package Repository
  • The repository will push its certificate automatically to the NAS, which is used to validate package integrity. Set the Trust Level to Synology Inc. and trusted publishers:
    Trust Level
  • Now browse the Community section in Package Center to install SANE Backends.
    Community-packages
    The repository only displays packages which are compatible with your specific model of NAS. If you don’t see SANE Backends in the list, then either your NAS model or your DSM version are not supported at this time. DSM 6.0 is the minimum supported version for this package.
  • When the package is started it will attempt to detect a scanner and output the results to the package log. If your scanner is listed in the log, that is all that is required at the backend.
  •  

    SANE Frontend for the Computer

    Since I have a variety of systems at home I spent some time investigating the various options. I list them in order of quality of experience below (updated in September 2019). You will need to manually specify the IP address of your NAS when setting up the scanner. Be aware that the scanner will need redefining on the computer if you change USB port later for instance. No credentials are needed for connection by default.

    • AirSane (macOS Mojave and earlier) – Hands down the most elegant SANE frontend solution, and one which I have now packaged separately for Synology. AirSane will autodetect any available SANE scanner and will advertise it via Bonjour to the macOS Image Capture application as a shared AirScan device. No configuration is required – it just works! Even if you don’t have a Mac it offers a simple WebUI for scanning.
      AirSane is in active development at the time of writing in September 2019, in fact its author SimulPiscator fixed a few issues I discovered while testing prior to packaging it.
    • WiaSane (Windows 10 and earlier) – Marc Hoersken has developed a SANE to WIA bridge which works very well, presenting the remote device to Windows Device Manager as if it is locally attached. Scanning using the Windows Fax and Scan application is seamless (preview, partial selections etc.), and other software with WIA support can be used, for instance NAPS2 (Not Another PDF Scanner 2) which offers OCR, and Paint.net, both of which are free. Although the latest version of WiaSane does now include a signed device driver which works in Windows 10, the device configuration wizard only seems to work at install time, and each time the scanner changes USB ID it has to be removed and reinstalled.
      WiaSane was last updated in February 2017
    • SwingSane As a Java application SwingSane will run on most operating systems. It is full-featured but it has some rough edges which caused me to think it was broken when I first evaluated it. This explanatory video by SwingSane’s author Roland Quast explains some of the non-obvious aspects.
      The most confusing thing is that the Settings tab was not offering me the appropriate settings for my scanner – for instance I could not set the resolution or the page size. Roland explains in the video that these main settings were specifically targeting Epson scanners, and that for all other models Custom Settings should be used, in particular he recommends to select Check Options which queries the backend driver for its parameters. The next problem is that with custom settings SwingSane will not select the whole bed of the scanner by default, only a small selection. If you export the settings and inspect the XML file you can discover the constraint values (maxima) for the br-x and br-y values in millimetres (bottom right of the selection rectangle), set those and save the settings. This is far too counter-intuitive when surely these values can be determined automatically by the software.
      One strength of SwingSane however, is that all the scans produced in the current session are stored in the preview tab and can be output to the same multi-page PDF (AirSane and Image Capture also permits this). Finally, the file requestor confused me on macOS (more so than on Windows). There is a dropdown for File Format which is blank (why show this?). I was expecting to specify the filename, but this is determined by the scan batch so really the file requestor is asking for a destination folder only. This should be stated in the window title. Furthermore on macOS you cannot save once you have browsed inside a folder – you have to navigate to the parent and select the destination folder without navigating to its contents! This is really not obvious and leads you to suspect the software is broken.
      SwingSane was last updated in April 2015.
    • SaneWinDS (Windows 10 and earlier) – This frontend does not hook into WIA so you will have to use it for the scan acquisition rather than acquiring from inside applications. It does a good job of presenting the available options, and will send the scans directly to the default application for the destination file type. There is no driver component – it’s just a software application so it works fine even on Windows 10.
      Author Alec Skelly last updated SaneWinDS in April 2015.
     

    Notes

    • The package can be stopped and started in Package Center but really this is to launch and provide the log output of sane-find-scanner and scanimage -L in the easiest way for non-technical users. saned uses the inetd super-server daemon so it is only loaded and invoked when a SANE frontend makes contact on port 6566. This means that really there is no performance nor memory penalty for leaving the package running all the time, even on very resource limited systems.
    • Unfortunately Synology decided to remove the ability to view package log files via the DSM 6.2 GUI and they’ve provided no information in the Developer Guide on how to integrate with the new Log Center apps. I lack the motivation to reverse engineer functionality the company significantly alters with each DSM release, so troubleshooting this package does now require some familiarity with SSH or SCP. The package log file is /var/packages/sane-backends/target/var/sane-find-scanner.log
    • Now that I have built the package with libusb 1.0 rather than using DSM’s version 0.1, it does automatically detect scanners that have been hot-plugged.
    • By default access is granted to SANE frontends originating from any IP address (0.0.0.0/0 and ::/0 for IPv6). This can be restricted by modifying /var/packages/sane-backends/target/etc/sane.d/saned.conf, or by configuring the DSM Firewall.
    • By default connections to the SANE backends do not require credentials. If you would like to secure devices, create the text file /var/packages/sane-backends/target/etc/sane.d/saned.users, specifying credentials in the format “user:password:backend”, e.g.:
      swingsane:fqkg3h328rge:plustek
    • The DSM 6.0 toolchains provide pre-built development libraries for certain components such as avahi and dbus which are complex to build. Although I did not build sane-backends with support for avahi (due to stability issues), these libraries are needed by the AirSane package hence the new requirement for DSM 6.0 at a minimum. Some much older macOS SANE frontends make the assumption that avahi will be used and so do not allow you to statically define a target IP address, so they cannot be used with this package.
    • Although most of the library search path behaviour can be controlled at compilation time, unfortunately the backend drivers are searched for using a static path, and sane-backends insists on saving its configuration in a static location. I am always careful that my packages do not damage the system they are installed on so I have kept all of this contained within /var/packages/sane-backends/target by cross-compiling in this same location on my development Linux VM. My build steps are documented on GitHub.
     

    Package scripts

    For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology 3rd Party Developer Guide.

    installer.sh

    #!/bin/sh
    
    #--------SANE Backends installer script
    #--------package maintained at pcloadletter.co.uk
    
    
    SYNO_CPU_ARCH="`uname -m`"
    [ "${SYNOPKG_DSM_ARCH}" == "comcerto2k" ] && SYNO_CPU_ARCH="armneon"
    [ "${SYNOPKG_DSM_ARCH}" == "armada375" ] && SYNO_CPU_ARCH="armneon"
    [ "${SYNOPKG_DSM_ARCH}" == "armada38x" ] && SYNO_CPU_ARCH="armneon"
    [ "${SYNOPKG_DSM_ARCH}" == "alpine" ] && SYNO_CPU_ARCH="armneon"
    [ "${SYNOPKG_DSM_ARCH}" == "alpine4k" ] && SYNO_CPU_ARCH="armneon"
    [ "${SYNOPKG_DSM_ARCH}" == "monaco" ] && SYNO_CPU_ARCH="armneon"
    NATIVE_BINS_URL="https://syno.pcloadletter.co.uk/bin/sane1.0.28-native-${SYNO_CPU_ARCH}.tar.xz"   
    NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
    INSTALL_FILES="${NATIVE_BINS_URL}"
    COMMENT="# Synology SANE Backends Package"
    TEMP_FOLDER="`find / -maxdepth 2 -path '/volume?/@tmp' | head -n 1`"
    PUBLIC_FOLDER="`synoshare --get public | sed -r "/Path/!d;s/^.*\[(.*)\].*$/\1/"`"
    source /etc/profile
    
    
    preinst ()
    {
      cd ${TEMP_FOLDER}
      for WGET_URL in ${INSTALL_FILES}
      do
        WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
        [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
        wget ${WGET_URL}
        if [ $? != 0 ]; then
          if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
            cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
          else
            echo "There was a problem downloading ${WGET_FILENAME} from the official download link, " >> $SYNOPKG_TEMP_LOGFILE
            echo "which was \"${WGET_URL}\" " >> $SYNOPKG_TEMP_LOGFILE
            echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. " >> $SYNOPKG_TEMP_LOGFILE
            exit 1
          fi
        fi
      done
    
      exit 0
    }
    
    
    postinst ()
    {
      #extract CPU-specific binaries
      cd ${SYNOPKG_PKGDEST}
      tar xJf ${TEMP_FOLDER}/${NATIVE_BINS_FILE} && rm ${TEMP_FOLDER}/${NATIVE_BINS_FILE}
    
      #allow access to saned by default (this can be restricted by IP/subnet)
      echo 0.0.0.0/0 >> /var/packages/${SYNOPKG_PKGNAME}/target/etc/sane.d/saned.conf
      echo ::/0 >> /var/packages/${SYNOPKG_PKGNAME}/target/etc/sane.d/saned.conf
    
      #add firewall config
      /usr/syno/bin/servicetool --install-configure-file --package /var/packages/${SYNOPKG_PKGNAME}/conf/${SYNOPKG_PKGNAME}.sc > /dev/null
    
      exit 0
    }
    
    
    preuninst ()
    {
      `dirname $0`/stop-start-status stop
    
      exit 0
    }
    
    
    postuninst ()
    {
      #remove system configuration changes
      sed -i "/${COMMENT}/d" /etc/services
      sed -i "/${COMMENT}/d" /etc/inetd.conf
    
      #remove firewall config
      if [ "${SYNOPKG_PKG_STATUS}" == "UNINSTALL" ]; then
        /usr/syno/bin/servicetool --remove-configure-file --package ${SYNOPKG_PKGNAME}.sc > /dev/null
      fi
    
      exit 0
    }
    
    
    preupgrade ()
    {
      `dirname $0`/stop-start-status stop
    
      exit 0
    }
    
    
    postupgrade ()
    {
      exit 0
    }
    
     

    start-stop-status.sh

    #!/bin/sh
    
    #--------SANE Backends start-stop-status script
    #--------package maintained at pcloadletter.co.uk
    
    PKG_FOLDER="/var/packages/sane-backends"
    DLOG="${PKG_FOLDER}/target/var/sane-find-scanner.log"
    COMMENT="# Synology SANE Backends Package"
    source /etc/profile
    source /root/.profile
    
    
    EnvCheck ()
    #updates to DSM will reset these changes so check them each startup
    {
      #/etc/services should contain 1 line added by this package tagged with trailing comments
      COUNT=`grep -c "${COMMENT}$" /etc/services`
      if [ $COUNT != 1 ]; then
    
        #remove any existing mods
        sed -i "/${COMMENT}/d" /etc/services
    
        #make the required changes
        echo "sane-port       6566/tcp                        ${COMMENT}" >> /etc/services
      fi
      #/etc/inetd.conf should contain 1 line added by this package tagged with trailing comments
      COUNT=`grep -c "${COMMENT}$" /etc/inetd.conf`
      if [ $COUNT != 1 ]; then
    
        #remove any existing mods
        sed -i "/${COMMENT}/d" /etc/inetd.conf
    
        #make the required changes      
        echo "sane-port stream tcp    nowait  root    ${PKG_FOLDER}/target/sbin/saned    saned  ${COMMENT}" >> /etc/inetd.conf
      fi
      #/usr/syno/etc/synoservice.d/pkgctl-sane-backends.cfg should contain an upstart entry for inetd
      COUNT=`grep -c "inetd" /usr/syno/etc/synoservice.d/pkgctl-sane-backends.cfg`
      if [ $COUNT != 1 ]; then
        sed -i -r "s/(\"pkgctl-sane-backends\")/\1,\"inetd\"/" /usr/syno/etc/synoservice.d/pkgctl-sane-backends.cfg
      fi
    }
    
    case $1 in
      start)
        EnvCheck
        reload inetd
        echo `date "+%F %R"` startup. `${PKG_FOLDER}/target/bin/sane-find-scanner -q` > ${PKG_FOLDER}/target/var/sane-find-scanner.log
        echo `date "+%F %R"` `${PKG_FOLDER}/target/bin/scanimage -L` >> ${PKG_FOLDER}/target/var/sane-find-scanner.log
        exit 0
      ;;
    
      stop)
        #remove any existing config changes
        sed -i "/${COMMENT}/d" /etc/services
        sed -i "/${COMMENT}/d" /etc/inetd.conf
        killall -q saned
        reload inetd
        exit 0
      ;;
    
      status)
        grep -q "${COMMENT}" /etc/inetd.conf && exit 0 || exit 1
      ;;
    
      log)
        echo "${DLOG}"
        exit 0
      ;;
    
    esac
    
     

    Changelog:

    • 0003 15/Sep/19 – Updated to SANE 1.0.28 and libusb 1.0.23 (DSM still uses libusb 0.1), fixed inetd start issue, allowed connections on IPv6 interfaces by default, added support for new Synology product models, added aarch64 build for 64bit ARM CPUs, published a separate package AirSane to automatically present SANE scanners to the macOS Image Capture application, updated the section on frontends in this page. Old package version available here for DSM 5.0 users.
    • 0002 20/Jul/16 – Added support for Marvell Armada 385 SoC
    • 0001 16/Dec/15 – Initial public release
     
     

Cross-compiling nano with UTF-8 support for Synology

export CROSS_PREFIX=i686-pc-linux-gnu
export TARGET=i686-pc-linux-gnu
export TOOLCHAIN=/usr/local/evansport-pc-linux-gnu
export AR=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ar
export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
export CC=${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc
export CXX=${TOOLCHAIN}/bin/${CROSS_PREFIX}-g++
export LD=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ld
export LDSHARED="${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc -shared "
export RANLIB=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ranlib
export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"
export LDFLAGS="-L${TOOLCHAIN}/lib"
export PKG_CONFIG_PATH="${TOOLCHAIN}/lib/pkgconfig"

#DSM 5.0 already includes libncursesw so we can link the nano binary to this
#fetch the source which was extracted from the Synology DSM 5.0 source tarball
wget https://dl.dropboxusercontent.com/u/1188556/ncurses-5.x.zip
unzip ncurses-5.x.zip
cd ncurses-5.x
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --with-shared --without-normal --without-progs --without-debug --enable-widec
make
make install
cd ..
wget http://www.nano-editor.org/dist/v2.2/nano-2.2.6.tar.gz
tar xvzf nano-2.2.6.tar.gz
cd nano-2.2.6
#src/nano.h:92:20: fatal error: curses.h: No such file or directory
#http://blog.csdn.net/alienspy/article/details/24135761
wget http://dl.dropboxusercontent.com/u/1188556/nano.h.patch
patch nano.h < nano.h.patch
CFLAGS="${CFLAGS} -I${TOOLCHAIN}/include/ncurses" ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-utf8 --disable-nls --enable-color --enable-extra --enable-multibuffer --enable-nanorc
#sudo apt-get install texinfo
sed -i -e "s%^CPPFLAGS = -I/usr/include/ncursesw%CPPFLAGS = -I${TOOLCHAIN}/include/ncurses%" Makefile
sed -i -e "s%^CPPFLAGS = -I/usr/include/ncursesw%CPPFLAGS = -I${TOOLCHAIN}/include/ncurses%" src/Makefile
make
make install
#now install it to /usr/local/bin
 

nano.h.patch

--- src/nano.h.old      2014-09-17 12:16:15.006206100 +0100
+++ src/nano.h  2014-09-17 12:19:01.226206100 +0100
@@ -86,10 +86,10 @@
 #define KEY_IC SL_KEY_IC
 /* Ncurses support. */
 #elif defined(HAVE_NCURSES_H)
-#include <ncurses.h>
+#include <ncursesw/ncurses.h>
 #else
 /* Curses support. */
-#include <curses.h>
+#include <ncursesw/curses.h>
 #endif /* CURSES_H */

 #ifdef ENABLE_NLS
 

~/.nanorc

# BASIC
syntax "BAS" "\.(bas|BAS|bas.txt)$"
color red "^ +[0-9]* "
color red "\b(GO TO|GO SUB) ([0-9]+)\b"
color yellow "\b(AND|NOT|OR)\b"
color yellow "(|=|<|>|<=|>=|<>)"
color magenta "\b(FOR [a-z]|NEXT [a-z]|GO TO|GO SUB|RETURN)\b"
color brightwhite "\b(IF|THEN)\b"
color green ""(\\.|[^\"])*""
color brightcyan "\bREM .*$"

##############################################################################
#
# Syntax highlighting for XML files
#
# Author:  Josef 'Jupp' Schugt, jupp(a)rubyforge.org
# License: GPL 2  or later
#
# Version: 2004-02-25
#
##############################################################################
syntax "ml" ".*\.([jrs]?html?|xml|sgml?)$"
color white "^.+$"
color green  start="<" end=">"
color cyan   "<[^> ]+"
color cyan   ">"
color yellow start="<!DOCTYPE" end="[/]?>"
color yellow start="<!--" end="-->"
color red    "&[^;]*;"

##############################################################################
#
# Syntax highlighting for HTTP codes
#
# Author:  Josef 'Jupp' Schugt, jupp(a)rubyforge.org
# License: GPL 2  or later
#
# Version: 2004-02-25
#
##############################################################################
syntax "urls"
color brightmagenta   "^.*$"
color cyan      "^(1[0-9][0-9]|20[256]|30[45]).*$"
color green      "^20[03].*$"
color brightyellow   "^(201|30[0-37]).*$"
color brightred      "^(204|[45][0-9][0-9]|666).*$"

##############################################################################
#
# Syntax highlighting for CSS files
#
# Author:  Simon Rupf, simon.rupf(a)int-ag.ch
# License: GPL 2  or later
#
# Version: 2005-02-14
#
##############################################################################
syntax "css" "\.css$"
color brightred     "."
color brightyellow  start="\{" end="\}"
color brightwhite           start=":" end="[;^\{]"
color brightblue    ":active|:focus|:hover|:link|:visited|:link|:after|:before|$"
color brightblue    start="\/\*" end="\\*/"
color green         ";|:|\{|\}"

syntax "sh" "\.sh$"
#icolor brightgreen "^[0-9A-Z_]+\(\)"
#color green "\b(case|do|done|elif|else|esac|exit|fi|for|function|if|in|local|read|return|select|shift|then|time|until|while)\b"
#color green "(\{|\}|\(|\)|\;|\]|\[|`|\\|\$|<|>|!|=|&|\|)"
#color green "-[Ldefgruwx]\b"
#color green "-(eq|ne|gt|lt|ge|le|s|n|z)\b"
#color brightblue "\b(cat|cd|chmod|chown|cp|echo|env|export|grep|install|let|ln|make|mkdir|mv|rm|sed|set|tar|touch|umask|unset)\b"
icolor brightwhite "\$\{?[0-9A-Z_!@#$*?-]+\}?"
color brightcyan "(^|[[:space:]])#.*$"
#color brightyellow ""(\\.|[^"])*"" "'(\\.|[^'])*'"
#color cyan ""(\\.|[^"])*"" "'(\\.|[^'])*'"
#color ,green "[[:space:]]+$"

Cross-compiling FFmpeg for Serviio with shared libraries on Synology NAS (for 7 CPU architectures)

This method will build FFmpeg 2.7.1 with the shared libraries (linked with relative paths) required for Serviio DLNA media server, using a Ubuntu/Xubuntu Desktop 14.04 virtual machine, however the same method probably works with many other Linux distributions. The target CPU architectures are ARMv5, ARMv7, ARM hard float ABI with NEON, Intel i686, Intel x64, and PowerPC e500v2.

The >10MB size of the static executable is kind of getting out of hand especially on embedded systems with very limited RAM. Using shared libs means many concurrent instances of FFmpeg can use broadly the same memory footprint, and package distribution binaries can be smaller – our target systems already have libmp3lame, libm, libz, librt, libgmp, libssl, libcrypto, libfreetype, libexpat, and libpthread which add up to several megabytes. I’m also guessing that the OS will decide when to unload them from RAM which could help when FFmpeg is being launched repeatedly in a short period of time (e.g. during library scanning).

The notes below also detail how to compile the Synology fork of FFmpeg 2.0.2 with Intel Evansport SMD (Streaming Media Drivers) hardware transcoding support. This source is itself based upon a fork by Intel which seems to form part of the Evansport SDK. It was not straightforward to compile, and now that I have released Serviio for Synology with hardware transcoding support I’m sharing my method in the hope it can help others. This older FFmpeg needs a patch to fix a long standing bug with DTS audio remuxing (reported by me) where the stream ID was incorrectly set:

--- ffmpeg-2.2.1/libavformat/mpegts.h
+++ ffmpeg-2.2.1-ticket1398/libavformat/mpegts.h
@@ -57,7 +57,8 @@
 #define STREAM_TYPE_VIDEO_DIRAC     0xd1
 
 #define STREAM_TYPE_AUDIO_AC3       0x81
-#define STREAM_TYPE_AUDIO_DTS       0x8a
+#define STREAM_TYPE_AUDIO_DTS       0x82
+#define STREAM_TYPE_AUDIO_TRUEHD    0x83
 
 typedef struct MpegTSContext MpegTSContext;
 
--- ffmpeg-2.2.1/libavformat/mpegtsenc.c
+++ ffmpeg-2.2.1-ticket1398/libavformat/mpegtsenc.c
@@ -311,6 +311,12 @@
         case AV_CODEC_ID_AC3:
             stream_type = STREAM_TYPE_AUDIO_AC3;
             break;
+        case AV_CODEC_ID_DTS:
+            stream_type = STREAM_TYPE_AUDIO_DTS;
+            break;
+        case AV_CODEC_ID_TRUEHD:
+            stream_type = STREAM_TYPE_AUDIO_TRUEHD;
+            break;
         default:
             stream_type = STREAM_TYPE_PRIVATE_DATA;
             break;

Although the unmodified FFmpeg source code will compile successfully for PowerPC CPUs, the FFmpeg binary will core dump when running any command on a video file. I contacted Synology Support for help with this issue back in October 2012 and was given a patch to use at that time which changes the use of a CPU register in the DSP assembly code. The FFmpeg source code has changed a bit recently meaning the patch was no longer valid, but I was able to discover the appropriate location in the code and create a new working patch (code was moved from dsputil_ppc.c to blockdsp.c):

diff -rupN ffmpeg/libavcodec/ppc/blockdsp.c ffmpeg-patched/libavcodec/ppc/blockdsp.c
--- ffmpeg/libavcodec/ppc/blockdsp.c	2015-06-19 21:44:40.000000000 +0100
+++ ffmpeg-patched/libavcodec/ppc/blockdsp.c	2015-07-11 04:06:27.326911100 +0100
@@ -115,8 +115,9 @@ static long check_dcbzl_effect(void)
     memset(fakedata, 0xFF, 1024);
 
     /* Below the constraint "b" seems to mean "address base register"
-     * in gcc-3.3 / RS/6000 speaks. Seems to avoid using r0, so.... */
-    __asm__ volatile ("dcbzl %0, %1" :: "b" (fakedata_middle), "r" (zero));
+     * in gcc-3.3 / RS/6000 speaks. Seems to avoid using r0, so.... 
+    __asm__ volatile ("dcbzl %0, %1" :: "b" (fakedata_middle), "r" (zero)); */
+    __asm__ volatile ("dcbz %0, %1" :: "r" (fakedata_middle), "r" (zero));
 
     for (i = 0; i < 1024; i++)
         if (fakedata[i] == (char) 0)
 
#-----set up Synology toolchain
cd ~/Downloads
#export DL_PATH="http://sourceforge.net/projects/dsgpl/files/DSM%205.0%20Tool%20Chains"
#export DL_PATH="http://sourceforge.net/projects/dsgpl/files/DSM%20Beta%205.2%20Tool%20Chains"
export DL_PATH="http://sourceforge.net/projects/dsgpl/files/DSM%205.2%20Tool%20Chains"
#-----add 32bit binary compatibility to Ubuntu 14.04 LTS (Syno toolchains are 32bit)
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
sudo apt-get install git
sudo apt-get install chrpath
#----------------------------------------------------
#-----NOW PASTE ONE OF THE FOLLOWING SIX BLOCKS TO THE TERMINAL, DEPENDING ON YOUR TARGET CPU TYPE
#-----Marvell Kirkwood mv6281/mv6282 (ARMv5te)
#-----Marvell Armada 370/375/XP (ARMv7l FPU)
#-----Mindspeed Comcerto 2000/Marvell Armada 375 (ARM Cortex-A9 NEON)
#-----Freescale PowerQUICC III MPC8533E/QorIQ P1022 (PPC e500v2 FPU)
#-----Intel Atom/Bromolow/Cedarview (i686 SSSE3)
#-----Intel Evansport (i686 SSSE3 various HW decoders, HW H.264 encoder)




#-----Marvell Kirkwood mv6281/mv6282 CPU is based on the ARMv5TE core which has DSP and Thumb instruction support
#-----However using Thumb on ARMv5 results in worse performance
#-----http://www.arm.com/products/processors/technologies/dsp-simd.php
#-----http://www.marvell.com/embedded-processors/kirkwood/assets/88f6282-3_pb.pdf
wget "${DL_PATH}/Marvell%2088F628x%20Linux%202.6.32/6281-gcc464_glibc215_88f6281-GPL.txz"
tar xvJf 6281-gcc464_glibc215_88f6281-GPL.txz
#-----Marvell gave the ARMv5 and ARMv7 toolchains the same name so rename to allow concurrent installations
mv arm-marvell-linux-gnueabi/ arm-none-linux-gnueabi/
#-----these next two packages are needed to build libshine
sudo apt-get install automake
sudo apt-get install libtool
export CROSS_PREFIX=arm-marvell-linux-gnueabi
export TOOLCHAIN=/usr/local/arm-none-linux-gnueabi
export TARGET=arm-marvell-linux-gnueabi
export MARCH="-march=armv5te -mtune=marvell-f -mtune=xscale"
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
#-----libx264:
#-----http://forum.doom9.org/showthread.php?t=160584
#-----https://mailman.videolan.org/pipermail/x264-devel/2010-December/008093.html
#-----x264 builds with NEON by default because x264 is so slow without NEON (and on any non-NEON chip) as to be useless.
#-----You can of course compile with --disable-asm on such chips, but we don't do it by default because
#-----we don't feel it's necessary to actively support chips on which x264 would be basically useless
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-asm --enable-pic --enable-strip"
#-----FFmpeg:
#-----ARMv5TE CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
#-----something changed in the FFmpeg configure script and now vfp and neon optimizations have to be explicitly disabled for ARMv5TE, which doesn't have those features in any case
export FF_CONFIG="./configure --arch=arm --cpu=armv5te --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-neon --disable-vfp --disable-armv6 --disable-armv6t2 --disable-debug --enable-pthreads --enable-libshine --enable-librtmp --enable-libass --enable-libspeex --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"
#export FF_CONFIG="./configure --arch=arm --cpu=armv5te --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-neon --disable-vfp --disable-armv6 --disable-armv6t2 --disable-debug --enable-pthreads --enable-libshine --enable-librtmp --enable-libass --enable-libspeex --enable-gpl --enable-libx264 --enable-gnutls --enable-nonfree --enable-libfdk_aac --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"
wget https://github.com/savonet/shine/zipball/master
unzip master
cd toots-shine-*
./bootstrap
cd ..
#-----fetch libfdk-aac fixed point math AAC encoder, binaries linked to it cannot be redistributed under GPL
#-----so it is excluded from the FFmpeg binary I built for the Serviio package
wget http://sourceforge.net/projects/opencore-amr/files/fdk-aac/fdk-aac-0.1.4.tar.gz
tar xvzf fdk-aac-0.1.4.tar.gz
cd fdk-aac-0.1.4
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..
 



#-----Marvell Armada 370/XP CPU is based on a dual issue ARMv7 core with Thumb-2, VFPv3-16, but no NEON vector unit
#-----http://www.marvell.com/embedded-processors/armada-300/assets/Marvell_ARMADA_370_SoC.pdf
#-----http://www.arm.com/products/processors/technologies/vector-floating-point.php
#-----Marvell Armada 375 CPU is based on dual ARM Cortex-A9 cores with NEON vector unit, but there is no Synology toolchain with NEON support
#-----Synology seems to have used this toolchain to build DSM for the Armada 375
#-----http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA_375_SoC-01_product_brief.pdf
wget "${DL_PATH}/Marvell%20Armada%20370%20Linux%203.2.40/armada370-gcc464_glibc215_hard_armada-GPL.txz"
tar xvJf armada370-gcc464_glibc215_hard_armada-GPL.txz
export CROSS_PREFIX=arm-marvell-linux-gnueabi
export TARGET=arm-marvell-linux-gnueabi
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
#-----Tune for Marvell PJ4 dual issue core (two instructions per clock)
#-----Thumb-2 can be used on ARMv6 or newer with no performance drop
export MARCH="-march=armv7-a -mcpu=marvell-pj4 -mtune=marvell-pj4 -mhard-float -mfpu=vfpv3-d16 -mthumb"
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
#-----libx264:
#-----http://forum.doom9.org/showthread.php?t=160584
#-----https://mailman.videolan.org/pipermail/x264-devel/2010-December/008093.html
#-----x264 builds with NEON by default because x264 is so slow without NEON (and on any non-NEON chip) as to be useless.
#-----You can of course compile with --disable-asm on such chips, but we don't do it by default because
#-----we don't feel it's necessary to actively support chips on which x264 would be basically useless
#-----http://lists.busybox.net/pipermail/buildroot/2014-October/107797.html
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-asm --enable-pic --enable-strip"
#-----FFmpeg:
#-----ARMv7 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=arm --cpu=armv7-a --enable-thumb --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-neon --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----Mindspeed Comcerto 2000 CPU is based on dual ARM Cortex-A9 cores with NEON vector unit
#-----http://www.mindspeed.com/products/cpe-processors/comcertoreg-2000
#-----http://www.arm.com/products/processors/cortex-a/cortex-a9.php
#-----http://www.arm.com/products/processors/technologies/neon.php
#-----Armada 375 toolchain uses glibc 2.15, Comcerto 2000 uses glibc 2.17 
wget "${DL_PATH}/Mindspeed%20Comcerto%202000%20Linux%203.2.40/comcerto2k-gcc464_glibc217_neon_comcerto-GPL.txz"
tar xvJf comcerto2k-gcc464_glibc217_neon_comcerto-GPL.txz
export CROSS_PREFIX=arm-cortexa9-linux-gnueabi
export TARGET=arm-cortexa9-linux-gnueabi
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
#-----it seems that in general neon should be used as the fpu when present unless there's a specific reason not to use it
export MARCH="-march=armv7-a -mcpu=cortex-a9 -mfpu=neon -mhard-float -mthumb"
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
#-----libx264:
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --enable-pic --enable-strip"
#-----FFmpeg:
#-----ARMv7 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=arm --cpu=cortex-a9 --enable-thumb --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----Freescale PowerQUICC III MPC8533E/QorIQ P1022 CPUs use the PowerPC e500v2 core with Signal Processing Engine (SPE) which is not a classic FPU design, but have no AltiVec vector unit 
#-----Some QorIQ models have e500mc cores with true FPUs but these are not used in 2013 series Synology NAS
#-----http://en.wikipedia.org/wiki/QorIQ
#-----http://cache.freescale.com/files/32bit/doc/fact_sheet/QP1022FS.pdf?fpsp=1
wget "${DL_PATH}/PowerPC%20QorIQ%20Linux%202.6.32/qoriq-gcc4374_eglibc2874_qoriq-GPL.txz"
tar xvJf qoriq-gcc4374_eglibc2874_qoriq-GPL.txz
export CROSS_PREFIX=powerpc-none-linux-gnuspe
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
export TARGET=powerpc-none-linux-gnuspe
export MARCH="-mcpu=8548 -mhard-float -mfloat-gprs=double"
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
#-----asm disabled since QorIQ has no AltiVec
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-asm --enable-pic --enable-strip"
#-----FFmpeg:#-----PowerPC e500v2 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=ppc --cpu=e500v2 --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----With the exception of Evansport (i686), Intel CPUs used in Synology products are all 64bit and SSSE3 capable but existing DSM libs are 32bit at time of writing (DSM 4.1), so use the i686 toolchain rather than x86_64
#-----libx264 requires yasm 1.20
sudo apt-get install yasm
#-----unlike the others this toolchain has an external dependency on libz
sudo apt-get install libz1:i386
wget "${DL_PATH}/Intel%20x86%20Linux%203.2.40%20%28Pineview%29/x64-gcc463_glibc213_i686_patched-GPL.txz"
tar xvJf x64-gcc463_glibc213_i686_patched-GPL.txz
export CROSS_PREFIX=i686-pc-linux-gnu
export TARGET=i686-pc-linux-gnu
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
#-----We don't really care how lame is built since ffmpeg will use the version in DSM.
#-----The DSM version is built without asm optimizations but these only make a 5% difference, and have been known to introduce instability
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --enable-pic --enable-strip"
#-----FFmpeg:
#-----Intel CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=x86 --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----Intel Evansport with support for hardware decoding of VC-1, H.264, MPEG-4, MPEG2, AAC and hardware encoding of H.264
#-----http://www.anandtech.com/show/8020/synology-ds214play-intel-evansport-almost-done-right/9
#-----http://ukdl.synology.com/download/spk/VideoStation/1.3-0547/
#-----libx264 requires yasm 1.20
sudo apt-get install yasm
wget "${DL_PATH}/Intel%20x86%20Linux%203.2.40%20%28Evansport%29/evansport-gcc463_glibc213_i686_patched-GPL.txz"
tar xvJf evansport-gcc463_glibc213_i686_patched-GPL.txz
#-----Both Intel toolchains the same name so rename to allow concurrent installations
mv i686-pc-linux-gnu evansport-pc-linux-gnu
#-----Synology headers and libs for the Intel Streaming Media Driver support
wget http://sourceforge.net/projects/dsgpl/files/Synology%20NAS%20GPL%20Source/5565branch/evansport-chroot.txz
mkdir evansport-chroot
cd evansport-chroot
tar xvJf ../evansport-chroot.txz
sudo cp -R lib/intelce-utilities/ ../evansport-pc-linux-gnu/lib
sudo cp -R include/intelce-utilities/ ../evansport-pc-linux-gnu/include
cd ..
#end of Intel SMD section
export CROSS_PREFIX=i686-pc-linux-gnu
export TARGET=i686-pc-linux-gnu
export MARCH="-march=atom"
export TOOLCHAIN=/usr/local/evansport-pc-linux-gnu
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --enable-pic --enable-strip"
#-----FFmpeg:
#-----Intel CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure.syno --arch=x86 --cpu=atom --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-libx264 --enable-libh264_smd --enable-smd --enable-gpl --enable-gnutls --pkg-config=pkg-config --extra-version=SMD_enabled_with_DTS_remux_fix-compiled_by_patters_for_Serviio"
#-----implicit function declaration issue for libavcodec/ismd_ffmpeg_audio_util.o, libavcodec/libx264.o, ffmpeg.o
#-----http://stackoverflow.com/questions/11561261/cc1-warnings-being-treated-as-errors
#-----http://stackoverflow.com/questions/13986301/is-gcc-doing-implicit-function-declarations-incorrectly-in-c99-mode
#-----how to workaround an error using make V=1 (re-issue gcc command, removing -Werror=implicit-function-declaration from command line)
#-----http://trac.ffmpeg.org/ticket/1794
#-----ffmpeg_g and ffprobe_g libavcodec/libavcodec.so: undefined reference to `json_object_array_length' used by (libavcodec/syno_trans_loading.c)
#-----add -ljson-c to the linker parameters




#-----Intel Core-i3 CPUs are 64bit and SSSE3 and seem to have their DSM built using x86_64
#-----libx264 requires yasm 1.20
sudo apt-get install yasm
wget "${DL_PATH}/Intel%20x86%20Linux%203.2.40%20%28Pineview%29/x64-gcc473_glibc217_x86_64-GPL.txz"
tar xvJf x64-gcc473_glibc217_x86_64-GPL.txz
export CROSS_PREFIX=x86_64-pc-linux-gnu
export TARGET=x86_64-pc-linux-gnu
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
#-----We don't really care how lame is built since ffmpeg will use the version in DSM.
#-----The DSM version is built without asm optimizations but these only make a 5% difference, and have been known to introduce instability
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --enable-pic --enable-strip"
#-----FFmpeg:
#-----Intel CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=x86 --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libspeex --enable-gpl --enable-libx264 --enable-gnutls --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"





#----------------------------------------------------
#-----script continues
sudo mv `echo $TOOLCHAIN | sed -r "s%^.*/(.*$)%\1%"` /usr/local
#-----some of the toolchains have bad permissions
sudo chmod -R 0755 ${TOOLCHAIN}
export PATH=${TOOLCHAIN}/bin:$PATH
export AR=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ar
export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
export CC=${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc
export CXX=${TOOLCHAIN}/bin/${CROSS_PREFIX}-g++
export LD=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ld
export LDSHARED="${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc -shared "
export RANLIB=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ranlib
export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"
export LDFLAGS="-L${TOOLCHAIN}/lib"
export PKG_CONFIG_PATH="${TOOLCHAIN}/lib/pkgconfig"


#-----fetch Synology's DSM 5565 sources for certain libraries we want to link against
wget https://dl.dropboxusercontent.com/u/1188556/DSM5565-src.tar.xz
tar xvJf DSM5565-src.tar.xz

#-----libz shared lib (toolchain only has static lib)
cd zlib-1.x
./configure --prefix=${TOOLCHAIN}
#-----owing to a bug in the configure script, libz cross compiles without a SONAME (check it with objdump -x)
#-----http://forums.gentoo.org/viewtopic-p-7156392.html
sed -i "s/\(^LDSHARED=.*$\)/\1 -Wl,-soname,libz.so.1,--version-script,zlib.map/" Makefile
make
make install
cd ..

#-----LAME shared lib
cd lame-398
$LAME_CONFIG
make
make install
cd ..

#-----expat shared lib
cd expat-2.x
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..

#-----freetype shared lib
cd freetype-2.x
#-----disable external zlib library since it appears Synology compiled DSM-included libfreetype binary this way
#-----failure to do this results in a hang during fc-cache generation - this took a significant effort to determine
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --with-zlib=no --enable-shared --disable-static
make
make install
cd ..

#-----shine shared lib for ARMv5 only (no FPU)
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && cd toots-shine-*
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && make
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && make install
[ "${TOOLCHAIN}" == "/usr/local/arm-none-linux-gnueabi" ] && cd ..

#-----misc shared libs for Intel Evansport FFmpeg build only
#-----use libx264 version from Synology sources since the Evansport ffmpeg may have specific dependencies on this version
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd x264
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && $X264_CONFIG
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && make
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && make install
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd ..
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd libxml2-2.x
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --without-python
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && make
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && make install
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd ..
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd json-c-0.x
#-----undefined reference to `rpl_malloc' when linking ffmpeg
#-----http://rickfoosusa.blogspot.co.uk/2011/11/howto-fix-undefined-reference-to.html 
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && ac_cv_func_malloc_0_nonnull=yes ac_cv_func_realloc_0_nonnull=yes ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu  --enable-shared --disable-static
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && make
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && make install
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd ..

#-----fribidi shared lib
wget http://fribidi.org/download/fribidi-0.19.6.tar.bz2
tar xvjf fribidi-0.19.6.tar.bz2
cd fribidi-0.19.6
#-----patch fribidi source since it won't compile - some sort of page size function is no longer available
#-----error on Intel: fribidi-run.c:70: error: 'PAGE_SIZE' undeclared (first use in this function)
#-----http://armbedded.eu/node/56
#-----http://lists.debian.org/debian-glibc/2006/10/msg00169.html
#-----http://my.safaribooksonline.com/book/operating-systems-and-server-administration/linux/0596009585/advanced-file-i-o/mapping_files_into_memory
[ "${CROSS_PREFIX}" == "i686-pc-linux-gnu" ] && sed -i "s|include <asm/page.h>|include <unistd.h>\n#   define PAGE_SIZE sysconf(_SC_PAGESIZE)|" lib/common.h
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..

#-----fontconfig shared lib
wget http://www.freedesktop.org/software/fontconfig/release/fontconfig-2.11.1.tar.gz
tar xvzf fontconfig-2.11.1.tar.gz
cd fontconfig-2.11.1
LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,\'\$\$ORIGIN/../lib:\$\$ORIGIN\' ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..

#-----libass shared lib
wget https://github.com/libass/libass/releases/download/0.12.1/libass-0.12.1.tar.xz
tar xvJf libass-0.12.1.tar.xz
cd libass-0.12.1
LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,\'\$\$ORIGIN\' ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..
#-----Although libass would link correctly to older FFmpeg versions, this mod is now necessary:
#-----http://ffmpeg.org/pipermail/ffmpeg-user/2012-November/010906.html
#-----no longer needed, note here in case of regression in future versions 
#sed -i -r "s/(^Libs: .*$)/\1 -lfontconfig -lfribidi -lfreetype -lexpat/" $TOOLCHAIN/lib/pkgconfig/libass.pc

#-----libx264 shared lib
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || wget ftp://ftp.videolan.org/pub/x264/snapshots/last_x264.tar.bz2
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || tar xvjf last_x264.tar.bz2
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || cd x264-snapshot-*
#-----NEON ASM needs to be built by GCC assembler not AS
[ "${TOOLCHAIN}" == "/usr/local/arm-cortexa9-linux-gnueabi" ] && export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc
#-----intel ASM needs to be built by yasm assembler
[ "${TOOLCHAIN}" == "/usr/local/i686-pc-linux-gnu" ] && export AS=yasm
[ "${TOOLCHAIN}" == "/usr/local/x86_64-pc-linux-gnu" ] && export AS=yasm
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || $X264_CONFIG
#-----back out assembler changes
[ "${TOOLCHAIN}" == "/usr/local/arm-cortexa9-linux-gnueabi" ] && export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
[ "${TOOLCHAIN}" == "/usr/local/i686-pc-linux-gnu" ] && export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
[ "${TOOLCHAIN}" == "/usr/local/x86_64-pc-linux-gnu" ] && export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || make
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || make install
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || cd ..

#-----GMP shared lib
wget https://gmplib.org/download/gmp/gmp-6.0.0a.tar.xz
tar -xJf gmp-6.0.0a.tar.xz
cd gmp-6.0.0
#-----disable ARM Thumb - GMP doesn't compile in Thumb mode even using this patch
#-----https://gmplib.org/list-archives/gmp-discuss/2014-March/005537.html
export MARCH=`echo $MARCH | sed -e "s/ -mthumb//"`
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"
[ "${TOOLCHAIN}" == "/usr/local/powerpc-none-linux-gnuspe" ] && mv ${TOOLCHAIN}/bin/flex ${TOOLCHAIN}/bin/flex.old
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
#-----re-enable thumb-2 mode where applicable
[ "${TOOLCHAIN}" == "/usr/local/arm-cortexa9-linux-gnueabi" ] && export MARCH="${MARCH} -mthumb" 
[ "${TOOLCHAIN}" == "/usr/local/arm-marvell-linux-gnueabi" ] && export MARCH="${MARCH} -mthumb"
[ "${TOOLCHAIN}" == "/usr/local/armv7-marvell-linux-gnueabi-hard" ] && export MARCH="${MARCH} -mthumb"
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}" 
cd ..

#-----nettle shared lib
#-----version compatibility issues with gnutls: https://crux.nu/bugs/?do=details&task_id=1023
wget https://ftp.gnu.org/gnu/nettle/nettle-2.7.1.tar.gz
tar xvzf nettle-2.7.1.tar.gz
cd nettle-2.7.1
LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,\'\$\$ORIGIN\' ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-openssl
make
make install
#-----this will fail to complete but it's not important: *** No rule to make target `../libnettle.a', needed by `sexp-conv'. Stop.
cd ..

#-----gnutls shared lib
wget ftp://ftp.gnutls.org/gcrypt/gnutls/v3.3/gnutls-3.3.14.tar.xz
tar -xJf gnutls-3.3.14.tar.xz 
cd gnutls-3.3.14
#-----use DSM's existing root CA certs bundle
LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,\'\$\$ORIGIN\' ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --with-default-trust-store-file=/etc/ssl/certs/ca-certificates.crt --enable-threads=posix --disable-doc --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
sed -i -r "s/(^Libs: .*$)/\1 -lnettle -lhogweed -lgmp/" $TOOLCHAIN/lib/pkgconfig/gnutls.pc
cd ..

#-----librtmp shared lib
git clone git://git.ffmpeg.org/rtmpdump
cd rtmpdump
make CROSS_COMPILE=${TOOLCHAIN}/bin/${CROSS_PREFIX}- SYS=posix prefix=${TOOLCHAIN} CRYPTO=GNUTLS INC=-I${TOOLCHAIN}/include XLDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,\'\$\$ORIGIN\'
#------librtmp.pc will have the wrong lib dependencies defined unless GNUTLS is specified again here
make install prefix=${TOOLCHAIN} CRYPTO=GNUTLS
cd ..

#-----libspeex shared lib
wget http://downloads.xiph.org/releases/speex/speex-1.2rc2.tar.gz
tar xvzf speex-1.2rc2.tar.gz
cd speex-1.2rc2
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..

#-----FFmpeg (Evansport uses a separate Synology-modified source bundle)
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || wget https://www.ffmpeg.org/releases/ffmpeg-2.7.1.tar.gz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || tar xvzf ffmpeg-2.7.1.tar.gz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] || cd ffmpeg-2.7.1
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && wget https://dl.dropboxusercontent.com/u/1188556/DSM5565-ffmpeg-2.0.x-src.tar.xz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && tar xvJf DSM5565-ffmpeg-2.0.x-src.tar.xz
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && cd ffmpeg-2.0.x

#-----Apply DTS audio remux fix for hardware players (incorrect stream ID was set in older versions of FFmpeg)
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && wget http://dl.dropboxusercontent.com/u/1188556/ffmpeg-ticket1398.patch
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && patch --verbose -p1 < ffmpeg-ticket1398.patch

#-----PowerPC-specific fix originally provided by Synology Support in Oct 2012, adapted after code re-org by ffmpeg
[ "${CROSS_PREFIX}" == "powerpc-none-linux-gnuspe" ] && wget http://dl.dropboxusercontent.com/u/1188556/ffmpeg-syno-ppc.patch
[ "${CROSS_PREFIX}" == "powerpc-none-linux-gnuspe" ] && patch --verbose -p1 < ffmpeg-syno-ppc.patch

#-----Evansport SMD driver scheduler uses inline asm which breaks compilation without -fasm
#-----http://stackoverflow.com/questions/18574943/gcc-inline-assembler-implicit-function-declaration-on-powerpc-arch
#-----http://ffmpeg.org/pipermail/ffmpeg-devel/2008-October/056533.html
#-----http://www.tldp.org/HOWTO/Assembly-HOWTO/gcc.html
#-----we also need to add the intelce-utlities locations of the SMD libraries
#-----libx264 will fail to compile with this added to include path, so only add it just before compiling ffmpeg
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] \
  && export CFLAGS="${CFLAGS} -fasm -I${TOOLCHAIN}/include/json-c -I${TOOLCHAIN}/include/intelce-utilities -I${TOOLCHAIN}/include/intelce-utilities/linux_user"

#-----Evansport SMD functionality is disabled by default
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && sed -i "1i#define SYNO_ENABLE_SMD" synoconfig.h
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && sed -i "1i#define SYNO_EVANSPORT" synoconfig.h

#-----set library linker to use relative library search path (../lib and .)
#-----tricky shell escaping of $ORIGIN is required http://itee.uq.edu.au/~daniel/using_origin/
#-----http://stackoverflow.com/questions/6562403/i-dont-understand-wl-rpath-wl
#-----however although FFmpeg gets a correct rpath, the downstream libraries get a value of "RIGIN/../lib:RIGIN"
#-----owing to some additional variable substitution happening - easier to build with XORIGIN and use chrpath later
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] \
  || LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-rpath,XORIGIN/../lib:XORIGIN,-rpath-link,${TOOLCHAIN}/lib ${FF_CONFIG} \
  && LDFLAGS=-Wl,-L${TOOLCHAIN}/lib,-L${TOOLCHAIN}/lib/intelce-utilities,-rpath,XORIGIN/../lib:XORIGIN,-rpath-link,${TOOLCHAIN}/lib,-rpath-link,${TOOLCHAIN}/lib/intelce-utilities ${FF_CONFIG}

#-----external libs (librtmp, libass) don't seem to link to ffmpeg (check ffmpeg binary with objdump -x)
#-----this is in fact normal because LDFLAGS as defined in config.mak contains '--as-needed'
#-----https://wiki.gentoo.org/wiki/Project:Quality_Assurance/As-needed
#-----e.g. FFmpeg does not contain any symbols from librtmp, but libavformat does
#-----use ldd on the target system and the full dependency chain will be shown
make
make install

#-----remove intelce-utilities from include path
[ "${TOOLCHAIN}" == "/usr/local/evansport-pc-linux-gnu" ] && export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"

mkdir native-${CROSS_PREFIX}
cd native-${CROSS_PREFIX}

#-----evansport FFmpeg 2.0.2
#-----fix up rpath settings
sudo chrpath -r '$ORIGIN/../lib:$ORIGIN' ${TOOLCHAIN}/bin/ffmpeg
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavcodec.so.55
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavdevice.so.55
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavfilter.so.3
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavformat.so.55
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavutil.so.52
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libpostproc.so.52
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswresample.so.0
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswscale.so.2
cp ${TOOLCHAIN}/lib/libavcodec.so.55 .
cp ${TOOLCHAIN}/lib/libavdevice.so.55 .
cp ${TOOLCHAIN}/lib/libavfilter.so.3 .
cp ${TOOLCHAIN}/lib/libavformat.so.55 .
cp ${TOOLCHAIN}/lib/libavutil.so.52 .
cp ${TOOLCHAIN}/lib/libpostproc.so.52 .
cp ${TOOLCHAIN}/lib/libswresample.so.0 .
cp ${TOOLCHAIN}/lib/libswscale.so.2 .
cp ${TOOLCHAIN}/lib/libx264.so.119 .

#-----FFmpeg 2.7.1
#-----fix up rpath settings
sudo chrpath -r '$ORIGIN/../lib:$ORIGIN' ${TOOLCHAIN}/bin/ffmpeg
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavcodec.so.56
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavdevice.so.56
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavfilter.so.5
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavformat.so.56
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libavutil.so.54
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libpostproc.so.53
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswresample.so.1
sudo chrpath -r '$ORIGIN' ${TOOLCHAIN}/lib/libswscale.so.3
cp ${TOOLCHAIN}/lib/libavcodec.so.56 .
cp ${TOOLCHAIN}/lib/libavdevice.so.56 .
cp ${TOOLCHAIN}/lib/libavfilter.so.5 .
cp ${TOOLCHAIN}/lib/libavformat.so.56 .
cp ${TOOLCHAIN}/lib/libavutil.so.54 .
cp ${TOOLCHAIN}/lib/libpostproc.so.53 .
cp ${TOOLCHAIN}/lib/libswresample.so.1 .
cp ${TOOLCHAIN}/lib/libswscale.so.3 .
cp ${TOOLCHAIN}/lib/libx264.so.146 .

cp ${TOOLCHAIN}/bin/ffmpeg .
cp ${TOOLCHAIN}/bin/fc-cache .
cp ${TOOLCHAIN}/lib/libass.so.5 .
cp ${TOOLCHAIN}/lib/libfontconfig.so.1 .
cp ${TOOLCHAIN}/lib/libfribidi.so.0 .
cp ${TOOLCHAIN}/lib/libnettle.so.4 .
cp ${TOOLCHAIN}/lib/libhogweed.so.2 .
cp ${TOOLCHAIN}/lib/libgnutls.so.28 .
cp ${TOOLCHAIN}/lib/librtmp.so.1 .
cp ${TOOLCHAIN}/lib/libspeex.so.1 .
[ ${TOOLCHAIN} == "/usr/local/arm-none-linux-gnueabi" ] && cp ${TOOLCHAIN}/lib/libshine.so.3 .
#cp ${TOOLCHAIN}/lib/libfdk-aac.so.1 .
sudo chmod +x *
sudo chown root:root *
XZ_OPT=-9 tar -cvJf ffmpeg-${CROSS_PREFIX}.tar.xz *
 

Here’s how the library dependencies look. There are no links to any libraries in /opt, and DSM-bundled libraries in /lib are linked to where possible. All the other new libraries are kept within the Serviio lib folder (../lib relative to the ffmpeg binary).

DS111> ldd /var/packages/Serviio/target/bin/ffmpeg
	libavdevice.so.56 => /volume1/@appstore/Serviio/bin/../lib/libavdevice.so.56 (0x40031000)
	libavfilter.so.5 => /volume1/@appstore/Serviio/bin/../lib/libavfilter.so.5 (0x4004f000)
	libavformat.so.56 => /volume1/@appstore/Serviio/bin/../lib/libavformat.so.56 (0x40174000)
	libavcodec.so.56 => /volume1/@appstore/Serviio/bin/../lib/libavcodec.so.56 (0x40321000)
	libpostproc.so.53 => /volume1/@appstore/Serviio/bin/../lib/libpostproc.so.53 (0x4138d000)
	libswresample.so.1 => /volume1/@appstore/Serviio/bin/../lib/libswresample.so.1 (0x413ad000)
	libswscale.so.3 => /volume1/@appstore/Serviio/bin/../lib/libswscale.so.3 (0x413d1000)
	libavutil.so.54 => /volume1/@appstore/Serviio/bin/../lib/libavutil.so.54 (0x41449000)
	libm.so.6 => /lib/libm.so.6 (0x414c2000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x4156b000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x41587000)
	libc.so.6 => /lib/libc.so.6 (0x415b0000)
	libass.so.5 => /volume1/@appstore/Serviio/bin/../lib/libass.so.5 (0x416f3000)
	librtmp.so.1 => /volume1/@appstore/Serviio/bin/../lib/librtmp.so.1 (0x41726000)
	libz.so.1 => /lib/libz.so.1 (0x4174e000)
	libgnutls.so.28 => /volume1/@appstore/Serviio/bin/../lib/libgnutls.so.28 (0x41772000)
	libx264.so.146 => /volume1/@appstore/Serviio/bin/../lib/libx264.so.146 (0x4187a000)
	libspeex.so.1 => /volume1/@appstore/Serviio/bin/../lib/libspeex.so.1 (0x419ea000)
	libshine.so.3 => /volume1/@appstore/Serviio/bin/../lib/libshine.so.3 (0x41a14000)
	librt.so.1 => /lib/librt.so.1 (0x41a2e000)
	/lib/ld-linux.so.3 (0x40000000)
	libfribidi.so.0 => /volume1/@appstore/Serviio/bin/../lib/libfribidi.so.0 (0x41a45000)
	libfontconfig.so.1 => /volume1/@appstore/Serviio/bin/../lib/libfontconfig.so.1 (0x41a6c000)
	libexpat.so.1 => /lib/libexpat.so.1 (0x41ab9000)
	libfreetype.so.6 => /lib/libfreetype.so.6 (0x41adb000)
	libhogweed.so.2 => /volume1/@appstore/Serviio/bin/../lib/libhogweed.so.2 (0x41b7e000)
	libnettle.so.4 => /volume1/@appstore/Serviio/bin/../lib/libnettle.so.4 (0x41bb5000)
	libgmp.so.10 => /lib/libgmp.so.10 (0x41bfc000)
	libbz2.so.1.0 => /lib/libbz2.so.1.0 (0x41c6b000)
	libpng16.so.16 => /lib/libpng16.so.16 (0x41c7c000)
DS111>
 

Here’s how the even more complex Evansport build with hardware transcoding support looks:

DS214Play> ldd /var/packages/Serviio/target/bin/ffmpeg
	linux-gate.so.1 =>  (0xffffe000)
	libavdevice.so.55 => /volume1/@appstore/Serviio/bin/../lib/libavdevice.so.55 (0xb775c000)
	libavfilter.so.3 => /volume1/@appstore/Serviio/bin/../lib/libavfilter.so.3 (0xb767d000)
	libavformat.so.55 => /volume1/@appstore/Serviio/bin/../lib/libavformat.so.55 (0xb74fd000)
	libavcodec.so.55 => /volume1/@appstore/Serviio/bin/../lib/libavcodec.so.55 (0xb677b000)
	libpostproc.so.52 => /volume1/@appstore/Serviio/bin/../lib/libpostproc.so.52 (0xb674b000)
	libswresample.so.0 => /volume1/@appstore/Serviio/bin/../lib/libswresample.so.0 (0xb6733000)
	libswscale.so.2 => /volume1/@appstore/Serviio/bin/../lib/libswscale.so.2 (0xb66c9000)
	libavutil.so.52 => /volume1/@appstore/Serviio/bin/../lib/libavutil.so.52 (0xb6687000)
	libffmpeg_plugin.so => /lib/libffmpeg_plugin.so (0xb6672000)
	libm.so.6 => /lib/libm.so.6 (0xb6648000)
	libjson-c.so.2 => /lib/libjson-c.so.2 (0xb663d000)
	libpthread.so.0 => /lib/libpthread.so.0 (0xb6622000)
	libc.so.6 => /lib/libc.so.6 (0xb64aa000)
	libismd_core.so => /lib/libismd_core.so (0xb649d000)
	libismd_viddec.so => /lib/libismd_viddec.so (0xb6494000)
	librt.so.1 => /lib/librt.so.1 (0xb648b000)
	libass.so.5 => /volume1/@appstore/Serviio/bin/../lib/libass.so.5 (0xb645f000)
	librtmp.so.1 => /volume1/@appstore/Serviio/bin/../lib/librtmp.so.1 (0xb6443000)
	libz.so.1 => /lib/libz.so.1 (0xb642d000)
	libgnutls.so.28 => /volume1/@appstore/Serviio/bin/../lib/libgnutls.so.28 (0xb6306000)
	libx264.so.119 => /volume1/@appstore/Serviio/bin/../lib/libx264.so.119 (0xb61b3000)
	libspeex.so.1 => /volume1/@appstore/Serviio/bin/../lib/libspeex.so.1 (0xb6193000)
	libmp3lame.so.0 => /lib/libmp3lame.so.0 (0xb6121000)
	libdl.so.2 => /lib/libdl.so.2 (0xb611c000)
	libxml2.so.2 => /lib/libxml2.so.2 (0xb5fc5000)
	libismd_demux.so => /lib/libismd_demux.so (0xb5fba000)
	libismd_vidrend.so => /lib/libismd_vidrend.so (0xb5fb2000)
	libismd_clock.so => /lib/libismd_clock.so (0xb5fb0000)
	libismd_audio.so => /lib/libismd_audio.so (0xb5fa1000)
	libismd_videnc.so => /lib/libismd_videnc.so (0xb5f9d000)
	libismd_vidpproc.so => /lib/libismd_vidpproc.so (0xb5f96000)
	libismdmessage.so => /lib/libismdmessage.so (0xb5f93000)
	libsven.so => /lib/libsven.so (0xb5e53000)
	libpal.so => /lib/libpal.so (0xb5e51000)
	libosal.so => /lib/libosal.so (0xb5e49000)
	libplatform_config.so => /lib/libplatform_config.so (0xb5e46000)
	libsystem_utils.so => /lib/libsystem_utils.so (0xb5e43000)
	libismd_clock_recovery.so => /lib/libismd_clock_recovery.so (0xb5e40000)
	/lib/ld-linux.so.2 (0xb7768000)
	libfribidi.so.0 => /volume1/@appstore/Serviio/bin/../lib/libfribidi.so.0 (0xb5e27000)
	libfontconfig.so.1 => /volume1/@appstore/Serviio/bin/../lib/libfontconfig.so.1 (0xb5de4000)
	libexpat.so.1 => /lib/libexpat.so.1 (0xb5dbc000)
	libfreetype.so.6 => /lib/libfreetype.so.6 (0xb5d08000)
	libhogweed.so.2 => /volume1/@appstore/Serviio/bin/../lib/libhogweed.so.2 (0xb5cdb000)
	libnettle.so.4 => /volume1/@appstore/Serviio/bin/../lib/libnettle.so.4 (0xb5ca3000)
	libgmp.so.10 => /lib/libgmp.so.10 (0xb5c33000)
	liblzma.so.5 => /lib/liblzma.so.5 (0xb5c0d000)
	libbz2.so.1.0 => /lib/libbz2.so.1.0 (0xb5bfc000)
	libpng16.so.16 => /lib/libpng16.so.16 (0xb5bc4000)
DS214Play> 

Bliss album art manager package for Synology NAS

bliss-UI

 

Bliss is a Java application written by Dan Gravell which can manage file names, tags and album art for your music collection and which can also enforce their consistency. It is designed to be left running once installed so that albums you add later will have these same policies applied to them automatically. It supports a wide range of music formats, and effortlessly deals with very large collections – apparently even ones containing quite obscure recordings. My own collection didn’t really put this to the test, since it doesn’t contain bootlegs, live sets or rarities plus I had also already obtained cover art for it (from back when CoverFlow first graced the screen of my iPhone 2G).

I could see from referrals to this blog that people were asking Dan for a Synology package which he didn’t have the time to investigate, and so I thought that it would make an interesting little project, especially since a NAS is the ideal device to run Bliss on. Although there was already a Howto post on the Synology forums, that guide only really covered getting the basic functionality of Bliss up and running – it was missing the best bits: the filesystem watching and audio fingerprinting features. These depend on natively compiled binaries, which Bliss doesn’t include for ARM or PowerPC CPUs. Getting these working provided precisely the sort of challenge I like. Not only were they difficult to compile, but getting them integrated into the various OSGi ‘bundles’ that make up Bliss was quite involved too.

Bliss uses an open source library called Chromaprint, itself part of the wider Acoustid project. The aim is to scan an audio file, to produce a fingerprint of the sound, and then to compare this against an open online database such as MusicBrainz.org to identify music regardless of compression codec used. Its author Lukáš Lalinský explains how it works.

 

Synology Package Installation

  • In Synology DSM’s Package Center, click Settings and add my package repository:
    Add Package Repository
  • The repository will push its certificate automatically to the NAS, which is used to validate package integrity. Set the Trust Level to Synology Inc. and trusted publishers:
    Trust Level
  • Since Bliss is a Java application, you will need to install one of my Java SE Embedded packages first (Java 7 or 8) if you have not already done so. Read the instructions on that page carefully too.
  • Now browse the Community section in Package Center to install Bliss:
    Community-packages
    The repository only displays packages which are compatible with your specific model of NAS. If you don’t see Bliss in the list, then either your NAS model or your DSM version are not supported at this time. DSM 5.0 is the minimum supported version for this package, though you will need DSM 6.0 or later for audio fingerprinting support.
  • When the Bliss package is running you can manage it using the icon in the main DSM application menu using the button in the top left corner:
    bliss-webui
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology 3rd Party Developer Guide.

installer.sh

#!/bin/sh

#--------BLISS installer script
#--------package maintained at pcloadletter.co.uk


DOWNLOAD_URL="`wget -qO- http://www.blisshq.com/app/latest-linux-version`"
DOWNLOAD_FILE="`echo ${DOWNLOAD_URL} | sed -r "s%^.*/(.*)%\1%"`"
SYNO_CPU_ARCH="`uname -m`"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && [ ${SYNOPKG_DSM_VERSION_MAJOR} -ge 6 ] && SYNO_CPU_ARCH="x64"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="i686"
[ "${SYNOPKG_DSM_ARCH}" == "armada375" ] && SYNO_CPU_ARCH="armv7l"
[ "${SYNOPKG_DSM_ARCH}" == "armada38x" ] && SYNO_CPU_ARCH="armhfneon"
[ "${SYNOPKG_DSM_ARCH}" == "comcerto2k" ] && SYNO_CPU_ARCH="armhfneon"
[ "${SYNOPKG_DSM_ARCH}" == "alpine" ] && SYNO_CPU_ARCH="armhfneon"
[ "${SYNOPKG_DSM_ARCH}" == "alpine4k" ] && SYNO_CPU_ARCH="armhfneon"
[ "${SYNOPKG_DSM_ARCH}" == "monaco" ] && SYNO_CPU_ARCH="armhfneon"
[ ${SYNOPKG_DSM_VERSION_MAJOR} -ge 6 ] && NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/bliss-native-${SYNO_CPU_ARCH}.tar.xz"
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
#'ua' prefix means wget user-agent will be customized
INSTALL_FILES="ua${DOWNLOAD_URL} ${NATIVE_BINS_URL}"
PID_FILE="${SYNOPKG_PKGDEST}/bliss.pid"
TEMP_FOLDER="`find / -maxdepth 2 -path '/volume?/@tmp' | head -n 1`"
APP_TEMP="${TEMP_FOLDER}/${SYNOPKG_PKGNAME}"
source /etc/profile


pre_checks ()
{
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. " >> $SYNOPKG_TEMP_LOGFILE
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5" >> $SYNOPKG_TEMP_LOGFILE
    exit 1
  fi

  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. " >> $SYNOPKG_TEMP_LOGFILE
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5" >> $SYNOPKG_TEMP_LOGFILE
    exit 1
  fi

  JAVA_VER=`java -version 2>&1 | sed -r "/^.* version/!d;s/^.* version \"[0-9]\.([0-9]).*$/\1/"`
  if [ ${JAVA_VER} -lt 7 ]; then
    echo "This version of Bliss requires Java 7 or newer. Please update your Java package. " >> $SYNOPKG_TEMP_LOGFILE
    exit 1
  fi

  if [ ${SYNOPKG_DSM_VERSION_MAJOR} -lt 6 ]; then
    echo "Please note that native binary support for song identification via audio fingerprinting requires DSM 6.0" >> $SYNOPKG_TEMP_LOGFILE
  fi
}


preinst ()
{
  pre_checks
  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    #this will allow blisshq.com to track the number of downloads from Synology users
    WGET_URL=`echo ${WGET_URL} | sed -e "s/^ua/--user-agent=Synology --referer=http:\/\/pcloadletter.co.uk\/2012\/09\/17\/bliss-package-for-synology /"`
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, " >> $SYNOPKG_TEMP_LOGFILE
        echo "which was \"${WGET_URL}\" " >> $SYNOPKG_TEMP_LOGFILE
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. " >> $SYNOPKG_TEMP_LOGFILE
        exit 1
      fi
    fi
  done
 
  exit 0
}


postinst ()
{
  #run the installer
  cd ${TEMP_FOLDER}
  echo "INSTALL_PATH=${SYNOPKG_PKGDEST}" > ${TEMP_FOLDER}/bliss-synology.properties
  java -jar ${TEMP_FOLDER}/${DOWNLOAD_FILE} -options ${TEMP_FOLDER}/bliss-synology.properties > /dev/null && rm ${TEMP_FOLDER}/${DOWNLOAD_FILE}
  rm ${TEMP_FOLDER}/bliss-synology.properties
  sed -i "s%^#!/bin/bash%#!/bin/sh%" ${SYNOPKG_PKGDEST}/bin/bliss.sh
  
  #stow jar files containing Synology versions of native code
  if [ -f ${TEMP_FOLDER}/${NATIVE_BINS_FILE} ]; then
    mkdir ${SYNOPKG_PKGDEST}/syno-native
    cd ${SYNOPKG_PKGDEST}/syno-native
    tar xJf ${TEMP_FOLDER}/${NATIVE_BINS_FILE}
  fi
  #record the CPU architecture
  echo "${SYNO_CPU_ARCH}" > syno_cpu_arch.txt

  #make changes to Bliss launcher script so that pid file is created for Java process
  sed -r -i "s%^(exec .*)$%\1 > ${SYNOPKG_PKGDEST}/bliss.out 2>\&1 \&%" ${SYNOPKG_PKGDEST}/bin/bliss.sh
  echo "echo \$! > ${PID_FILE}" >> ${SYNOPKG_PKGDEST}/bin/bliss.sh

  #set some additional system properties (temp folder, prefs sync interval)
  EXTRA_OPTS="-Djava.io.tmpdir=${TEMP_FOLDER} -Djava.util.prefs.syncInterval=86400 -Djava.net.preferIPv4Stack=true"
  sed -r -i "s%-splash:bliss-splash.png%${EXTRA_OPTS}%" ${SYNOPKG_PKGDEST}/bin/bliss.sh
  sed -r -i "s%-XX:HeapDumpPath=/tmp%-XX:HeapDumpPath=${TEMP_FOLDER}%" ${SYNOPKG_PKGDEST}/bin/bliss.sh

  #create log file to allow package start errors to be captured
  [ -e ${SYNOPKG_PKGDEST}/bliss.out ] || touch ${SYNOPKG_PKGDEST}/bliss.out

  #add firewall config
  /usr/syno/bin/servicetool --install-configure-file --package /var/packages/${SYNOPKG_PKGNAME}/scripts/${SYNOPKG_PKGNAME}.sc > /dev/null

  exit 0
}


preuninst ()
{
  `dirname $0`/stop-start-status stop

  exit 0
}


postuninst ()
{
  #clean up temp
  [ -d ${TEMP_FOLDER}/Bliss ] && rm -rf ${TEMP_FOLDER}/Bliss

  #remove firewall config
  if [ "${SYNOPKG_PKG_STATUS}" == "UNINSTALL" ]; then
    /usr/syno/bin/servicetool --remove-configure-file --package ${SYNOPKG_PKGNAME}.sc > /dev/null
  fi

  exit 0
}


preupgrade ()
{
  `dirname $0`/stop-start-status stop
  pre_checks
  
  exit 0
}


postupgrade ()
{
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------BLISS start-stop-status script
#--------package maintained at pcloadletter.co.uk


PKG_FOLDER="`dirname $0 | cut -f1-4 -d'/'`"
ENGINE_SCRIPT="${PKG_FOLDER}/target/bin/bliss.sh"
DNAME="`dirname $0 | cut -f4 -d'/'`"
PID_FILE="${PKG_FOLDER}/target/bliss.pid"
DLOG="${PKG_FOLDER}/target/bliss.out"
SYNO_CPU_ARCH="`cat ${PKG_FOLDER}/target/syno-native/syno_cpu_arch.txt`"
source /etc/profile
source /root/.profile


start_daemon ()
{
  #update the package version number in case of an in-app update
  BLISS_BUNDLE_DIR="`grep -r --include='*.info' com.elsten.bliss.bundle ${PKG_FOLDER}/target/felix-cache/ | cut -f1 -d':'`"
  BLISS_BUNDLE_DIR="`dirname ${BLISS_BUNDLE_DIR}`"
  BLISS_VERSION=`grep "version" ${PKG_FOLDER}/INFO | sed -r "s/^.*([0-9]{8}).*$/\1/"`
  if [ -d ${BLISS_BUNDLE_DIR} ]; then
    find ${BLISS_BUNDLE_DIR} -name *.jar > /tmp/bliss-v-check.txt
    while IFS="" read -r FILE_TO_PARSE; do
      if [ -e ${FILE_TO_PARSE} ]; then
        #no unzip command in DSM 6.0
        if [ -e /usr/bin/7z ]; then
          DETECTED_VERSION=`7z e -so ${FILE_TO_PARSE} META-INF/MANIFEST.MF 2> /dev/null | grep Bundle-Version | cut -f4 -d'.' |  cut -c1-8`
        else
          DETECTED_VERSION=`unzip -p ${FILE_TO_PARSE} META-INF/MANIFEST.MF | grep Bundle-Version | cut -f4 -d'.' |  cut -c1-8`
        fi
      fi
      if [ ${DETECTED_VERSION} -gt ${BLISS_VERSION} ]; then
        BLISS_VERSION=${DETECTED_VERSION}
      fi
    done < /tmp/bliss-v-check.txt
  fi
  rm /tmp/bliss-v-check.txt
  sed -r -i "s/^version=\"[0-9]{8}/version=\"${BLISS_VERSION}/" /var/packages/Bliss/INFO
  
  #update the CPU-specific repository customizations (in case of an in-app update)
  #catch both armv5te and armv7l
  if [ "${SYNO_CPU_ARCH}" != "${SYNO_CPU_ARCH/arm/}" ]; then
    sed -i "s/policy\.tag\.auto\.linux\.x86/policy\.tag\.auto\.linux\.ARM_le/g" ${PKG_FOLDER}/target/bliss-bundle/repository.xml
  fi
  if [ "${SYNO_CPU_ARCH}" == "ppc" ]; then
    sed -i "s/policy\.tag\.auto\.linux\.x86/policy\.tag\.auto\.linux\.PowerPC/g" ${PKG_FOLDER}/target/bliss-bundle/repository.xml
  fi
 
  #overwrite native lib bundles with syno versions (in case of an in-app update)
  if [ -e ${PKG_FOLDER}/target/syno-native/ ]; then
    cp ${PKG_FOLDER}/target/syno-native/* ${PKG_FOLDER}/target/bliss-bundle
  fi

  cd ${PKG_FOLDER}
  ${ENGINE_SCRIPT} > /dev/null 2>> ${DLOG}
  if [ -z ${SYNOPKG_PKGDEST} ]; then
    #script was manually invoked, need this to show status change in Package Center
    [ -e ${PKG_FOLDER}/enabled ] || touch ${PKG_FOLDER}/enabled
  fi
}

    
stop_daemon ()
{
  echo "Stopping ${DNAME}" >> ${DLOG}
  kill `cat ${PID_FILE}`
  wait_for_status 1 20 || kill -9 `cat ${PID_FILE}`
  rm -f ${PID_FILE}
  if [ -z ${SYNOPKG_PKGDEST} ]; then
    #script was manually invoked, need this to show status change in Package Center
    [ -e ${PKG_FOLDER}/enabled ] && rm ${PKG_FOLDER}/enabled
  fi
}


daemon_status ()
{
  if [ -f ${PID_FILE} ] && kill -0 `cat ${PID_FILE}` > /dev/null 2>&1; then
    return
  fi
  rm -f ${PID_FILE}
  return 1
}


wait_for_status ()
{
  counter=$2
  while [ ${counter} -gt 0 ]; do
    daemon_status
    [ $? -eq $1 ] && return
    let counter=counter-1
    sleep 1
  done
  return 1
}


case $1 in
  start)
    if daemon_status; then
      echo ${DNAME} is already running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo Starting ${DNAME} ...
      start_daemon
      exit $?
    fi
  ;;

  stop)
    if daemon_status; then
      echo Stopping ${DNAME} ...
      stop_daemon
      exit $?
    else
      echo ${DNAME} is not running
      exit 0
    fi
  ;;

  restart)
    stop_daemon
    start_daemon
    exit $?
  ;;

  status)
    if daemon_status; then
      echo ${DNAME} is running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo ${DNAME} is not running
      exit 1
    fi
  ;;

  log)
    echo "${DLOG}"
    exit 0
  ;;

  *)
    echo "Usage: $0 {start|stop|status|restart}" >&2
    exit 1
  ;;

esac
 

Changelog:

  • 20160606-0011 Substantial overhaul for DSM 6.0, incorporating many enhancements developed for other packages, updated to Bliss version 20160606, DSM 6.0 newer is now required for audio track fingerprinting (fpcalc is compiled to depend on ffmpeg 2.7.1), added support for several newer Synology products, improved accuracy of temp folder detection, in-app updating should also be fixed
  • 20150522-0010 Substantial re-write (hence the long delay):
    Updated to Bliss version 20150522
    DSM 5.0 newer is now required (fpcalc is compiled to depend on FFmpeg 2.0.2)
    Now that Intel systems running DSM 5.0+ use a newer glibc, replacement Intel binaries are no longer needed
    Added support for Mindspeed Comcerto 2000 CPU in DS414j
    Added support for Intel Atom C2538 (avoton) CPU in various models
    Added support for ppc853x CPU in older PowerPC models
    Added support for Marvell Armada 375 CPU in DS215j
    Added support for Intel Evansport CPU in DS214Play and DS415Play
    Switched to using root account – no more adding account permissions, package upgrades will no longer break this
    DSM Firewall application definition added
    Tested with DSM Task Scheduler to allow package to start/stop at certain times of day, saving RAM when not needed
    Daemon init script now uses a proper PID file instead of the unreliable method of using grep on the output of ps
    Daemon init script can be run from the command line
    Switched to .tar.xz compression for native binaries to reduce web hosting storage footprint
    Improved accuracy of temp folder detection
    Package is now signed with repository private key
    User Agent customization while downloading Bliss package from blisshq.com to allow download stats gathering
  • 20130213-0009 Updated to Bliss 20130213, and will correctly report version in Package Center after an in-app update
  • 20130131-0008 Updated to Bliss 20130131
  • 20121112-0007 Fixes for DSM 4.2
  • 20121112-006 Updated to Bliss 20121112
  • 20121019-005 Updated to Bliss 20121019
  • 20121002-004 Updated to Bliss 20121002
  • 20120830-003 Added support for Freescale QorIQ PowerPC CPUs used in some Synology x13 series products, PowerPC processors in previous Synology generations with older glibc versions are not supported
  • 20120830-002 Hopefully fixed Java prefs polling issue that prevented NAS hibernation
  • 20120830-001 initial public release

 

Build Notes

Chromaprint uses some complex maths functions that FFmpeg can provide (specifically Fourier Transform), and FFmpeg’s shared libraries are already included with Synology DSM. Building Chromaprint linked to those existing libraries results in a minuscule 78KB build of fpcalc, rather than the statically compiled ones for various OS and CPU architectures included with Bliss, which weigh in at several megabytes each. I think I’m finally ‘getting’ what open source is all about, which is nice since that was my objective in experimenting with my NAS. To prevent fpcalc building and linking to its dynamic library libchromaprint.so and to get it to detect FFmpeg properly I had to carefully inspect the Makefiles to find the correct build syntax:
FFMPEG_DIR=${TOOLCHAIN} cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_EXAMPLES=ON -DBUILD_SHARED_LIBS=NO .
FFMPEG_DIR is the base folder from which CMake will look for lib/libavcodec.so and include/libavcodec/avfft.h.

For watching the filesystem Bliss uses JNotify to hook into the Linux kernel’s inotify subsystem. Getting this compiled was tricky. It seems no one has reported successfully compiling it for an ARM CPU, and JNotify’s author Omry Yadan wasn’t aware of anyone doing this either. The problem is that compilation halts with these errors:

In file included from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
../inotify-syscalls.h:35:1: error: "__NR_inotify_init" redefined
In file included from /opt/include/sys/syscall.h:25,
                 from ../inotify-syscalls.h:4,
                 from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
/opt/include/asm/unistd.h:344:1: error: this is the location of the previous definition
In file included from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
../inotify-syscalls.h:36:1: error: "__NR_inotify_add_watch" redefined
In file included from /opt/include/sys/syscall.h:25,
                 from ../inotify-syscalls.h:4,
                 from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
/opt/include/asm/unistd.h:345:1: error: this is the location of the previous definition
In file included from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
../inotify-syscalls.h:37:1: error: "__NR_inotify_rm_watch" redefined
In file included from /opt/include/sys/syscall.h:25,
                 from ../inotify-syscalls.h:4,
                 from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
/opt/include/asm/unistd.h:346:1: error: this is the location of the previous definition

By searching in a very generic way for a solution I found this post on stackoverflow.com which helped me to patch the header inotify-syscalls.h to get around the problem. Since there was no JDK for ARM embedded systems at build time I included the headers from OpenJDK6 which I took from a Ubuntu VM.

Compiling for Intel also required a fix. I was getting the same error as featured in this post on the JNotify user forum on SourceForge.net:
expected specifier-qualifier-list before ‘pid_t’

Despite some people’s apparent success simply rearranging the order of the includes in net_contentobjects_jnotify_linux_JNotify_linux.c this didn’t help me. I’m not sure quite how I stumbled upon it, but I found the solution staring at me on this page:
The size_t, ssize_t, uid_t, gid_t, off_t and pid_t types are defined as described in sys/types.h

I inserted an additional include in for sys/types.h and it compiled ok. It’s worth pointing out that, although all Intel Synology units are x86-64 architecture, the Oracle JRE for Embedded is x86 (32 bit), so I used the i686 toolchain. Synology DSM’s FFmpeg shared libraries are also 32 bit so my build of fpcalc needed to comply with this. Bliss nonetheless expects the binary to be called fpcalc_linux64 since Bliss is detecting the underlying Linux CPU architecture.

Once I had got past the obstacle of compiling the native code, I needed to liaise back and forth with Dan to understand how Bliss was dealing with its libraries and how I could replace the built-in versions. Originally this was quite a kludge but Dan has since abstracted out the native binaries into their own OSGI bundle fragments which makes things a lot easier, and allows Bliss to survive in-app updates until those native components are superseded. The Synology package provides the following architecture-specific jar files (with corresponding edits to their manifest) which contain the fpcalc binary from Chromaprint. Thank you to Dan for all the quick answers!

  • net.contentobjects.jnotify.linux.ARM_le-0.94.0.jar
  • net.contentobjects.jnotify.linux.PowerPC-0.94.0.jar
  • com.elsten.bliss.policy.tag.auto.linux.x86-1.0.1.jar
  • com.elsten.bliss.policy.tag.auto.linux.amd64-1.0.1.jar
  • com.elsten.bliss.policy.tag.auto.linux.ARM_le-1.0.1.jar (various versions)
  • com.elsten.bliss.policy.tag.auto.linux.PowerPC-1.0.1.jar
Here are my fixes to JNotify to get the native library part compiled, since they may help someone else:

 

Patch for native compile of JNotify 0.94 on ARM Synology

diff -crB jnotify-vanilla/inotify-syscalls.h jnotify-arm/inotify-syscalls.h
*** jnotify-vanilla/inotify-syscalls.h	2005-11-30 15:07:56.000000000 +0000
--- jnotify-arm/inotify-syscalls.h	2012-09-14 02:43:44.032130098 +0100
***************
*** 32,40 ****
  # define __NR_inotify_add_watch	152
  # define __NR_inotify_rm_watch	156
  #elif defined (__arm__)
! # define __NR_inotify_init	316
! # define __NR_inotify_add_watch	317
! # define __NR_inotify_rm_watch	318
  #elif defined (__SH4__)
  # define __NR_inotify_init	290
  # define __NR_inotify_add_watch	291
--- 32,46 ----
  # define __NR_inotify_add_watch	152
  # define __NR_inotify_rm_watch	156
  #elif defined (__arm__)
! # ifndef __NR_inotify_init
! #  define __NR_inotify_init     316
! # endif
! # ifndef __NR_inotify_add_watch
! #  define __NR_inotify_add_watch 317
! # endif
! # ifndef __NR_inotify_rm_watch
! #  define __NR_inotify_rm_watch 318
! # endif
  #elif defined (__SH4__)
  # define __NR_inotify_init	290
  # define __NR_inotify_add_watch	291
diff -crB jnotify-vanilla/Release/subdir.mk jnotify-arm/Release/subdir.mk
*** jnotify-vanilla/Release/subdir.mk	2011-02-28 18:07:20.000000000 +0000
--- jnotify-arm/Release/subdir.mk	2012-09-14 02:29:00.000000000 +0100
***************
*** 17,23 ****
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	gcc -I/usr/lib/jvm/java-6-sun/include -I/usr/lib/jvm/java-6-sun/include/linux -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
  
--- 17,23 ----
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	gcc -I/volume1/public/temp/jdk_include/ -I/volume1/public/temp/jdk_include/linux -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
 

Patch for cross compile of JNotify 0.94 for other Synology CPU architectures, using Ubuntu Desktop 12

diff -crB jnotify-vanilla/net_contentobjects_jnotify_linux_JNotify_linux.c jnotify-i686/net_contentobjects_jnotify_linux_JNotify_linux.c
*** jnotify-vanilla/net_contentobjects_jnotify_linux_JNotify_linux.c	2011-02-28 18:07:20.000000000 +0000
--- jnotify-xcomp/net_contentobjects_jnotify_linux_JNotify_linux.c	2012-09-14 00:41:53.455010206 +0100
***************
*** 36,41 ****
--- 36,42 ----
  #include <sys/time.h>
  #include <sys/select.h>
  #include <sys/ioctl.h>
+ #include <sys/types.h>
  #include <errno.h>
  #include <stdio.h>
  #include <unistd.h>
diff -crB jnotify-vanilla/Release/makefile jnotify-i686/Release/makefile
*** jnotify-vanilla/Release/makefile	2011-02-28 18:07:20.000000000 +0000
--- jnotify-xcomp/Release/makefile	2012-09-14 00:37:56.475007855 +0100
***************
*** 28,34 ****
  libjnotify.so: $(OBJS) $(USER_OBJS)
  	@echo 'Building target: $@'
  	@echo 'Invoking: GCC C Linker'
! 	gcc -shared -o"libjnotify.so" $(OBJS) $(USER_OBJS) $(LIBS)
  	@echo 'Finished building target: $@'
  	@echo ' '
  
--- 28,34 ----
  libjnotify.so: $(OBJS) $(USER_OBJS)
  	@echo 'Building target: $@'
  	@echo 'Invoking: GCC C Linker'
! 	$(CC) -shared -o"libjnotify.so" $(OBJS) $(USER_OBJS) $(LIBS)
  	@echo 'Finished building target: $@'
  	@echo ' '
  
diff -crB jnotify-vanilla/Release/subdir.mk jnotify-i686/Release/subdir.mk
*** jnotify-vanilla/Release/subdir.mk	2011-02-28 18:07:20.000000000 +0000
--- jnotify-xcomp/Release/subdir.mk	2012-09-14 00:37:33.835007731 +0100
***************
*** 17,23 ****
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	gcc -I/usr/lib/jvm/java-6-sun/include -I/usr/lib/jvm/java-6-sun/include/linux -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
  
--- 17,23 ----
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	$(CC) -I$(TOOLCHAIN)/include -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
 
 

Synology NAS for SME workloads – my design decisions

Synology-RS2212RP+

Background

It all started last year with a humble single drive DS111 which I bought for home, speculating that I could probably run Serviio on it to stream my movies to my DLNA enabled Bluray player. I managed to compile FFmpeg and its dependencies, install Java and get Serviio running and it has been golden ever since. Since then I’ve learned a lot more about Linux, packaged many other pieces of software and reverse-engineered my own Synology package repository, which at the last count over 5,000 NASes had checked into. The value of the Synology products lies in their great DSM software.

A few months later at work I decided to buy an RS411 to replace an appallingly slow FireWire Drobo (sub 10MB/sec!) that one of my colleagues was using to store video production rushes. Using the same four 2TB drives that had been inside the Drobo it has behaved impeccably – and it is currently in the process of mirroring 3TB to CrashPlan courtesy of my own app package. Having passed this test, I decided that Synology was a worthy candidate for a more serious purchase. However I noticed that there wasn’t much information about their SME products online so I’m sharing my research here.

The need for 2nd tier storage

I have used a 15K SAS EqualLogic iSCSI storage array for VMware workloads since 2009 but this is quite full. It can’t accommodate the data which I need to migrate from older arrays which are at end of life. This data (most of it flat files) is very much 2nd tier – I need lots of space, but I don’t really care too much about latency or throughput. It’s also predominantly static data. I do need at least some additional VMware storage, so I can use vMotion to decant and re-arrange VMs on other storage arrays. A larger Synology rack mount NAS therefore presents itself as a very good value way of addressing this need, while keeping the risk of failure acceptably low.

Which model?

The choice is actually pretty easy. A redundant power supply is fairly mandatory in business since this is the most likely thing to fail after a drive. The unit is under warranty for three years, but can you really survive for the day or two of downtime it would most likely take to get a replacement on site? Once this requirement is considered, there are actually very few models to choose from – just three in fact: the RS812RP+ (1U 4 bay), the RS2212RP+ (2U 10 bay) and the RS3412RPxs (2U 10 bay). The 4 bay model isn’t big enough for my needs so that narrows the field. The RS3412RPxs is quite a price hike over the RS2212RP+, but you do get 4 x 1GbE ports with the option of an add-in card with 2 x 10GbE ports. Considering that the EqualLogic PS4000XV unit I’m using for 1st tier storage manages fine with 2 x 1GbE on each controller I think this is a little overkill for my needs and besides, I don’t have any 10GbE switches (more on bandwidth later). Other physical enhancements are a faster CPU, ECC server RAM, and the ability to add two InfiniBand-connected RS1211 (2U 12 bay) expansion units rather than one.

Synology now offer VMware certified VAAI support from DSM 4.1 on the xs models (currently in beta). They actually support more VAAI primitives than EqualLogic, including freeing up space on a thin provisioned LUN when blocks are deleted. Dell EqualLogic brazenly advertised this as a feature in a vSphere 4.0 marketing PDF back in 2009 and I had to chase tech support for weeks to discover that it was “coming soon”. To date this functionality is still missing. The latest ETA is that it will ship with EqualLogic firmware 6.00 whenever that arrives. Though this is a software feature, Synology are using it to differentiate the more expensive products. More CPU is required during these VAAI operations, though blogger John Nash suggests that it isn’t much of an overhead.

If you need high performance during cloning and copying operations, or are considering using a Synology NAS as your 1st tier storage then perhaps you should consider the xs range.

Which drives?

The choices are bewildering at first. Many of the cheapest drives are the ‘green’ ones which typically spin at 5,400RPM. Their performance won’t be as good as 7,200RPM models, and they also tend to have more aggressive head parking and spindown timer settings in their firmwares. Western Digital ones are notorious for this to the extent that these drives have dramatically shorter lifespans if this is not disabled. To do this is tedious and requires a PC and a DOS boot disk. Making a bootable MS-DOS USB key can try the patience of even the calmest person!

UPDATE – it seems from the Synology Forum that DSM build 3.2-1922 and later will automatically disable the idle timer on these drives. You can check the status by running this one-liner while logged into SSH as root:

for d in `/usr/syno/bin/synodiskport -sata` ; do echo "*** /dev/$d ***"; /usr/syno/bin/syno_disk_ctl --wd-idle -g /dev/$d; done

You can force the disabling of the timer with that same tool:

/syno/bin/syno_disk_ctl --wd-idle -d /dev/sda

The next choice is between Enterprise class and Desktop class drives. This is quite subjective, because for years we have been taught that only SCSI/SAS drives were meant to be sufficiently reliable for continuous use. Typically the Enterprise class drives will have a 5 year manufacturer warranty and the Desktop ones will have 3 years. Often it takes a call to the manufacturer’s customer service helpline to determine the true warranty cover for a particular drive since retailers often seem to misreport this detail on their website. The Enterprise ones are significantly more expensive (£160 Vs £90 for a 2TB drive).

There is one additional feature on Enterprise class drives – TLER (Time Limited Error Recovery). There are a few articles about how this relates to RAID arrays and it proved quite a distraction while researching this NAS purchase. The concept is this: when a drive encounters a failed block during a write operation there is a delay while that drive remaps the physical block to the spare blocks on the drive. On a desktop PC your OS would hang for a moment until the drive responds that the write was successful. A typical hardware RAID controller is intolerant of a delay here and will potentially fail the entire drive if this happens, even though only a single block is faulty. TLER allows to drive to inform the controller that the write is delayed, but not failed. The side effect of not having TLER support would be frequent drive rebuilds from parity, which can be very slow when you’re dealing with 2TB disks – not to mention impairing performance. The good news though, is that the Synology products use a Linux software RAID implementation, so TLER support becomes irrelevant.

Given what’s at stake it’s highly advisable to select drives which are in the Synology HCL. The NAS may be able to overcome some particular drive firmware quirks in software (like idle timers on some models etc.), and their presence on the list does also mean that Synology have tested them thoroughly. I decided to purchase 11 drives so I would have one spare on site ready for insertion directly after a failure. RAID parity can take a long time to rebuild so you don’t want to be waiting for a replacement. Bear in mind that returning a drive under a manufacturer warranty could take a week or two.

Apparently one of the value-added things with enterprise grade SAN storage is that individual drives will be selected from different production batches to minimize the chances of simultaneous failures. This does remain a risk for NAS storage, and all the RAID levels in the world cannot help you in that scenario.

My Order

  • Bare RS2212RP+ 10 bay rackmount NAS (around £1,700 – all prices are excluding VAT).
  • 11 x Hitachi Desktar HDS723020BLA642 drives including 3 year manufacturer warranty (around £1,000).
  • The unit has 1GB of DDR3 RAM soldered on the mainboard, with an empty SODIMM slot which I would advise populating with a 2GB Kingston RAM module, part number KVR1066D3S8S7/2G (a mere £10), just in case you want to install additional software packages later.
  • Synology 2U sliding rack rail kit, part number 13-082URS010 (£80). The static 1U rail kit for the RS411 was pretty poorly designed but this one is an improvement. It is still a bit time consuming to set up compared to modern snap-to-fix rails from the likes of Dell and HP.

Setup – How to serve the data

A Synology NAS offers several ways to actually store the data:

  • Using the NAS as a file server in its own right, using SMB, AFP, or NFS
  • iSCSI at block level (dedicated partitions)
  • iSCSI at file level (more flexible, but a small performance hit)

For my non-critical RS411, using it as an Active Directory integrated file server has proved to be very reliable. However, for this new NAS I needed LUNs for VMware. I could have perhaps defined a Disk Group and dedicated some storage to iSCSI, and some to a normal ext4 volume. I had experimented with iSCSI, but there are several problems:

  • Online research does reveal that there have been some significant iSCSI reliability issues on Synology, though admittedly these issues could possibly date from when DSM first introduced iSCSI functionality.
  • To use iSCSI multipathing on Synology the two NAS network interfaces must be on separate subnets. This is at odds with the same-subnet approach of Dell EqualLogic storage, which the rest of my VMware infrastructure uses. This would mean that hosts using iSCSI storage would need additional iSCSI initiators, significantly increasing complexity.
  • It is customary to isolate iSCSI traffic onto a separate storage network infrastructure, but the Synology NAS does not possess a separate management NIC. So if it is placed on a storage LAN it will not be easily managed/monitored/updated, nor even be able to send email alerts when error conditions arise. This was a show-stopper for me. I think Synology ought to consider at least allowing management traffic to use a different VLAN even if it must use the same physical NICs. However, VLANing iSCSI traffic is something most storage vendors advise against.

All of which naturally lead us onto NFS which is very easy to configure and well supported by VMware. Multipathing isn’t possible for a single NFS share, so the best strategy is to bond the NAS network interfaces into a link aggregation group (‘Route Based on IP Hash’). This does mean however, that no hypervisor’s connection to the NFS storage IP can use more than 1Gbps of bandwidth. This gives a theoretical peak throughput of 1024/8 = 128MB/sec. Considering that each individual SATA hard disk in the array is capable of providing roughly this same sustained transfer rate, this figure is somewhat disappointing. The NAS can deliver much faster speeds than this but is restricted by its 1GbE interfaces. Some NFS storage appliances help to mitigate this limitation to a degree by allowing you to configure multiple storage IP addresses. You could then split your VMs between several NFS shares, each with a different destination IP which could be routed down a different physical link. In this way a single hypervisor could saturate both links. Not so for Synology NAS unfortunately.

If raw performance is important to you, perhaps you should reconsider the xs series’ 2 x 10GbE optional add-in card. Remember though that the stock xs config (4 x GbE) will still suffer from this NFS performance capping of a single NFS connection at 1GbE. It should be noted however that multiple hypervisors accessing this storage will each be able to achieve this transfer rate, up to the maximum performance of the RAID array (around 200MB/sec for an RS2212RP+ according to the official performance figures, rising to around 10 times that figure for the xs series – presumably with the 10GbE add-in card).

As per this blog post, VMware will preferentially route NFS traffic down the first kernel port that is on the same subnet as the target NFS share if one exists, if not it will connect using the management interface via the default gateway. So adding more kernel ports won’t help. My VMware hypervisor servers use 2 x GbE for management traffic, 2 x GbE for VM network traffic, and 2 x GbE for iSCSI. Though I had enough spare NICs, connecting another pair of interfaces solely for NFS was a little overkill, especially since I know that the IOPS requirement for this storage is low. I was also running out of ports on the network patch panel in that cabinet. I did test the performance using dedicated interfaces but unsurprisingly I found it no better. In theory it’s a bad idea to use management network pNICs for anything else since that could slow vMotion operations or in extreme scenarios even prevent remote management. However, vMotion traffic is also constrained by the same limitations of ‘Route Based on IP Hash’ link aggregation policy – i.e. no single connection can saturate more than one physical link (1GbE). In my environment I’m unlikely to be migrating multiple VMs by vMotion concurrently so I have decided to use the management connections for NFS traffic too.

Benchmarking and RAID level

I found the simplest way to benchmark the transfer rates was to perform vMotion operations while keeping the Resource Monitor app open in DSM, and then referring to Cacti graphs of my switch ports to sanity check the results. The network switch is a Cisco 3750 two unit stack, with the MTU temporarily configured to a max value of 9000 bytes.

  • Single NFS share transfer rates reading and writing were both around 120MB/sec at the stock MTU setting of 1500 (around 30% CPU load). That’s almost full GbE line speed.
  • The same transfers occurred using only 15% CPU load with jumbo frames enabled, though the actual transfer rates worsened to around 60-70MB/sec. Consequently I think jumbo frames are pointless here.
  • The CPU use did not significantly increase between RAID5 and RAID6.

I decided therefore to keep an MTU of 1500 and to use RAID6 since this buys a lot of additional resilience. The usable capacity of this VMware ready NAS is now 14TB. It has redundant power fed from two different UPS units on different power circuits, and it has aggregated network uplinks into separate switch stack members. All in all that’s pretty darn good for £2,800 + VAT.

CrashPlan PROe Server package for Synology NAS

CrashPlan is a popular online backup solution, with most people using it to protect their data in the Cloud. However, by licensing CrashPlan PROe server you can be that Cloud and act as the storage provider for other client machines running CrashPlan PROe.

I was recently contracted to implement this on Synology hardware for North Bay Technologies, an IT services company in San Francisco. Instead of undertaking a careful manual install that would be difficult to maintain in future, I decided to go one better – to build a package which integrates properly with Synology DSM. I then back-merged most of the changes into the existing CrashPlan client package scripts so everything is as consistent as possible. It was agreed that I would also publish this to the community, so here it is!

CrashPlan-PROe-dashboard

I should stress at this point that although this package could technically install on Synology products with ARM or QorIQ processors, you should only use this on a model with an Intel CPU. Ideally you should equip it with more than 1GB of RAM too, because the application requires 1GB all for itself. The package repo will not advertise it to ARM systems, because they have far too little available RAM.

As with the CrashPlan client packages I have made, I have been careful to comply with Code 42 Software’s stipulation that no one redistributes their work.

CrashPlan-PROe-server-info

 

Installation

  • This package is for Intel CPUs only. It will work on an unmodified NAS, no hacking or bootstrapping required.
  • More than 1GB of RAM is recommended.
  • You will need to install my Java SE for Embedded package first. Read the instructions on that page carefully too.
  • In the User Control Panel in DSM, enable the User Homes service.
  • Purchase your Master License Key and licences (or obtain a trial key) from crashplan.com, download the PROe Server installer for Linux, and save in the public shared folder on your NAS. You should have created this folder when you installed the Java package.
  • Install the CrashPlan PROe server Synology package directly from Package Center in DSM. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk.
 

Notes

  • The package expects the end user to have separately downloaded the CrashPlan PROe Server installer for Linux, and it presents them with the official EULA during installation.
  • For details of TCP ports used, to help you set up firewalling and/or port forwarding please consult the requirements document.
  • Once running, CrashPlan PROe server is configured by a web dashboard on https://yourNasIP:4285/console/ This link can be found in the package info screen in Package Center in Synology DSM when the package is running.
  • Full support documentation is available here.
  • The DSM Package Center upgrade functionality allows you to move between my package versions without losing settings or data, but if you’re moving to a new CrashPlan PROe Server version you will need to do that manually via the admin web app, using the Linux downloads (with file extension .upgrade) available from Code 42 Software. Depending on how up to date your version is, you may need to update incrementally through several versions. Before you apply a Code 42 update package, you should manually install the latest Synology package for the specific PROe Server version you’re currently runnning. This will ensure the update scripts are handled correctly. So for example if you’re upgrading from 3.5.4 to 3.5.5 you should manually install cpproeserver3.5.4-merged-020.spk over the top first. You can find up to date package versions for each PROe Server build here:
    cpproeserver3.2.1.2-merged-0020.spk
    cpproeserver3.3.0.2-merged-0020.spk
    cpproeserver3.3.0.3-merged-0020.spk
    cpproeserver3.3.0.4-merged-0020.spk
    cpproeserver3.4.1-merged-0020.spk
    cpproeserver3.4.1.5-merged-0020.spk
    cpproeserver3.5.1.1-merged-0020.spk
    cpproeserver3.5.3.2-merged-0020.spk
    cpproeserver3.5.4-merged-0020.spk
    cpproeserver3.5.5-merged-0020.spk
  • The engine daemon script checks the amount of system RAM and scales the Java heap size appropriately (up to the default maximum of 1024MB). This can be overridden in a persistent way if you are backing up very large backup sets by editing /volume1/@appstore/cpproeserver/syno_package.vars.
  • As with my other syno packages, this user account password is randomized when it is created using a perl script called passgen (I followed the example of the Transmission package). DSM Package Center runs as the root user so my script starts the package using an su command. This means that you can change the password yourself and CrashPlan will still work.
  • The default location for saving backup data is set to /volume1/cpproeserver (where /volume1 is you primary storage volume) to eliminate the chance of them being destroyed accidentally by uninstalling the package.
  • The package supports upgrading to future versions while preserving the machine identity, logs, login details, and cache.
  • The log which is displayed in the package’s Log tab is actually the activity history. If you’re trying to troubleshoot an issue you may need to use an SSH session to inspect the more detailed log files which are stored in /volume1/cpproeserver/log
  • I’m not sure if it works for the PROe products, but I would really appreciate it if you could use this affiliate link when purchasing your licences (you may need to browse to the PROe section of the website using the links in the footer of that page). If this package saves you the several days worth of work I put into making it, please also consider donating using the PayPal button on the right hand side of the page. Thanks!
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology Package wiki.

installer.sh

#!/bin/sh

#--------CRASHPLAN PROe server installer script
#--------package maintained at pcloadletter.co.uk

DOWNLOAD_PATH="http://download.crashplan.com/installs/proserver/CP_VER"
DOWNLOAD_FILE="CrashPlanPROServer_CP_VER_Linux.tgz"
DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
TGZ_FILE="CrashPlanPROServer.tgz"
#remove file extension
DOWNLOAD_FILE="`echo ${DOWNLOAD_FILE} | sed -e 's/.tgz$//'`"
EXTRACTED_FOLDER="${DOWNLOAD_FILE}"
DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_PASS="`openssl rand 12 -base64 2>/dev/null`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
DAEMON_HOME="/var/services/homes/${DAEMON_USER}"
OPTDIR="${SYNOPKG_PKGDEST}/proserver/server"
VARS_FILE="${OPTDIR}/.install.vars"
ENGINE_SCRIPT="proserver"
SYNO_CPU_ARCH="`uname -m`"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="i686"
NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/crashplan-native-${SYNO_CPU_ARCH}.tgz"   
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
INSTALL_FILES="${NATIVE_BINS_URL}"
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
#this is where the user data will go, so it persists after a package uninstall
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/${DAEMON_USER}"
VARLOGDIR=${MANIFEST_FOLDER}/log
LOG_FILE="${VARLOGDIR}/com_backup42_app.log.0"
UPGRADE_FILES=""
UPGRADE_FOLDERS="activemq-data conf db keys"

source /etc/profile
PUBLIC_FOLDER="`cat /usr/syno/etc/smb.conf | sed -r '/\/public$/!d;s/^.*path=(\/volume[0-9]{1,4}\/public).*$/\1/'`"

  
preinst ()
{
  if [ -z ${PUBLIC_FOLDER} ]; then
    echo "A shared folder called 'public' could not be found - note this name is case-sensitive. "
    echo "Please create this using the Shared Folder DSM Control Panel and try again."
    exit 1
  fi
  
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  #is the User Home service enabled?
  UH_SERVICE=maybe
  synouser --add userhometest Testing123 "User Home test user" 0 "" ""
  UHT_HOMEDIR=`cat /etc/passwd | sed -r '/User Home test user/!d;s/^.*:User Home test user:(.*):.*$/\1/'`
  if echo $UHT_HOMEDIR | grep '/var/services/homes/' > /dev/null; then
    if [ ! -d $UHT_HOMEDIR ]; then
      UH_SERVICE=false
    fi
  fi
  synouser --del userhometest
  #remove home directory (needed since DSM 4.1)
  [ -e /var/services/homes/userhometest ] && rm -r /var/services/homes/userhometest
  if [ "${UH_SERVICE}" == "false" ]; then
    echo "The User Home service is not enabled. Please enable this feature in the User control panel in DSM."
    exit 1
  fi
  
  CP_BINARY_FOUND=
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tgz ] && CP_BINARY_FOUND=true
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tar ] && CP_BINARY_FOUND=true
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tar.tar ] && CP_BINARY_FOUND=true
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.gz ] && CP_BINARY_FOUND=true
  
  if [ -z ${CP_BINARY_FOUND} ]; then
    echo "CrashPlan PROe server Linux installer not found. "
    echo "I was expecting the file ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tgz "
    echo "Please visit crashplan.com, download the installer from ${DOWNLOAD_URL} "
    echo "and place it in the 'public' shared folder on your NAS."
    exit 1
  fi

  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, "
        echo "which was \"${WGET_URL}\" "
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. "
        exit 1
      fi
    fi
  done
  
  exit 0
}


postinst ()
{
  VAROPTDIR=${MANIFEST_FOLDER}/data
  VARLOGDIR=${MANIFEST_FOLDER}/log
  ETCDIR=${OPTDIR}/bin
  INITDIR=${OPTDIR}/bin
  RUNLVLDIR=${OPTDIR}/bin
  
  #create daemon user
  synouser --add ${DAEMON_USER} ${DAEMON_PASS} "${DAEMON_ID}" 0 "" ""
  
  #save the daemon user's homedir as variable in that user's profile
  #this is needed because new users seem to inherit a HOME value of /root which they have no permissions for.
  su - ${DAEMON_USER} -s /bin/sh -c "echo export HOME=\'${DAEMON_HOME}\' >> .profile"

  #extract CPU-specific additional binaries
  mkdir ${SYNOPKG_PKGDEST}/bin
  cd ${SYNOPKG_PKGDEST}/bin
  tar xzf ${TEMP_FOLDER}/${NATIVE_BINS_FILE} && rm ${TEMP_FOLDER}/${NATIVE_BINS_FILE}

  mkdir -p ${OPTDIR}
  mkdir -p ${INITDIR}
  mkdir -p ${RUNLVLDIR}
  mkdir -p ${VAROPTDIR}
  mkdir -p ${VARLOGDIR}
  cd ${PUBLIC_FOLDER}
  
  #extract CrashPlan Linux installer (Web browsers love to interfere with .tar.gz files)
  if [ -f ${DOWNLOAD_FILE}.tgz ]; then
    #Firefox seems to be the only browser that leaves it alone
    tar xzf ${DOWNLOAD_FILE}.tgz
  elif [ -f ${DOWNLOAD_FILE}.gz ]; then
    #Chrome
    tar xzf ${DOWNLOAD_FILE}.gz
  elif [ -f ${DOWNLOAD_FILE}.tar ]; then
    #Safari
    tar xf ${DOWNLOAD_FILE}.tar
  elif [ -f ${DOWNLOAD_FILE}.tar.tar ]; then
    #Internet Explorer
    tar xzf ${DOWNLOAD_FILE}.tar.tar
  fi
  
  mkdir -p ${OPTDIR}/content-custom
  mkdir -p ${OPTDIR}/installs
  mkdir ${VAROPTDIR}/backupArchives
  mkdir ${VAROPTDIR}/backupCache
  mkdir ${VAROPTDIR}/dumps
  chown -R ${DAEMON_USER} ${MANIFEST_FOLDER}
  
  #extract nested tgz archive
  cd ${OPTDIR}
  tar xozf "${PUBLIC_FOLDER}/${EXTRACTED_FOLDER}/${TGZ_FILE}"
  
  echo "#uncomment to expand Java max heap size beyond prescribed value of 1024M (will survive upgrades)" > ${SYNOPKG_PKGDEST}/syno_package.vars
  echo "#USR_MAX_HEAP=1024M" >> ${SYNOPKG_PKGDEST}/syno_package.vars
  echo >> ${SYNOPKG_PKGDEST}/syno_package.vars
  
  #create a valid identity file if there is no existing GUID
  GUID=
  if [ -f ${VAROPTDIR}/.identity ] ; then
    . ${VAROPTDIR}/.identity
  fi
  if [ "x${GUID}" == "x" ]; then
    echo -n "GUID=" > ${VAROPTDIR}/.identity
    java -cp "${OPTDIR}/lib/com.backup42.app.jar" com.code42.utils.UniqueId >> ${VAROPTDIR}/.identity
    . ${VAROPTDIR}/.identity
    if [ "x${GUID}" == "x" ] ; then
      echo "Failed to create valid server identity. Identity Path: ${VAROPTDIR}/.identity"
      exit 1
    fi
  fi
 
  #amend entries in default server config file
  sed -i "s%<OPT>%${OPTDIR}%" ${OPTDIR}/conf/conf_proe.properties
  sed -i "s%<VAROPT>%${VAROPTDIR}%" ${OPTDIR}/conf/conf_proe.properties
  sed -i "s%<VARLOGDIR>%${VARLOGDIR}%" ${OPTDIR}/conf/conf_proe.properties
  
  #save install variables which Crashplan expects its own installer script to create
  echo "" > ${VARS_FILE}
  echo "OPTDIR=${OPTDIR}" >> ${VARS_FILE}
  echo "VAROPTDIR=${VAROPTDIR}" >> ${VARS_FILE}
  echo "VARLOGDIR=${VARLOGDIR}" >> ${VARS_FILE}
  echo "ETCDIR=${ETCDIR}" >> ${VARS_FILE}
  echo "INITDIR=${INITDIR}" >> ${VARS_FILE}
  echo "RUNLVLD=${RUNLVLDIR}" >> ${VARS_FILE}
  echo "INSTALLDATE=`date +%Y%m%d`" >> ${VARS_FILE}
  echo "JAVACOMMON=\${JAVA_HOME}/bin/java" >> ${VARS_FILE}

  #remove temp files
  rm -r ${PUBLIC_FOLDER}/${EXTRACTED_FOLDER}
  
  #change owner of CrashPlan folder tree
  chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}

  echo "CrashPlan PROe Server has been installed. When you start the package a few moments of first-time initialization "
  echo "are needed before the management application will be available in your web browser. You can check the Log tab "
  echo "to discover when it has fully started. "
  echo "http://localhost:4280/console "
  echo "https://localhost:4285/console "
  echo "Please note that your clients will communicate with the server on TCP port 4282."
  
  exit 0
}


preuninst ()
{
  #make sure engine is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} stop"
  sleep 2
  
  exit 0
}


postuninst ()
{
  if [ -f ${SYNOPKG_PKGDEST}/syno_package.vars ]; then
    source ${SYNOPKG_PKGDEST}/syno_package.vars
  fi
  
  if [ "${LIBFFI_SYMLINK}" == "YES" ]; then
    rm /lib/libffi.so.5
  fi

  #if it doesn't exist, but is still a link then it's a broken link and should also be deleted
  if [ ! -e /lib/libffi.so.5 ]; then
    [ -L /lib/libffi.so.5 ] && rm /lib/libffi.so.5
  fi

  #remove daemon user
  synouser --del ${DAEMON_USER}
  
  #remove daemon user's home directory (needed since DSM 4.1)
  [ -e /var/services/homes/${DAEMON_USER} ] && rm -r /var/services/homes/${DAEMON_USER}
  
  exit 0
}

preupgrade ()
{
  #make sure engine is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} stop"
  sleep 2
  
  #if config data exists back it up
  if [ -d ${OPTDIR}/keys ]; then
    mkdir -p ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
      if [ -d ${OPTDIR}/${FOLDER_TO_MIGRATE} ]; then
        mv ${OPTDIR}/${FOLDER_TO_MIGRATE} ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig
      fi
    done
  fi

  exit 0
}


postupgrade ()
{
  #use the migrated config data from the previous version
  if [ -d ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig/keys ]; then
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
    if [ -d ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig/${FOLDER_TO_MIGRATE} ]; then
      cp -R ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig/${FOLDER_TO_MIGRATE} ${OPTDIR}
    fi
    done
    rmdir ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig

    #make log entry
    TIMESTAMP="`date +%x` `date +%I:%M%p`"
    echo "I ${TIMESTAMP} Synology Package Center updated ${SYNOPKG_PKGNAME} to version ${SYNOPKG_PKGVER}" >> ${LOG_FILE}
    
    #daemon user has been deleted and recreated so we need to reset ownership (new UID)
    chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
  fi
  
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------CRASHPLAN PROe server start-stop-status script
#--------package maintained at pcloadletter.co.uk

DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_HOME="/var/services/homes/${DAEMON_USER}"
OPTDIR="${SYNOPKG_PKGDEST}/proserver/server"
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/${DAEMON_USER}"
VARLOGDIR=${MANIFEST_FOLDER}/log
LOG_FILE="${VARLOGDIR}/history.log.0"
ENGINE_SCRIPT="proserver"
APP_NAME="CPServer"
SCRIPTS_TO_EDIT="${ENGINE_SCRIPT} proservermonitor"
ENGINE_CFG="${ENGINE_SCRIPT}"
LIBFFI_SO_NAMES="5 6" #armada370 build of libjnidispatch.so is newer, and uses libffi.so.6
CFG_PARAM="JAVA_MEM_ARGS"
#note that the vars in the next two string values are escaped for evaluation later on
JAVA_MEM_ARGS="-Xss128k -Xms\${JAVA_MIN_HEAP}m -Xmx\${JAVA_MAX_HEAP}m"
JAVA_GC_ARGS="-XX:+DisableExplicitGC -XX:+UseAdaptiveGCBoundary -XX:PermSize=\${JAVA_MIN_HEAP}m -XX:MaxPermSize=\${JAVA_MIN_HEAP}m"
source ${OPTDIR}/.install.vars

JAVA_MIN_HEAP=`grep "^${CFG_PARAM}=" "${OPTDIR}/bin/${ENGINE_CFG}" | sed -r "s/^.*-Xms([0-9]+)[Mm] .*$/\1/"` 
SYNO_CPU_ARCH="`uname -m`"

case $1 in
  start)
    #set the current timezone for Java so that log timestamps are accurate
    #we need to use the modern timezone names so that Java can figure out DST 
    SYNO_TZ=`cat /etc/synoinfo.conf | grep timezone | cut -f2 -d'"'`
    SYNO_TZ=`grep "^${SYNO_TZ}" /usr/share/zoneinfo/Timezone/tzname | sed -e "s/^.*= //"`
    grep "^export TZ" ${DAEMON_HOME}/.profile > /dev/null \
     && sed -i "s%^export TZ=.*$%export TZ='${SYNO_TZ}'%" ${DAEMON_HOME}/.profile \
     || echo export TZ=\'${SYNO_TZ}\' >> ${DAEMON_HOME}/.profile
    #check persistent variables from syno_package.vars
    USR_MAX_HEAP=0
    if [ -f ${SYNOPKG_PKGDEST}/syno_package.vars ]; then
      source ${SYNOPKG_PKGDEST}/syno_package.vars
    fi
    USR_MAX_HEAP=`echo $USR_MAX_HEAP | sed -e "s/[mM]//"`

    #create or repair libffi symlink if a DSM upgrade has removed it
    for FFI_VER in ${LIBFFI_SO_NAMES}; do 
      if [ -e ${OPTDIR}/lib/libffi.so.${FFI_VER} ]; then
        if [ ! -e /lib/libffi.so.${FFI_VER} ]; then
          #if it doesn't exist, but is still a link then it's a broken link and should be deleted
          [ -L /lib/libffi.so.${FFI_VER} ] && rm /lib/libffi.so.${FFI_VER}
          ln -s ${OPTDIR}/lib/libffi.so.${FFI_VER} /lib/libffi.so.${FFI_VER}
        fi
      fi
    done

    #fix up some of the binary paths and fix some command syntax for busybox 
    #moved this to start-stop-status from installer.sh because Code42 push updates and these
    #new scripts will need this treatment too
    FIND_TARGETS=
    for TARGET in ${SCRIPTS_TO_EDIT}; do
      FIND_TARGETS="${FIND_TARGETS} -o -name ${TARGET}"
    done
    find ${OPTDIR} \( -name \*.sh ${FIND_TARGETS} \) | while IFS="" read -r FILE_TO_EDIT; do
      if [ -e ${FILE_TO_EDIT} ]; then
        #this list of substitutions will probably need expanding as new CrashPlan updates are released
        sed -i "s%^#!/bin/bash%#!${SYNOPKG_PKGDEST}/bin/bash%" "${FILE_TO_EDIT}"
        sed -i -r "s%(^\s*)nice -n%\1${SYNOPKG_PKGDEST}/bin/nice -n%" "${FILE_TO_EDIT}"
        sed -i -r "s%(^\s*)(/bin/ps|ps) [^\|]*\|%\1/bin/ps w \|%" "${FILE_TO_EDIT}"
        sed -i -r "s%\`ps [^\|]*\|%\`ps w \|%" "${FILE_TO_EDIT}"
        sed -i "s/rm -fv/rm -f/" "${FILE_TO_EDIT}"
        sed -i "s/mv -fv/mv -f/" "${FILE_TO_EDIT}"
      fi
    done

    #an upgrade script that has been launched via the web app will usually have failed until the above
    #changes are made so we need to find it and start it, if it exists
    UPGRADE_SCRIPT=`find ${OPTDIR}/upgrade -name "upgrade.sh"`
    if [ -n "${UPGRADE_SCRIPT}" ]; then
      rm ${OPTDIR}/${ENGINE_SCRIPT}.pid
      SCRIPT_HOME=`dirname $UPGRADE_SCRIPT`

      #make CrashPlan log entry
      TIMESTAMP="`date +%x` `date +%I:%M%p`"
      echo "I ${TIMESTAMP} Synology repairing upgrade in ${SCRIPT_HOME}" >> ${LOG_FILE}

      mv ${SCRIPT_HOME}/upgrade.log ${SCRIPT_HOME}/upgrade.log.old
      chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
      su - ${DAEMON_USER} -s /bin/sh -c "cd ${SCRIPT_HOME} ; . upgrade.sh"
      mv ${SCRIPT_HOME}/upgrade.sh ${SCRIPT_HOME}/upgrade.sh.old
      exit 0
    fi

    #updates may also overwrite our native binaries
    if [ "${SYNO_CPU_ARCH}" != "x86_64" ]; then
      cp -f ${SYNOPKG_PKGDEST}/bin/libjtux.so ${OPTDIR}
      cp -f ${SYNOPKG_PKGDEST}/bin/jna-3.2.5.jar ${OPTDIR}/lib
      cp -f ${SYNOPKG_PKGDEST}/bin/libffi.so.* ${OPTDIR}/lib
    fi

    #set appropriate Java max heap size
    RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
    if [ $RAM -le 128 ]; then
      JAVA_MAX_HEAP=80
    elif [ $RAM -le 256 ]; then
      JAVA_MAX_HEAP=192
    elif [ $RAM -le 512 ]; then
      JAVA_MAX_HEAP=384
    elif [ $RAM -le 1024 ]; then
      JAVA_MAX_HEAP=896
    #CrashPlan PROe server's default max heap is 1GB
    elif [ $RAM -gt 1024 ]; then
      JAVA_MAX_HEAP=1024
    fi
    if [ $USR_MAX_HEAP -gt $JAVA_MAX_HEAP ]; then
      JAVA_MAX_HEAP=${USR_MAX_HEAP}
    fi
    if [ $JAVA_MAX_HEAP -lt $JAVA_MIN_HEAP ]; then
      #can't have a max heap lower than min heap (ARM low RAM systems)
      JAVA_MIN_HEAP=${JAVA_MAX_HEAP}
    fi

    #reset ownership of all files to daemon user, so that manual edits to config files won't cause problems
    chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
    chown -R ${DAEMON_USER} ${DAEMON_HOME}    
    
    #CrashPlan PROe server will read customized vars from a separate file
    eval echo "JAVA_GC_ARGS='\"'${JAVA_GC_ARGS}'\"'" > ${OPTDIR}/.proserverrc
    eval echo "JAVA_MEM_ARGS='\"'${JAVA_MEM_ARGS}'\"'" >> ${OPTDIR}/.proserverrc
    su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} start"
    exit 0
  ;;

  stop)
    su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} stop"
    exit 0
  ;;

  status)
    PID=`/bin/ps w| grep "app=${APP_NAME}" | grep -v grep | awk '{ print $1 }'`
    if [[ -n "$PID" ]]; then
      exit 0
    else
      exit 1
    fi
  ;;

  log)
    echo "${LOG_FILE}"
    exit 0
  ;;
esac
 

Changelog:

  • 020 added support for Intel Atom Evansport CPU in some new DSx14 products
  • 019 update to CrashPlan PROe Server 3.5.5, improved update handling
  • 018 update to CrashPlan PROe Server 3.4.1.5, improved update handling, fixes for DSM 4.2
  • 017 update to CrashPlan PROe Server 3.4.1, improved update handling
  • 016 update to CrashPlan PROe Server 3.3.0.4
  • 015 further fixes for the update mechanism
  • 014 created a wrapper for the ps command, and a symlink for /bin/bash which should hopefully allow server upgrade scripts from Code 42 to run
  • 013 fixed a timezone detection bug
  • 012 fixed a bug with the script editing logic which amends Code 42’s scripts to work with busybox shell tools
  • 011 updated to CrashPlan PROe Server 3.3.0.3
  • 010 intial public release
 
 

CrashPlan packages for Synology NAS

UPDATE – CrashPlan For Home (green branding) was retired by Code 42 Software on 22/08/2017. See migration notes below to find out how to transfer to CrashPlan for Small Business on Synology at the special discounted rate.

CrashPlan is a popular online backup solution which supports continuous syncing. With this your NAS can become even more resilient, particularly against the threat of ransomware.

There are now only two product versions:

  • Small Business: CrashPlan PRO (blue branding). Unlimited cloud backup subscription, $10 per device per month. Reporting via Admin Console. No peer-to-peer backups
  • Enterprise: CrashPlan PROe (black branding). Cloud backup subscription typically billed by storage usage, also available from third parties.

The instructions and notes on this page apply to both versions of the Synology package.

CrashPlanPRO-Windows

CrashPlan is a Java application which can be difficult to install on a NAS. Way back in January 2012 I decided to simplify it into a Synology package, since I had already created several others. It has been through many versions since that time, as the changelog below shows. Although it used to work on Synology products with ARM and PowerPC CPUs, it unfortunately became Intel-only in October 2016 due to Code 42 Software adding a reliance on some proprietary libraries.

Licence compliance is another challenge – Code 42’s EULA prohibits redistribution. I had to make the Synology package use the regular CrashPlan for Linux download (after the end user agrees to the Code 42 EULA). I then had to write my own script to extract this archive and mimic the Code 42 installer behaviour, but without the interactive prompts of the original.

 

Synology Package Installation

  • In Synology DSM’s Package Center, click Settings and add my package repository:
    Add Package Repository
  • The repository will push its certificate automatically to the NAS, which is used to validate package integrity. Set the Trust Level to Synology Inc. and trusted publishers:
    Trust Level
  • Now browse the Community section in Package Center to install CrashPlan:
    Community-packages
    The repository only displays packages which are compatible with your specific model of NAS. If you don’t see CrashPlan in the list, then either your NAS model or your DSM version are not supported at this time. DSM 5.0 is the minimum supported version for this package, and an Intel CPU is required.
  • Since CrashPlan is a Java application, it needs a Java Runtime Environment (JRE) to function. It is recommended that you select to have the package install a dedicated Java 8 runtime. For licensing reasons I cannot include Java with this package, so you will need to agree to the licence terms and download it yourself from Oracle’s website. The package expects to find this .tar.gz file in a shared folder called ‘public’. If you go ahead and try to install the package without it, the error message will indicate precisely which Java file you need for your system type, and it will provide a TinyURL link to the appropriate Oracle download page.
  • To install CrashPlan PRO you will first need to log into the Admin Console and download the Linux App from the App Download section and also place this in the ‘public’ shared folder on your NAS.
  • If you have a multi-bay NAS, use the Shared Folder control panel to create the shared folder called public (it must be all lower case). On single bay models this is created by default. Assign it with Read/Write privileges for everyone.
  • If you have trouble getting the Java or CrashPlan PRO app files recognised by this package, try downloading them with Firefox. It seems to be the only web browser that doesn’t try to uncompress the files, or rename them without warning. I also suggest that you leave the Java file and the public folder present once you have installed the package, so that you won’t need to fetch this again to install future updates to the CrashPlan package.
  • CrashPlan is installed in headless mode – backup engine only. This will configured by a desktop client, but operates independently of it.
  • The first time you start the CrashPlan package you will need to stop it and restart it before you can connect the client. This is because a config file that is only created on first run needs to be edited by one of my scripts. The engine is then configured to listen on all interfaces on the default port 4243.
 

CrashPlan Client Installation

  • Once the CrashPlan engine is running on the NAS, you can manage it by installing CrashPlan on another computer, and by configuring it to connect to the NAS instance of the CrashPlan Engine.
  • Make sure that you install the version of the CrashPlan client that matches the version running on the NAS. If the NAS version gets upgraded later, you will need to update your client computer too.
  • The Linux CrashPlan PRO client must be downloaded from the Admin Console and placed in the ‘public’ folder on your NAS in order to successfully install the Synology package.
  • By default the client is configured to connect to the CrashPlan engine running on the local computer. Run this command on your NAS from an SSH session:
    echo `cat /var/lib/crashplan/.ui_info`
    Note those are backticks not quotes. This will give you a port number (4243), followed by an authentication token, followed by the IP binding (0.0.0.0 means the server is listening for connections on all interfaces) e.g.:
    4243,9ac9b642-ba26-4578-b705-124c6efc920b,0.0.0.0
    port,--------------token-----------------,binding

    Copy this token value and use this value to replace the token in the equivalent config file on the computer that you would like to run the CrashPlan client on – located here:
    C:\ProgramData\CrashPlan\.ui_info (Windows)
    “/Library/Application Support/CrashPlan/.ui_info” (Mac OS X installed for all users)
    “~/Library/Application Support/CrashPlan/.ui_info” (Mac OS X installed for single user)
    /var/lib/crashplan/.ui_info (Linux)
    You will not be able to connect the client unless the client token matches on the NAS token. On the client you also need to amend the IP address value after the token to match the Synology NAS IP address.
    so using the example above, your computer’s CrashPlan client config file would be edited to:
    4243,9ac9b642-ba26-4578-b705-124c6efc920b,192.168.1.100
    assuming that the Synology NAS has the IP 192.168.1.100
    If it still won’t connect, check that the ServicePort value is set to 4243 in the following files:
    C:\ProgramData\CrashPlan\conf\ui_(username).properties (Windows)
    “/Library/Application Support/CrashPlan/ui.properties” (Mac OS X installed for all users)
    “~/Library/Application Support/CrashPlan/ui.properties” (Mac OS X installed for single user)
    /usr/local/crashplan/conf (Linux)
    /var/lib/crashplan/.ui_info (Synology) – this value does change spontaneously if there’s a port conflict e.g. you started two versions of the package concurrently (CrashPlan and CrashPlan PRO)
  • As a result of the nightmarish complexity of recent product changes Code42 has now published a support article with more detail on running headless systems including config file locations on all supported operating systems, and for ‘all users’ versus single user installs etc.
  • You should disable the CrashPlan service on your computer if you intend only to use the client. In Windows, open the Services section in Computer Management and stop the CrashPlan Backup Service. In the service Properties set the Startup Type to Manual. You can also disable the CrashPlan System Tray notification application by removing it from Task Manager > More Details > Start-up Tab (Windows 8/Windows 10) or the All Users Startup Start Menu folder (Windows 7).
    To accomplish the same on Mac OS X, run the following commands one by one:

    sudo launchctl unload /Library/LaunchDaemons/com.crashplan.engine.plist
    sudo mv /Library/LaunchDaemons/com.crashplan.engine.plist /Library/LaunchDaemons/com.crashplan.engine.plist.bak

    The CrashPlan menu bar application can be disabled in System Preferences > Users & Groups > Current User > Login Items

 

Migration from CrashPlan For Home to CrashPlan For Small Business (CrashPlan PRO)

  • Leave the regular green branded CrashPlan 4.8.3 Synology package installed.
  • Go through the online migration using the link in the email notification you received from Code 42 on 22/08/2017. This seems to trigger the CrashPlan client to begin an update to 4.9 which will fail. It will also migrate your account onto a CrashPlan PRO server. The web page is likely to stall on the Migrating step, but no matter. The process is meant to take you to the store but it seems to be quite flakey. If you see the store page with a $0.00 amount in the basket, this has correctly referred you for the introductory offer. Apparently the $9.99 price thereafter shown on that screen is a mistake and the correct price of $2.50 is shown on a later screen in the process I think. Enter your credit card details and check out if you can. If not, continue.
  • Log into the CrashPlan PRO Admin Console as per these instructions, and download the CrashPlan PRO 4.9 client for Linux, and the 4.9 client for your remote console computer. Ignore the red message in the bottom left of the Admin Console about registering, and do not sign up for the free trial. Preferably use Firefox for the Linux version download – most of the other web browsers will try to unpack the .tgz archive, which you do not want to happen.
  • Configure the CrashPlan PRO 4.9 client on your computer to connect to your Syno as per the usual instructions on this blog post.
  • Put the downloaded Linux CrashPlan PRO 4.9 client .tgz file in the ‘public’ shared folder on your NAS. The package will no longer download this automatically as it did in previous versions.
  • From the Community section of DSM Package Center, install the CrashPlan PRO 4.9 package concurrently with your existing CrashPlan 4.8.3 Syno package.
  • This will stop the CrashPlan package and automatically import its configuration. Notice that it will also backup your old CrashPlan .identity file and leave it in the ‘public’ shared folder, just in case something goes wrong.
  • Start the CrashPlan PRO Synology package, and connect your CrashPlan PRO console from your computer.
  • You should see your protected folders as usual. At first mine reported something like “insufficient device licences”, but the next time I started up it changed to “subscription expired”.
  • Uninstall the CrashPlan 4.8.3 Synology package, this is no longer required.
  • At this point if the store referral didn’t work in the second step, you need to sign into the Admin Console. While signed in, navigate to this link which I was given by Code 42 support. If it works, you should see a store page with some blue font text and a $0.00 basket value. If it didn’t work you will get bounced to the Consumer Next Steps webpage: “Important Changes to CrashPlan for Home” – the one with the video of the CEO explaining the situation. I had to do this a few times before it worked. Once the store referral link worked and I had confirmed my payment details my CrashPlan PRO client immediately started working. Enjoy!
 

Notes

  • The package uses the intact CrashPlan installer directly from Code 42 Software, following acceptance of its EULA. I am complying with the directive that no one redistributes it.
  • The engine daemon script checks the amount of system RAM and scales the Java heap size appropriately (up to the default maximum of 512MB). This can be overridden in a persistent way if you are backing up large backup sets by editing /var/packages/CrashPlan/target/syno_package.vars. If you are considering buying a NAS purely to use CrashPlan and intend to back up more than a few hundred GB then I strongly advise buying one of the models with upgradeable RAM. Memory is very limited on the cheaper models. I have found that a 512MB heap was insufficient to back up more than 2TB of files on a Windows server and that was the situation many years ago. It kept restarting the backup engine every few minutes until I increased the heap to 1024MB. Many users of the package have found that they have to increase the heap size or CrashPlan will halt its activity. This can be mitigated by dividing your backup into several smaller backup sets which are scheduled to be protected at different times. Note that from package version 0041, using the dedicated JRE on a 64bit Intel NAS will allow a heap size greater than 4GB since the JRE is 64bit (requires DSM 6.0 in most cases).
  • If you need to manage CrashPlan from a remote location, I suggest you do so using SSH tunnelling as per this support document.
  • The package supports upgrading to future versions while preserving the machine identity, logs, login details, and cache. Upgrades can now take place without requiring a login from the client afterwards.
  • If you remove the package completely and re-install it later, you can re-attach to previous backups. When you log in to the Desktop Client with your existing account after a re-install, you can select “adopt computer” to merge the records, and preserve your existing backups. I haven’t tested whether this also re-attaches links to friends’ CrashPlan computers and backup sets, though the latter does seem possible in the Friends section of the GUI. It’s probably a good idea to test that this survives a package reinstall before you start relying on it. Sometimes, particularly with CrashPlan PRO I think, the adopt option is not offered. In this case you can log into CrashPlan Central and retrieve your computer’s GUID. On the CrashPlan client, double-click on the logo in the top right and you’ll enter a command line mode. You can use the GUID command to change the system’s GUID to the one you just retrieved from your account.
  • The log which is displayed in the package’s Log tab is actually the activity history. If you are trying to troubleshoot an issue you will need to use an SSH session to inspect these log files:
    /var/packages/CrashPlan/target/log/engine_output.log
    /var/packages/CrashPlan/target/log/engine_error.log
    /var/packages/CrashPlan/target/log/app.log
  • When CrashPlan downloads and attempts to run an automatic update, the script will most likely fail and stop the package. This is typically caused by syntax differences with the Synology versions of certain Linux shell commands (like rm, mv, or ps). The startup script will attempt to apply the published upgrade the next time the package is started.
  • Although CrashPlan’s activity can be scheduled within the application, in order to save RAM some users may wish to restrict running the CrashPlan engine to specific times of day using the Task Scheduler in DSM Control Panel:
    Schedule service start
    Note that regardless of real-time backup, by default CrashPlan will scan the whole backup selection for changes at 3:00am. Include this time within your Task Scheduler time window or else CrashPlan will not capture file changes which occurred while it was inactive:
    Schedule Service Start

  • If you decide to sign up for one of CrashPlan’s paid backup services as a result of my work on this, please consider donating using the PayPal button on the right of this page.
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology 3rd Party Developer Guide.

installer.sh

#!/bin/sh

#--------CRASHPLAN installer script
#--------package maintained at pcloadletter.co.uk


DOWNLOAD_PATH="http://download2.code42.com/installs/linux/install/${SYNOPKG_PKGNAME}"
CP_EXTRACTED_FOLDER="crashplan-install"
OLD_JNA_NEEDED="false"
[ "${SYNOPKG_PKGNAME}" == "CrashPlan" ] && DOWNLOAD_FILE="CrashPlan_4.8.3_Linux.tgz"
[ "${SYNOPKG_PKGNAME}" == "CrashPlanPRO" ] && DOWNLOAD_FILE="CrashPlanPRO_4.*_Linux.tgz"
if [ "${SYNOPKG_PKGNAME}" == "CrashPlanPROe" ]; then
  CP_EXTRACTED_FOLDER="${SYNOPKG_PKGNAME}-install"
  OLD_JNA_NEEDED="true"
  [ "${WIZARD_VER_483}" == "true" ] && { CPPROE_VER="4.8.3"; CP_EXTRACTED_FOLDER="crashplan-install"; OLD_JNA_NEEDED="false"; }
  [ "${WIZARD_VER_480}" == "true" ] && { CPPROE_VER="4.8.0"; CP_EXTRACTED_FOLDER="crashplan-install"; OLD_JNA_NEEDED="false"; }
  [ "${WIZARD_VER_470}" == "true" ] && { CPPROE_VER="4.7.0"; CP_EXTRACTED_FOLDER="crashplan-install"; OLD_JNA_NEEDED="false"; }
  [ "${WIZARD_VER_460}" == "true" ] && { CPPROE_VER="4.6.0"; CP_EXTRACTED_FOLDER="crashplan-install"; OLD_JNA_NEEDED="false"; }
  [ "${WIZARD_VER_452}" == "true" ] && { CPPROE_VER="4.5.2"; CP_EXTRACTED_FOLDER="crashplan-install"; OLD_JNA_NEEDED="false"; }
  [ "${WIZARD_VER_450}" == "true" ] && { CPPROE_VER="4.5.0"; CP_EXTRACTED_FOLDER="crashplan-install"; OLD_JNA_NEEDED="false"; }
  [ "${WIZARD_VER_441}" == "true" ] && { CPPROE_VER="4.4.1"; CP_EXTRACTED_FOLDER="crashplan-install"; OLD_JNA_NEEDED="false"; }
  [ "${WIZARD_VER_430}" == "true" ] && CPPROE_VER="4.3.0"
  [ "${WIZARD_VER_420}" == "true" ] && CPPROE_VER="4.2.0"
  [ "${WIZARD_VER_370}" == "true" ] && CPPROE_VER="3.7.0"
  [ "${WIZARD_VER_364}" == "true" ] && CPPROE_VER="3.6.4"
  [ "${WIZARD_VER_363}" == "true" ] && CPPROE_VER="3.6.3"
  [ "${WIZARD_VER_3614}" == "true" ] && CPPROE_VER="3.6.1.4"
  [ "${WIZARD_VER_353}" == "true" ] && CPPROE_VER="3.5.3"
  [ "${WIZARD_VER_341}" == "true" ] && CPPROE_VER="3.4.1"
  [ "${WIZARD_VER_33}" == "true" ] && CPPROE_VER="3.3"
  DOWNLOAD_FILE="CrashPlanPROe_${CPPROE_VER}_Linux.tgz"
fi
DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
CPI_FILE="${SYNOPKG_PKGNAME}_*.cpi"
OPTDIR="${SYNOPKG_PKGDEST}"
VARS_FILE="${OPTDIR}/install.vars"
SYNO_CPU_ARCH="`uname -m`"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="i686"
[ "${SYNO_CPU_ARCH}" == "armv5tel" ] && SYNO_CPU_ARCH="armel"
[ "${SYNOPKG_DSM_ARCH}" == "armada375" ] && SYNO_CPU_ARCH="armv7l"
[ "${SYNOPKG_DSM_ARCH}" == "armada38x" ] && SYNO_CPU_ARCH="armhf"
[ "${SYNOPKG_DSM_ARCH}" == "comcerto2k" ] && SYNO_CPU_ARCH="armhf"
[ "${SYNOPKG_DSM_ARCH}" == "alpine" ] && SYNO_CPU_ARCH="armhf"
[ "${SYNOPKG_DSM_ARCH}" == "alpine4k" ] && SYNO_CPU_ARCH="armhf"
[ "${SYNOPKG_DSM_ARCH}" == "monaco" ] && SYNO_CPU_ARCH="armhf"
[ "${SYNOPKG_DSM_ARCH}" == "rtd1296" ] && SYNO_CPU_ARCH="armhf"
NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/crashplan-native-${SYNO_CPU_ARCH}.tar.xz"   
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
OLD_JNA_URL="http://packages.pcloadletter.co.uk/downloads/crashplan-native-old-${SYNO_CPU_ARCH}.tar.xz"   
OLD_JNA_FILE="`echo ${OLD_JNA_URL} | sed -r "s%^.*/(.*)%\1%"`"
INSTALL_FILES="${DOWNLOAD_URL} ${NATIVE_BINS_URL}"
[ "${OLD_JNA_NEEDED}" == "true" ] && INSTALL_FILES="${INSTALL_FILES} ${OLD_JNA_URL}"
TEMP_FOLDER="`find / -maxdepth 2 -path '/volume?/@tmp' | head -n 1`"
#the Manifest folder is where friends' backup data is stored
#we set it outside the app folder so it persists after a package uninstall
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/crashplan"
LOG_FILE="${SYNOPKG_PKGDEST}/log/history.log.0"
UPGRADE_FILES="syno_package.vars conf/my.service.xml conf/service.login conf/service.model"
UPGRADE_FOLDERS="log cache"
PUBLIC_FOLDER="`synoshare --get public | sed -r "/Path/!d;s/^.*\[(.*)\].*$/\1/"`"
#dedicated JRE section
if [ "${WIZARD_JRE_CP}" == "true" ]; then
  DOWNLOAD_URL="http://tinyurl.com/javaembed"
  EXTRACTED_FOLDER="ejdk1.8.0_151"
  #detect systems capable of running 64bit JRE which can address more than 4GB of RAM
  [ "${SYNOPKG_DSM_ARCH}" == "x64" ] && SYNO_CPU_ARCH="x64"
  [ "`uname -m`" == "x86_64" ] && [ ${SYNOPKG_DSM_VERSION_MAJOR} -ge 6 ] && SYNO_CPU_ARCH="x64"
  if [ "${SYNO_CPU_ARCH}" == "armel" ]; then
    JAVA_BINARY="ejdk-8u151-linux-arm-sflt.tar.gz"
    JAVA_BUILD="ARMv5/ARMv6/ARMv7 Linux - SoftFP ABI, Little Endian 2"
  elif [ "${SYNO_CPU_ARCH}" == "armv7l" ]; then
    JAVA_BINARY="ejdk-8u151-linux-arm-sflt.tar.gz"
    JAVA_BUILD="ARMv5/ARMv6/ARMv7 Linux - SoftFP ABI, Little Endian 2"
  elif [ "${SYNO_CPU_ARCH}" == "armhf" ]; then
    JAVA_BINARY="ejdk-8u151-linux-armv6-vfp-hflt.tar.gz"
    JAVA_BUILD="ARMv6/ARMv7 Linux - VFP, HardFP ABI, Little Endian 1"
  elif [ "${SYNO_CPU_ARCH}" == "ppc" ]; then
    #Oracle have discontinued Java 8 for PowerPC after update 6
    JAVA_BINARY="ejdk-8u6-fcs-b23-linux-ppc-e500v2-12_jun_2014.tar.gz"
    JAVA_BUILD="Power Architecture Linux - Headless - e500v2 with double-precision SPE Floating Point Unit"
    EXTRACTED_FOLDER="ejdk1.8.0_06"
    DOWNLOAD_URL="http://tinyurl.com/java8ppc"
  elif [ "${SYNO_CPU_ARCH}" == "i686" ]; then
    JAVA_BINARY="ejdk-8u151-linux-i586.tar.gz"
    JAVA_BUILD="x86 Linux Small Footprint - Headless"
  elif [ "${SYNO_CPU_ARCH}" == "x64" ]; then
    JAVA_BINARY="jre-8u151-linux-x64.tar.gz"
    JAVA_BUILD="Linux x64"
    EXTRACTED_FOLDER="jre1.8.0_151"
    DOWNLOAD_URL="http://tinyurl.com/java8x64"
  fi
fi
JAVA_BINARY=`echo ${JAVA_BINARY} | cut -f1 -d'.'`
source /etc/profile


pre_checks ()
{
  #These checks are called from preinst and from preupgrade functions to prevent failures resulting in a partially upgraded package
  if [ "${WIZARD_JRE_CP}" == "true" ]; then
    synoshare -get public > /dev/null || (
      echo "A shared folder called 'public' could not be found - note this name is case-sensitive. " >> $SYNOPKG_TEMP_LOGFILE
      echo "Please create this using the Shared Folder DSM Control Panel and try again." >> $SYNOPKG_TEMP_LOGFILE
      exit 1
    )

    JAVA_BINARY_FOUND=
    [ -f ${PUBLIC_FOLDER}/${JAVA_BINARY}.tar.gz ] && JAVA_BINARY_FOUND=true
    [ -f ${PUBLIC_FOLDER}/${JAVA_BINARY}.tar ] && JAVA_BINARY_FOUND=true
    [ -f ${PUBLIC_FOLDER}/${JAVA_BINARY}.tar.tar ] && JAVA_BINARY_FOUND=true
    [ -f ${PUBLIC_FOLDER}/${JAVA_BINARY}.gz ] && JAVA_BINARY_FOUND=true
     
    if [ -z ${JAVA_BINARY_FOUND} ]; then
      echo "Java binary bundle not found. " >> $SYNOPKG_TEMP_LOGFILE
      echo "I was expecting the file ${PUBLIC_FOLDER}/${JAVA_BINARY}.tar.gz. " >> $SYNOPKG_TEMP_LOGFILE
      echo "Please agree to the Oracle licence at ${DOWNLOAD_URL}, then download the '${JAVA_BUILD}' package" >> $SYNOPKG_TEMP_LOGFILE
      echo "and place it in the 'public' shared folder on your NAS. This download cannot be automated even if " >> $SYNOPKG_TEMP_LOGFILE
      echo "displaying a package EULA could potentially cover the legal aspect, because files hosted on Oracle's " >> $SYNOPKG_TEMP_LOGFILE
      echo "server are protected by a session cookie requiring a JavaScript enabled browser." >> $SYNOPKG_TEMP_LOGFILE
      exit 1
    fi
  else
    if [ -z ${JAVA_HOME} ]; then
      echo "Java is not installed or not properly configured. JAVA_HOME is not defined. " >> $SYNOPKG_TEMP_LOGFILE
      echo "Download and install the Java Synology package from http://wp.me/pVshC-z5" >> $SYNOPKG_TEMP_LOGFILE
      exit 1
    fi

    if [ ! -f ${JAVA_HOME}/bin/java ]; then
      echo "Java is not installed or not properly configured. The Java binary could not be located. " >> $SYNOPKG_TEMP_LOGFILE
      echo "Download and install the Java Synology package from http://wp.me/pVshC-z5" >> $SYNOPKG_TEMP_LOGFILE
      exit 1
    fi

    if [ "${WIZARD_JRE_SYS}" == "true" ]; then
      JAVA_VER=`java -version 2>&1 | sed -r "/^.* version/!d;s/^.* version \"[0-9]\.([0-9]).*$/\1/"`
      if [ ${JAVA_VER} -lt 8 ]; then
        echo "This version of CrashPlan requires Java 8 or newer. Please update your Java package. "
        exit 1
      fi
    fi
  fi
}


preinst ()
{
  pre_checks
  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, " >> $SYNOPKG_TEMP_LOGFILE
        echo "which was \"${WGET_URL}\" " >> $SYNOPKG_TEMP_LOGFILE
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. " >> $SYNOPKG_TEMP_LOGFILE
        exit 1
      fi
    fi
  done
 
  exit 0
}


postinst ()
{
  if [ "${WIZARD_JRE_CP}" == "true" ]; then
    #extract Java (Web browsers love to interfere with .tar.gz files)
    cd ${PUBLIC_FOLDER}
    if [ -f ${JAVA_BINARY}.tar.gz ]; then
      #Firefox seems to be the only browser that leaves it alone
      tar xzf ${JAVA_BINARY}.tar.gz
    elif [ -f ${JAVA_BINARY}.gz ]; then
      #Chrome
      tar xzf ${JAVA_BINARY}.gz
    elif [ -f ${JAVA_BINARY}.tar ]; then
      #Safari
      tar xf ${JAVA_BINARY}.tar
    elif [ -f ${JAVA_BINARY}.tar.tar ]; then
      #Internet Explorer
      tar xzf ${JAVA_BINARY}.tar.tar
    fi
    mv ${EXTRACTED_FOLDER} ${SYNOPKG_PKGDEST}/jre-syno
    JRE_PATH="`find ${OPTDIR}/jre-syno/ -name jre`"
    [ -z ${JRE_PATH} ] && JRE_PATH=${OPTDIR}/jre-syno
    #change owner of folder tree
    chown -R root:root ${SYNOPKG_PKGDEST}
  fi
   
  #extract CPU-specific additional binaries
  mkdir ${SYNOPKG_PKGDEST}/bin
  cd ${SYNOPKG_PKGDEST}/bin
  tar xJf ${TEMP_FOLDER}/${NATIVE_BINS_FILE} && rm ${TEMP_FOLDER}/${NATIVE_BINS_FILE}
  [ "${OLD_JNA_NEEDED}" == "true" ] && tar xJf ${TEMP_FOLDER}/${OLD_JNA_FILE} && rm ${TEMP_FOLDER}/${OLD_JNA_FILE}

  #extract main archive
  cd ${TEMP_FOLDER}
  tar xzf ${TEMP_FOLDER}/${DOWNLOAD_FILE} && rm ${TEMP_FOLDER}/${DOWNLOAD_FILE} 
  
  #extract cpio archive
  cd ${SYNOPKG_PKGDEST}
  cat "${TEMP_FOLDER}/${CP_EXTRACTED_FOLDER}"/${CPI_FILE} | gzip -d -c - | ${SYNOPKG_PKGDEST}/bin/cpio -i --no-preserve-owner
  
  echo "#uncomment to expand Java max heap size beyond prescribed value (will survive upgrades)" > ${SYNOPKG_PKGDEST}/syno_package.vars
  echo "#you probably only want more than the recommended 1024M if you're backing up extremely large volumes of files" >> ${SYNOPKG_PKGDEST}/syno_package.vars
  echo "#USR_MAX_HEAP=1024M" >> ${SYNOPKG_PKGDEST}/syno_package.vars
  echo >> ${SYNOPKG_PKGDEST}/syno_package.vars

  cp ${TEMP_FOLDER}/${CP_EXTRACTED_FOLDER}/scripts/CrashPlanEngine ${OPTDIR}/bin
  cp ${TEMP_FOLDER}/${CP_EXTRACTED_FOLDER}/scripts/run.conf ${OPTDIR}/bin
  mkdir -p ${MANIFEST_FOLDER}/backupArchives    
  
  #save install variables which Crashplan expects its own installer script to create
  echo TARGETDIR=${SYNOPKG_PKGDEST} > ${VARS_FILE}
  echo BINSDIR=/bin >> ${VARS_FILE}
  echo MANIFESTDIR=${MANIFEST_FOLDER}/backupArchives >> ${VARS_FILE}
  #leave these ones out which should help upgrades from Code42 to work (based on examining an upgrade script)
  #echo INITDIR=/etc/init.d >> ${VARS_FILE}
  #echo RUNLVLDIR=/usr/syno/etc/rc.d >> ${VARS_FILE}
  echo INSTALLDATE=`date +%Y%m%d` >> ${VARS_FILE}
  [ "${WIZARD_JRE_CP}" == "true" ] && echo JAVACOMMON=${JRE_PATH}/bin/java >> ${VARS_FILE}
  [ "${WIZARD_JRE_SYS}" == "true" ] && echo JAVACOMMON=\${JAVA_HOME}/bin/java >> ${VARS_FILE}
  cat ${TEMP_FOLDER}/${CP_EXTRACTED_FOLDER}/install.defaults >> ${VARS_FILE}
  
  #remove temp files
  rm -r ${TEMP_FOLDER}/${CP_EXTRACTED_FOLDER}
  
  #add firewall config
  /usr/syno/bin/servicetool --install-configure-file --package /var/packages/${SYNOPKG_PKGNAME}/scripts/${SYNOPKG_PKGNAME}.sc > /dev/null
  
  #amend CrashPlanPROe client version
  [ "${SYNOPKG_PKGNAME}" == "CrashPlanPROe" ] && sed -i -r "s/^version=\".*(-.*$)/version=\"${CPPROE_VER}\1/" /var/packages/${SYNOPKG_PKGNAME}/INFO

  #are we transitioning an existing CrashPlan account to CrashPlan For Small Business?
  if [ "${SYNOPKG_PKGNAME}" == "CrashPlanPRO" ]; then
    if [ -e /var/packages/CrashPlan/scripts/start-stop-status ]; then
      /var/packages/CrashPlan/scripts/start-stop-status stop
      cp /var/lib/crashplan/.identity ${PUBLIC_FOLDER}/crashplan-identity.bak
      cp -R /var/packages/CrashPlan/target/conf/ ${OPTDIR}/
    fi  
  fi

  exit 0
}


preuninst ()
{
  `dirname $0`/stop-start-status stop

  exit 0
}


postuninst ()
{
  if [ -f ${SYNOPKG_PKGDEST}/syno_package.vars ]; then
    source ${SYNOPKG_PKGDEST}/syno_package.vars
  fi
  [ -e ${OPTDIR}/lib/libffi.so.5 ] && rm ${OPTDIR}/lib/libffi.so.5

  #delete symlink if it no longer resolves - PowerPC only
  if [ ! -e /lib/libffi.so.5 ]; then
    [ -L /lib/libffi.so.5 ] && rm /lib/libffi.so.5
  fi

  #remove firewall config
  if [ "${SYNOPKG_PKG_STATUS}" == "UNINSTALL" ]; then
    /usr/syno/bin/servicetool --remove-configure-file --package ${SYNOPKG_PKGNAME}.sc > /dev/null
  fi

 exit 0
}


preupgrade ()
{
  `dirname $0`/stop-start-status stop
  pre_checks
  #if identity exists back up config
  if [ -f /var/lib/crashplan/.identity ]; then
    mkdir -p ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/conf
    for FILE_TO_MIGRATE in ${UPGRADE_FILES}; do
      if [ -f ${OPTDIR}/${FILE_TO_MIGRATE} ]; then
        cp ${OPTDIR}/${FILE_TO_MIGRATE} ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FILE_TO_MIGRATE}
      fi
    done
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
      if [ -d ${OPTDIR}/${FOLDER_TO_MIGRATE} ]; then
        mv ${OPTDIR}/${FOLDER_TO_MIGRATE} ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig
      fi
    done
  fi

  exit 0
}


postupgrade ()
{
  #use the migrated identity and config data from the previous version
  if [ -f ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/conf/my.service.xml ]; then
    for FILE_TO_MIGRATE in ${UPGRADE_FILES}; do
      if [ -f ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FILE_TO_MIGRATE} ]; then
        mv ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FILE_TO_MIGRATE} ${OPTDIR}/${FILE_TO_MIGRATE}
      fi
    done
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
    if [ -d ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FOLDER_TO_MIGRATE} ]; then
      mv ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FOLDER_TO_MIGRATE} ${OPTDIR}
    fi
    done
    rmdir ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/conf
    rmdir ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig
    
    #make CrashPlan log entry
    TIMESTAMP="`date "+%D %I:%M%p"`"
    echo "I ${TIMESTAMP} Synology Package Center updated ${SYNOPKG_PKGNAME} to version ${SYNOPKG_PKGVER}" >> ${LOG_FILE}
  fi
  
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------CRASHPLAN start-stop-status script
#--------package maintained at pcloadletter.co.uk


TEMP_FOLDER="`find / -maxdepth 2 -path '/volume?/@tmp' | head -n 1`"
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/crashplan" 
ENGINE_CFG="run.conf"
PKG_FOLDER="`dirname $0 | cut -f1-4 -d'/'`"
DNAME="`dirname $0 | cut -f4 -d'/'`"
OPTDIR="${PKG_FOLDER}/target"
PID_FILE="${OPTDIR}/${DNAME}.pid"
DLOG="${OPTDIR}/log/history.log.0"
CFG_PARAM="SRV_JAVA_OPTS"
JAVA_MIN_HEAP=`grep "^${CFG_PARAM}=" "${OPTDIR}/bin/${ENGINE_CFG}" | sed -r "s/^.*-Xms([0-9]+)[Mm] .*$/\1/"` 
SYNO_CPU_ARCH="`uname -m`"
TIMESTAMP="`date "+%D %I:%M%p"`"
FULL_CP="${OPTDIR}/lib/com.backup42.desktop.jar:${OPTDIR}/lang"
source ${OPTDIR}/install.vars
source /etc/profile
source /root/.profile


start_daemon ()
{
  #check persistent variables from syno_package.vars
  USR_MAX_HEAP=0
  if [ -f ${OPTDIR}/syno_package.vars ]; then
    source ${OPTDIR}/syno_package.vars
  fi
  USR_MAX_HEAP=`echo $USR_MAX_HEAP | sed -e "s/[mM]//"`

  #do we need to restore the identity file - has a DSM upgrade scrubbed /var/lib/crashplan?
  if [ ! -e /var/lib/crashplan ]; then
    mkdir /var/lib/crashplan
    [ -e ${OPTDIR}/conf/var-backup/.identity ] && cp ${OPTDIR}/conf/var-backup/.identity /var/lib/crashplan/
  fi

  #fix up some of the binary paths and fix some command syntax for busybox 
  #moved this to start-stop-status.sh from installer.sh because Code42 push updates and these
  #new scripts will need this treatment too
  find ${OPTDIR}/ -name "*.sh" | while IFS="" read -r FILE_TO_EDIT; do
    if [ -e ${FILE_TO_EDIT} ]; then
      #this list of substitutions will probably need expanding as new CrashPlan updates are released
      sed -i "s%^#!/bin/bash%#!$/bin/sh%" "${FILE_TO_EDIT}"
      sed -i -r "s%(^\s*)(/bin/cpio |cpio ) %\1/${OPTDIR}/bin/cpio %" "${FILE_TO_EDIT}"
      sed -i -r "s%(^\s*)(/bin/ps|ps) [^w][^\|]*\|%\1/bin/ps w \|%" "${FILE_TO_EDIT}"
      sed -i -r "s%\`ps [^w][^\|]*\|%\`ps w \|%" "${FILE_TO_EDIT}"
      sed -i -r "s%^ps [^w][^\|]*\|%ps w \|%" "${FILE_TO_EDIT}"
      sed -i "s/rm -fv/rm -f/" "${FILE_TO_EDIT}"
      sed -i "s/mv -fv/mv -f/" "${FILE_TO_EDIT}"
    fi
  done

  #use this daemon init script rather than the unreliable Code42 stock one which greps the ps output
  sed -i "s%^ENGINE_SCRIPT=.*$%ENGINE_SCRIPT=$0%" ${OPTDIR}/bin/restartLinux.sh

  #any downloaded upgrade script will usually have failed despite the above changes
  #so ignore the script and explicitly extract the new java code using the chrisnelson.ca method 
  #thanks to Jeff Bingham for tweaks 
  UPGRADE_JAR=`find ${OPTDIR}/upgrade -maxdepth 1 -name "*.jar" | tail -1`
  if [ -n "${UPGRADE_JAR}" ]; then
    rm ${OPTDIR}/*.pid > /dev/null
 
    #make CrashPlan log entry
    echo "I ${TIMESTAMP} Synology extracting upgrade from ${UPGRADE_JAR}" >> ${DLOG}

    UPGRADE_VER=`echo ${SCRIPT_HOME} | sed -r "s/^.*\/([0-9_]+)\.[0-9]+/\1/"`
    #DSM 6.0 no longer includes unzip, use 7z instead
    unzip -o ${OPTDIR}/upgrade/${UPGRADE_VER}.jar "*.jar" -d ${OPTDIR}/lib/ || 7z e -y ${OPTDIR}/upgrade/${UPGRADE_VER}.jar "*.jar" -o${OPTDIR}/lib/ > /dev/null
    unzip -o ${OPTDIR}/upgrade/${UPGRADE_VER}.jar "lang/*" -d ${OPTDIR} || 7z e -y ${OPTDIR}/upgrade/${UPGRADE_VER}.jar "lang/*" -o${OPTDIR} > /dev/null
    mv ${UPGRADE_JAR} ${TEMP_FOLDER}/ > /dev/null
    exec $0
  fi

  #updates may also overwrite our native binaries
  [ -e ${OPTDIR}/bin/libffi.so.5 ] && cp -f ${OPTDIR}/bin/libffi.so.5 ${OPTDIR}/lib/
  [ -e ${OPTDIR}/bin/libjtux.so ] && cp -f ${OPTDIR}/bin/libjtux.so ${OPTDIR}/
  [ -e ${OPTDIR}/bin/jna-3.2.5.jar ] && cp -f ${OPTDIR}/bin/jna-3.2.5.jar ${OPTDIR}/lib/
  if [ -e ${OPTDIR}/bin/jna.jar ] && [ -e ${OPTDIR}/lib/jna.jar ]; then
    cp -f ${OPTDIR}/bin/jna.jar ${OPTDIR}/lib/
  fi

  #create or repair libffi.so.5 symlink if a DSM upgrade has removed it - PowerPC only
  if [ -e ${OPTDIR}/lib/libffi.so.5 ]; then
    if [ ! -e /lib/libffi.so.5 ]; then
      #if it doesn't exist, but is still a link then it's a broken link and should be deleted first
      [ -L /lib/libffi.so.5 ] && rm /lib/libffi.so.5
      ln -s ${OPTDIR}/lib/libffi.so.5 /lib/libffi.so.5
    fi
  fi

  #set appropriate Java max heap size
  RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
  if [ $RAM -le 128 ]; then
    JAVA_MAX_HEAP=80
  elif [ $RAM -le 256 ]; then
    JAVA_MAX_HEAP=192
  elif [ $RAM -le 512 ]; then
    JAVA_MAX_HEAP=384
  elif [ $RAM -le 1024 ]; then
    JAVA_MAX_HEAP=512
  elif [ $RAM -gt 1024 ]; then
    JAVA_MAX_HEAP=1024
  fi
  if [ $USR_MAX_HEAP -gt $JAVA_MAX_HEAP ]; then
    JAVA_MAX_HEAP=${USR_MAX_HEAP}
  fi   
  if [ $JAVA_MAX_HEAP -lt $JAVA_MIN_HEAP ]; then
    #can't have a max heap lower than min heap (ARM low RAM systems)
    $JAVA_MAX_HEAP=$JAVA_MIN_HEAP
  fi
  sed -i -r "s/(^${CFG_PARAM}=.*) -Xmx[0-9]+[mM] (.*$)/\1 -Xmx${JAVA_MAX_HEAP}m \2/" "${OPTDIR}/bin/${ENGINE_CFG}"
  
  #disable the use of the x86-optimized external Fast MD5 library if running on ARM and PPC CPUs
  #seems to be the default behaviour now but that may change again
  [ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="i686"
  if [ "${SYNO_CPU_ARCH}" != "i686" ]; then
    grep "^${CFG_PARAM}=.*c42\.native\.md5\.enabled" "${OPTDIR}/bin/${ENGINE_CFG}" > /dev/null \
     || sed -i -r "s/(^${CFG_PARAM}=\".*)\"$/\1 -Dc42.native.md5.enabled=false\"/" "${OPTDIR}/bin/${ENGINE_CFG}"
  fi

  #move the Java temp directory from the default of /tmp
  grep "^${CFG_PARAM}=.*Djava\.io\.tmpdir" "${OPTDIR}/bin/${ENGINE_CFG}" > /dev/null \
   || sed -i -r "s%(^${CFG_PARAM}=\".*)\"$%\1 -Djava.io.tmpdir=${TEMP_FOLDER}\"%" "${OPTDIR}/bin/${ENGINE_CFG}"

  #now edit the XML config file, which only exists after first run
  if [ -f ${OPTDIR}/conf/my.service.xml ]; then

    #allow direct connections from CrashPlan Desktop client on remote systems
    #you must edit the value of serviceHost in conf/ui.properties on the client you connect with
    #users report that this value is sometimes reset so now it's set every service startup 
    sed -i "s/<serviceHost>127\.0\.0\.1<\/serviceHost>/<serviceHost>0\.0\.0\.0<\/serviceHost>/" "${OPTDIR}/conf/my.service.xml"
    #default changed in CrashPlan 4.3
    sed -i "s/<serviceHost>localhost<\/serviceHost>/<serviceHost>0\.0\.0\.0<\/serviceHost>/" "${OPTDIR}/conf/my.service.xml"
    #since CrashPlan 4.4 another config file to allow remote console connections
    sed -i "s/127\.0\.0\.1/0\.0\.0\.0/" /var/lib/crashplan/.ui_info
     
    #this change is made only once in case you want to customize the friends' backup location
    if [ "${MANIFEST_PATH_SET}" != "True" ]; then

      #keep friends' backup data outside the application folder to make accidental deletion less likely 
      sed -i "s%<manifestPath>.*</manifestPath>%<manifestPath>${MANIFEST_FOLDER}/backupArchives/</manifestPath>%" "${OPTDIR}/conf/my.service.xml"
      echo "MANIFEST_PATH_SET=True" >> ${OPTDIR}/syno_package.vars
    fi

    #since CrashPlan version 3.5.3 the value javaMemoryHeapMax also needs setting to match that used in bin/run.conf
    sed -i -r "s%(<javaMemoryHeapMax>)[0-9]+[mM](</javaMemoryHeapMax>)%\1${JAVA_MAX_HEAP}m\2%" "${OPTDIR}/conf/my.service.xml"

    #make sure CrashPlan is not binding to the IPv6 stack
    grep "\-Djava\.net\.preferIPv4Stack=true" "${OPTDIR}/bin/${ENGINE_CFG}" > /dev/null \
     || sed -i -r "s/(^${CFG_PARAM}=\".*)\"$/\1 -Djava.net.preferIPv4Stack=true\"/" "${OPTDIR}/bin/${ENGINE_CFG}"
   else
    echo "Check the package log to ensure the package has started successfully, then stop and restart the package to allow desktop client connections." > "${SYNOPKG_TEMP_LOGFILE}"
  fi

  #increase the system-wide maximum number of open files from Synology default of 24466
  [ `cat /proc/sys/fs/file-max` -lt 65536 ] && echo "65536" > /proc/sys/fs/file-max

  #raise the maximum open file count from the Synology default of 1024 - thanks Casper K. for figuring this out
  #http://support.code42.com/Administrator/3.6_And_4.0/Troubleshooting/Too_Many_Open_Files
  ulimit -n 65536

  #ensure that Code 42 have not amended install.vars to force the use of their own (Intel) JRE
  if [ -e ${OPTDIR}/jre-syno ]; then
    JRE_PATH="`find ${OPTDIR}/jre-syno/ -name jre`"
    [ -z ${JRE_PATH} ] && JRE_PATH=${OPTDIR}/jre-syno
    sed -i -r "s|^(JAVACOMMON=).*$|\1\${JRE_PATH}/bin/java|" ${OPTDIR}/install.vars
    
    #if missing, set timezone and locale for dedicated JRE   
    if [ -z ${TZ} ]; then
      SYNO_TZ=`cat /etc/synoinfo.conf | grep timezone | cut -f2 -d'"'`
      #fix for DST time in DSM 5.2 thanks to MinimServer Syno package author
      [ -e /usr/share/zoneinfo/Timezone/synotztable.json ] \
       && SYNO_TZ=`jq ".${SYNO_TZ} | .nameInTZDB" /usr/share/zoneinfo/Timezone/synotztable.json | sed -e "s/\"//g"` \
       || SYNO_TZ=`grep "^${SYNO_TZ}" /usr/share/zoneinfo/Timezone/tzname | sed -e "s/^.*= //"`
      export TZ=${SYNO_TZ}
    fi
    [ -z ${LANG} ] && export LANG=en_US.utf8
    export CLASSPATH=.:${OPTDIR}/jre-syno/lib

  else
    sed -i -r "s|^(JAVACOMMON=).*$|\1\${JAVA_HOME}/bin/java|" ${OPTDIR}/install.vars
  fi

  source ${OPTDIR}/bin/run.conf
  source ${OPTDIR}/install.vars
  cd ${OPTDIR}
  $JAVACOMMON $SRV_JAVA_OPTS -classpath $FULL_CP com.backup42.service.CPService > ${OPTDIR}/log/engine_output.log 2> ${OPTDIR}/log/engine_error.log &
  if [ $! -gt 0 ]; then
    echo $! > $PID_FILE
    renice 19 $! > /dev/null
    if [ -z "${SYNOPKG_PKGDEST}" ]; then
      #script was manually invoked, need this to show status change in Package Center      
      [ -e ${PKG_FOLDER}/enabled ] || touch ${PKG_FOLDER}/enabled
    fi
  else
    echo "${DNAME} failed to start, check ${OPTDIR}/log/engine_error.log" > "${SYNOPKG_TEMP_LOGFILE}"
    echo "${DNAME} failed to start, check ${OPTDIR}/log/engine_error.log" >&2
    exit 1
  fi
}

stop_daemon ()
{
  echo "I ${TIMESTAMP} Stopping ${DNAME}" >> ${DLOG}
  kill `cat ${PID_FILE}`
  wait_for_status 1 20 || kill -9 `cat ${PID_FILE}`
  rm -f ${PID_FILE}
  if [ -z ${SYNOPKG_PKGDEST} ]; then
    #script was manually invoked, need this to show status change in Package Center
    [ -e ${PKG_FOLDER}/enabled ] && rm ${PKG_FOLDER}/enabled
  fi
  #backup identity file in case DSM upgrade removes it
  [ -e ${OPTDIR}/conf/var-backup ] || mkdir ${OPTDIR}/conf/var-backup 
  cp /var/lib/crashplan/.identity ${OPTDIR}/conf/var-backup/
}

daemon_status ()
{
  if [ -f ${PID_FILE} ] && kill -0 `cat ${PID_FILE}` > /dev/null 2>&1; then
    return
  fi
  rm -f ${PID_FILE}
  return 1
}

wait_for_status ()
{
  counter=$2
  while [ ${counter} -gt 0 ]; do
    daemon_status
    [ $? -eq $1 ] && return
    let counter=counter-1
    sleep 1
  done
  return 1
}


case $1 in
  start)
    if daemon_status; then
      echo ${DNAME} is already running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo Starting ${DNAME} ...
      start_daemon
      exit $?
    fi
  ;;

  stop)
    if daemon_status; then
      echo Stopping ${DNAME} ...
      stop_daemon
      exit $?
    else
      echo ${DNAME} is not running
      exit 0
    fi
  ;;

  restart)
    stop_daemon
    start_daemon
    exit $?
  ;;

  status)
    if daemon_status; then
      echo ${DNAME} is running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo ${DNAME} is not running
      exit 1
    fi
  ;;

  log)
    echo "${DLOG}"
    exit 0
  ;;

  *)
    echo "Usage: $0 {start|stop|status|restart}" >&2
    exit 1
  ;;

esac
 

install_uifile & upgrade_uifile

[
  {
    "step_title": "Client Version Selection",
    "items": [
      {
        "type": "singleselect",
        "desc": "Please select the CrashPlanPROe client version that is appropriate for your backup destination server:",
        "subitems": [
          {
            "key": "WIZARD_VER_483",
            "desc": "4.8.3",
            "defaultValue": true
          },          {
            "key": "WIZARD_VER_480",
            "desc": "4.8.0",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_470",
            "desc": "4.7.0",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_460",
            "desc": "4.6.0",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_452",
            "desc": "4.5.2",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_450",
            "desc": "4.5.0",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_441",
            "desc": "4.4.1",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_430",
            "desc": "4.3.0",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_420",
            "desc": "4.2.0",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_370",
            "desc": "3.7.0",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_364",
            "desc": "3.6.4",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_363",
            "desc": "3.6.3",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_3614",
            "desc": "3.6.1.4",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_353",
            "desc": "3.5.3",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_341",
            "desc": "3.4.1",
            "defaultValue": false
          },
          {
            "key": "WIZARD_VER_33",
            "desc": "3.3",
            "defaultValue": false
          }
        ]
      }
    ]
  },
  {
    "step_title": "Java Runtime Environment Selection",
    "items": [
      {
        "type": "singleselect",
        "desc": "Please select the Java version which you would like CrashPlan to use:",
        "subitems": [
          {
            "key": "WIZARD_JRE_SYS",
            "desc": "Default system Java version",
            "defaultValue": false
          },
          {
            "key": "WIZARD_JRE_CP",
            "desc": "Dedicated installation of Java 8",
            "defaultValue": true
          }
        ]
      }
    ]
  }
]
 

Changelog:

  • 0031 Added TCP 4242 to the firewall services (computer to computer connections)
  • 0047 30/Oct/17 – Updated dedicated Java version to 8 update 151, added support for additional Intel CPUs in x18 Synology products.
  • 0046 26/Aug/17 – Updated to CrashPlan PRO 4.9, added support for migration from CrashPlan For Home to CrashPlan For Small Business (CrashPlan PRO). Please read the Migration section on this page for instructions.
  • 0045 02/Aug/17 – Updated to CrashPlan 4.8.3, updated dedicated Java version to 8 update 144
  • 0044 21/Jan/17 – Updated dedicated Java version to 8 update 121
  • 0043 07/Jan/17 – Updated dedicated Java version to 8 update 111, added support for Intel Broadwell and Grantley CPUs
  • 0042 03/Oct/16 – Updated to CrashPlan 4.8.0, Java 8 is now required, added optional dedicated Java 8 Runtime instead of the default system one including 64bit Java support on 64 bit Intel CPUs to permit memory allocation larger than 4GB. Support for non-Intel platforms withdrawn owing to Code42’s reliance on proprietary native code library libc42archive.so
  • 0041 20/Jul/16 – Improved auto-upgrade compatibility (hopefully), added option to have CrashPlan use a dedicated Java 7 Runtime instead of the default system one, including 64bit Java support on 64 bit Intel CPUs to permit memory allocation larger than 4GB
  • 0040 25/May/16 – Added cpio to the path in the running context of start-stop-status.sh
  • 0039 25/May/16 – Updated to CrashPlan 4.7.0, at each launch forced the use of the system JRE over the CrashPlan bundled Intel one, added Maven build of JNA 4.1.0 for ARMv7 systems consistent with the version bundled with CrashPlan
  • 0038 27/Apr/16 – Updated to CrashPlan 4.6.0, and improved support for Code 42 pushed updates
  • 0037 21/Jan/16 – Updated to CrashPlan 4.5.2
  • 0036 14/Dec/15 – Updated to CrashPlan 4.5.0, separate firewall definitions for management client and for friends backup, added support for DS716+ and DS216play
  • 0035 06/Nov/15 – Fixed the update to 4.4.1_59, new installs now listen for remote connections after second startup (was broken from 4.4), updated client install documentation with more file locations and added a link to a new Code42 support doc
    EITHER completely remove and reinstall the package (which will require a rescan of the entire backup set) OR alternatively please delete all except for one of the failed upgrade numbered subfolders in /var/packages/CrashPlan/target/upgrade before upgrading. There will be one folder for each time CrashPlan tried and failed to start since Code42 pushed the update
  • 0034 04/Oct/15 – Updated to CrashPlan 4.4.1, bundled newer JNA native libraries to match those from Code42, PLEASE READ UPDATED BLOG POST INSTRUCTIONS FOR CLIENT INSTALL this version introduced yet another requirement for the client
  • 0033 12/Aug/15 – Fixed version 0032 client connection issue for fresh installs
  • 0032 12/Jul/15 – Updated to CrashPlan 4.3, PLEASE READ UPDATED BLOG POST INSTRUCTIONS FOR CLIENT INSTALL this version introduced an extra requirement, changed update repair to use the chrisnelson.ca method, forced CrashPlan to prefer IPv4 over IPv6 bindings, removed some legacy version migration scripting, updated main blog post documentation
  • 0031 20/May/15 – Updated to CrashPlan 4.2, cross compiled a newer cpio binary for some architectures which were segfaulting while unpacking main CrashPlan archive, added port 4242 to the firewall definition (friend backups), package is now signed with repository private key
  • 0030 16/Feb/15 – Fixed show-stopping issue with version 0029 for systems with more than one volume
  • 0029 21/Jan/15 – Updated to CrashPlan version 3.7.0, improved detection of temp folder (prevent use of /var/@tmp), added support for Annapurna Alpine AL514 CPU (armhf) in DS2015xs, added support for Marvell Armada 375 CPU (armhf) in DS215j, abandoned practical efforts to try to support Code42’s upgrade scripts, abandoned inotify support (realtime backup) on PowerPC after many failed attempts with self-built and pre-built jtux and jna libraries, back-merged older libffi support for old PowerPC binaries after it was removed in 0028 re-write
  • 0028 22/Oct/14 – Substantial re-write:
    Updated to CrashPlan version 3.6.4
    DSM 5.0 or newer is now required
    libjnidispatch.so taken from Debian JNA 3.2.7 package with dependency on newer libffi.so.6 (included in DSM 5.0)
    jna-3.2.5.jar emptied of irrelevant CPU architecture libs to reduce size
    Increased default max heap size from 512MB to 1GB on systems with more than 1GB RAM
    Intel CPUs no longer need the awkward glibc version-faking shim to enable inotify support (for real-time backup)
    Switched to using root account – no more adding account permissions for backup, package upgrades will no longer break this
    DSM Firewall application definition added
    Tested with DSM Task Scheduler to allow backups between certain times of day only, saving RAM when not in use
    Daemon init script now uses a proper PID file instead of Code42’s unreliable method of using grep on the output of ps
    Daemon init script can be run from the command line
    Removal of bash binary dependency now Code42’s CrashPlanEngine script is no longer used
    Removal of nice binary dependency, using BusyBox equivalent renice
    Unified ARMv5 and ARMv7 external binary package (armle)
    Added support for Mindspeed Comcerto 2000 CPU (comcerto2k – armhf) in DS414j
    Added support for Intel Atom C2538 (avoton) CPU in DS415+
    Added support to choose which version of CrashPlan PROe client to download, since some servers may still require legacy versions
    Switched to .tar.xz compression for native binaries to reduce web hosting footprint
  • 0027 20/Mar/14 – Fixed open file handle limit for very large backup sets (ulimit fix)
  • 0026 16/Feb/14 – Updated all CrashPlan clients to version 3.6.3, improved handling of Java temp files
  • 0025 30/Jan/14 – glibc version shim no longer used on Intel Synology models running DSM 5.0
  • 0024 30/Jan/14 – Updated to CrashPlan PROe 3.6.1.4 and added support for PowerPC 2010 Synology models running DSM 5.0
  • 0023 30/Jan/14 – Added support for Intel Atom Evansport and Armada XP CPUs in new DSx14 products
  • 0022 10/Jun/13 – Updated all CrashPlan client versions to 3.5.3, compiled native binary dependencies to add support for Armada 370 CPU (DS213j), start-stop-status.sh now updates the new javaMemoryHeapMax value in my.service.xml to the value defined in syno_package.vars
  • 0021 01/Mar/13 – Updated CrashPlan to version 3.5.2
  • 0020 21/Jan/13 – Fixes for DSM 4.2
  • 018 Updated CrashPlan PRO to version 3.4.1
  • 017 Updated CrashPlan and CrashPlan PROe to version 3.4.1, and improved in-app update handling
  • 016 Added support for Freescale QorIQ CPUs in some x13 series Synology models, and installer script now downloads native binaries separately to reduce repo hosting bandwidth, PowerQUICC PowerPC processors in previous Synology generations with older glibc versions are not supported
  • 015 Added support for easy scheduling via cron – see updated Notes section
  • 014 DSM 4.1 user profile permissions fix
  • 013 implemented update handling for future automatic updates from Code 42, and incremented CrashPlanPRO client to release version 3.2.1
  • 012 incremented CrashPlanPROe client to release version 3.3
  • 011 minor fix to allow a wildcard on the cpio archive name inside the main installer package (to fix CP PROe client since Code 42 Software had amended the cpio file version to 3.2.1.2)
  • 010 minor bug fix relating to daemon home directory path
  • 009 rewrote the scripts to be even easier to maintain and unified as much as possible with my imminent CrashPlan PROe server package, fixed a timezone bug (tightened regex matching), moved the script-amending logic from installer.sh to start-stop-status.sh with it now applying to all .sh scripts each startup so perhaps updates from Code42 might work in future, if wget fails to fetch the installer from Code42 the installer will look for the file in the public shared folder
  • 008 merged the 14 package scripts each (7 for ARM, 7 for Intel) for CP, CP PRO, & CP PROe – 42 scripts in total – down to just two! ARM & Intel are now supported by the same package, Intel synos now have working inotify support (Real-Time Backup) thanks to rwojo’s shim to pass the glibc version check, upgrade process now retains login, cache and log data (no more re-scanning), users can specify a persistent larger max heap size for very large backup sets
  • 007 fixed a bug that broke CrashPlan if the Java folder moved (if you changed version)
  • 006 installation now fails without User Home service enabled, fixed Daylight Saving Time support, automated replacing the ARM libffi.so symlink which is destroyed by DSM upgrades, stopped assuming the primary storage volume is /volume1, reset ownership on /var/lib/crashplan and the Friends backup location after installs and upgrades
  • 005 added warning to restart daemon after 1st run, and improved upgrade process again
  • 004 updated to CrashPlan 3.2.1 and improved package upgrade process, forced binding to 0.0.0.0 each startup
  • 003 fixed ownership of /volume1/crashplan folder
  • 002 updated to CrashPlan 3.2
  • 001 30/Jan/12 – intial public release
 
 

Serviio 2.0 package for Synology NAS

Serviio 1.6 in DSM Package Center

Update – Serviio version 2.0 is a new major version, so existing Serviio 1.x licences will not unlock the Pro features without a fresh licence purchase. If you wish to downgrade back to version 1.10.1 you may manually install the older package (merged architectures, or evansport) but your media library will need to be rebuilt.

Running the Serviio DLNA server on a NAS is a very efficient solution. It allows all your media to be constantly available to all your devices, all without the need for a dedicated media PC. Most TV and Bluray players are already capable media renderers – so indexing the library, retrieving metadata, and streaming the content are where Serviio fits in. Device support is continually improved by a community of contributors, and this crowd-sourced aspect was what initially drew me to the project, leading me to create the Sony Bluray device profile back in 2010. Serviio can also connect to online sources of content such as catch-up TV portals and allow fuss free content playback without advertising, even on devices which lack native support. These online plugins are also community maintained.

This new version of the Synology package updates Serviio to version 2.0 which fixes some issues and introduces multiple user support (release notes). Version 1.10 improved MediaBrowser, and also 4K media support. Version 1.9 replaced the Flash Player dependency in MediaBrowser with an HTML5 player. Serviio 1.7 brought audio track selection preferences, and automatic subtitle downloading. The most significant enhancement from version 1.6 onwards is the new web management console which removes the need to install the separate Serviio Console application on a computer. The web console uses a modern adaptive design which works well on mobile devices, and it is now integrated with DSM so you will see its icon in the application menu when the package is running:

Serviio-DSM-integration
Serviio 1.6 web console
The package supports hardware transcoding on the DS214play and DS415play with Intel Evansport media processor. This support required substantial personal effort and was made possible by the very kind donation of a new DS214play unit from MrK/Boretom a fellow package maintainer for ASUStor devices.

 

Limitations

If you use a TV, Bluray or media player bought in the last few years it is likely to have very good format support. In this case even a low specification NAS will be adequate for streaming most video content. There are some limitations however, in particular with video transcoding.

Serviio’s MediaBrowser is a Pro version only feature, and is designed to allow direct playback of media on tablet devices without the need for dedicated apps. Please note that Apple iOS devices require video in H.264 and audio in stereo AAC format when streaming. With the exception of the “play” models already mentioned, most Synology NAS models lack the CPU power to transcode video into H.264 in realtime. Even if the source video is already H.264, ARMv5 family CPUs (which lack an FPU) in the older models lack the power even to transcode AAC audio in realtime without libfdk_aac. Unfortunately I am unable to distribute an FFmpeg binary compiled against this library under the terms of its software license.

Ever since Serviio version 1.2 renderers that do not natively support subtitles via DLNA can have them burned into the video during transcoding, but as already explained a NAS usually lacks the CPU power to do this (more info on subs here). If you absolutely need hard subs for a particular film, you could start playback on your device then stop it and Serviio will continue transcoding. Then you will be able to try again in a few hours since it will play from the transcode cache folder. Newer TVs tend to support inline SRT subs, but for those with older models I can recommend the PlayStation 4 Media Player as a competent DLNA renderer with inline subs support.

I was able to compile FFmpeg for all eight supported architectures (ARMv5, ARMv7, ARMv7 with NEON, ARMv8, Intel i686, Intel x64, Intel Evansport media processor, and PowerPC e500v2) with the additional libraries Serviio requires while using the libs included within DSM as far as possible. I have published my compilation methods to make the process easier for others in future since parts of it were difficult to figure out. Only the Intel CPU or higher end ARM CPUs are likely to be of any practical use transcoding to H.264, since the other CPU types lack vector units (no AltiVec in QorIQ/PowerQUICC, and no NEON in Armada 370/XP).

 

Synology Package Installation

  • In Synology DSM’s Package Center, click Settings and add my package repository:
    Add Package Repository
  • The repository will push its certificate automatically to the NAS, which is used to validate package integrity. Set the Trust Level to Synology Inc. and trusted publishers:
    Trust Level
  • Serviio now requires Java 8 for OS-native filesystem watching, so you will need to install my Java SE Embedded package first if you have not already done so. Read the instructions on that page carefully too.
  • Now browse the Community section in Package Center to install Serviio:
    Community-packages
    Community packages are not listed in the All category for some reason. The repository only displays packages which are compatible with your specific model of NAS. If you don’t see Serviio in the list, then either your NAS model or your DSM version are not supported at this time. DSM 6.0 is the minimum supported version for this package.
  • When Serviio is started for the first time, a plugins folder will be created in the public shared folder (usually /volume1/public/serviio/plugins/). Download any Serviio online content plugins that you require and place the .groovy files in that location.
  • If you need subtitle support for glyphs not present in the default DejaVu fonts, add additional fonts to the following folder inside your NAS’s public shared folder (usually /volume1/public/serviio/fonts/).
 

Notes

  • You should reserve an IP address for your NAS (most home broadband routers will let you do this in the DHCP options), or use a static IP address. This will prevent Serviio from appearing multiple times in the menus of your renderer devices as its IP changes.
  • In the console disable ‘Generate thumbnails for local videos’ in the Metadata tab. Failure to do this can make adding videos to your library very slow on older NAS systems. With this disabled Serviio will still fetch thumbnail images automatically from the online databases.
  • Don’t alter the transcoded files location on the Transcoding tab, it’s already set to a sensible location for Synology systems. This particular folder is deleted each time your NAS restarts, and Serviio cleans up its temp files automatically in any case.
  • When adding folders to the media library use the Browse button. This will show the root of the entire NAS Linux filesystem, so you will find your DSM shared folders inside /volume1.
  • If you’re upgrading from a previous Serviio version you should check for updates of your online content plugins.
  • MediaBrowser (Pro only feature) can be opened using the link in the side bar of the web console (http://IPofYourNAS:23424/mediabrowser). If you want to use the MediaBrowser from a remote location you would need to either port forward 23424 on your router, or use SSH tunnelling. You can also manage Serviio remotely by SSH tunnelling port 23423 which the console uses. Both of these service definitions are added to the DSM Firewall.
  • Advanced users who wish to edit device profiles or enable debug logging can find the Serviio files in /var/packages/Serviio/target. You will need to use an SSH session to access this folder – it cannot be navigated to using DSM’s File Station. You will need to restart the Serviio package for profile changes to take effect.
  • Some users may wish to restrict running Serviio to specific times of day in order to free up RAM for a scheduled backup window for instance. This can be achieved by creating tasks to start and stop Serviio using the Task Scheduler in DSM Control Panel:
    Schedule service start
  • The package uses an FFmpeg wrapper script to modify the usual transcoding behaviour of Serviio to use the Intel Evansport hardware decoding and encoding available in the DS214play and DS415play models. I have left the FFmpeg wrapper scripts in the package even when not in use (in the Serviio/bin folder), because they are still useful for people trying to test a particular workaround or encoder setting. The wrapper was a tricky script to get working due to variable expansion precedence issues and quote handling. If you need it, just change the ffmpeg.location system property in /var/packages/Serviio/target/bin/serviio.sh to point to your modified wrapper script.
  • If additional fonts are not working as expected, some troubleshooting is available. You can re-create the fontconfig cache by connecting to the NAS via an SSH session as admin, and then running:
    sudo -i
    (enter admin password to become root)
    rm -rf ~/.fontconfig.cache
    FC_DEBUG=128 /var/packages/Serviio/target/bin/fc-cache --verbose
 

Hardware transcoding support

  • DS214play and DS415play with Intel Evansport SoC are the only models supported for hardware transcoding. Serviio uses the multimedia tool FFmpeg for manipulating media files, and I am only able to build a hardware-assisted FFmpeg for DS214play and DS215play. Subsequent to those products, Synology has marketed a number of other systems with hardware transcoding features but these are not supported by Serviio.
    The DS216play with STiH412 Monaco SoC uses a specialised build of Gstreamer for the Synology transcoding solution rather than FFmpeg, so its hardware features cannot be used by Serviio.
    It seems likely that Synology is also using Gstreamer for the Intel Braswell or newer generation CPUs with QuikSync, since DSM ships with an older FFmpeg (2.7.1) than the version which introduced QuikSync support (2.8.0). To implement FFmpeg QuikSync hardware transcoding support requires Linux kernel patches for libmfx support which is not currently included in DSM 6.x, so Serviio support for Intel QuikSync on Synology is unlikely unless Synology in future switches to using FFmpeg for its own hardware transcoding solution.
    The DS218play with Realtek RTD1296 SoC uses Gstreamer with OpenMAX drivers for the Synology transcoding solution. Although I was able to compile the ARMv8 build of FFmpeg with libomx support, unfortunately the Belaggio OpenMAX IL will not register these Realtek drivers which are distributed in the VideoStation package, therefore the hardware transcoding features of this model cannot be used by Serviio.
  • For the supported systems FFmpeg will use hardware decoding wherever possible.
  • Although multiple hardware assisted decodes can take place simultaneously, only a single hardware encode is available. The package intelligently avoids race conditions for this.
  • Any FFmpeg commandline to encode using libx264 will automatically be sent to the hardware encoder when available.
  • profiles.xml has been modified so that the preferred transcode format for most devices for non-natively supported videos is H.264.
  • In order to add hardware H.264 encoding to a device profile, specify h264 as the desired codec and the FFmpeg wrapper script will make the necessary substitution automatically.
  • When the hardware encoder is used, the low quality settings in Serviio’s default libx264 options to favour encoding speed are ignored. The hardware encoder performance is not altered by quality settings.
  • If a media stream is being both hardware decoded and hardware encoded, it will use pipelined mode. Based on responses from Synology support during development this means that both jobs occur on the media processor without exposing the intermediate bitstream to FFmpeg. Consequently it is not possible to burn subtitles in pipelined mode. As a result my FFmpeg wrapper script will run the decode part of the job in software in order to do this. The unfortunate result is that the encoder cannot quite achieve real time transcoding during subtitle compositing. For full 1080p HD material it achieves around 20fps, whereas most movies are 24fps. This means that in order to watch a film with hard subs you would need to start it off, pause or stop the renderer and return a bit later to view. An alternative would be to mux the content into an MKV container with the desired subtitles before adding it to the Serviio library, assuming that your device supports SRTs in MKV.
  • The hardware encoded H.264 streams are not 100% compliant with the specification and fail validation using the tool DGAVCDec. For most tested renderers this does not affect playback (Bravia, Sony Bluray, iOS). FlowPlayer which had been the embedded player in MediaBrowser did not play these files satisfactorily and dropped many frames resulting in jerky playback. However since Serviio version 1.9 MediaBrowser uses an HTML5 player which is not affected by this issue.
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology 3rd Party Developer Guide.

installer.sh

#!/bin/sh

#--------SERVIIO installer script
#--------package maintained at pcloadletter.co.uk


DOWNLOAD_PATH="http://download.serviio.org/releases"
DOWNLOAD_FILE="serviio-2.0-linux.tar.gz"
EXTRACTED_FOLDER="serviio-2.0"
DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
SYNO_CPU_ARCH="`uname -m`"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="x64"
[ "${SYNOPKG_DSM_ARCH}" == "comcerto2k" ] && SYNO_CPU_ARCH="armneon"
[ "${SYNOPKG_DSM_ARCH}" == "armada375" ] && SYNO_CPU_ARCH="armneon"
[ "${SYNOPKG_DSM_ARCH}" == "armada38x" ] && SYNO_CPU_ARCH="armneon"
[ "${SYNOPKG_DSM_ARCH}" == "alpine" ] && SYNO_CPU_ARCH="armneon"
[ "${SYNOPKG_DSM_ARCH}" == "alpine4k" ] && SYNO_CPU_ARCH="armneon"
[ "${SYNOPKG_DSM_ARCH}" == "monaco" ] && SYNO_CPU_ARCH="armneon"
[ "${WIZARD_ENC_HW}" == "true" ] && SYNO_CPU_ARCH="i686evansport"
NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/serviio-native-${SYNO_CPU_ARCH}.tar.xz"   
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
FONTS_URL="http://sourceforge.net/projects/dejavu/files/dejavu/2.37/dejavu-fonts-ttf-2.37.tar.bz2"
FONTS_FILE="`echo ${FONTS_URL} | sed -r "s%^.*/(.*)%\1%"`"
#'ua' prefix means wget user-agent will be customized
INSTALL_FILES="ua${DOWNLOAD_URL} ${NATIVE_BINS_URL} ${FONTS_URL}"
PID_FILE="${SYNOPKG_PKGDEST}/serviio.pid"
COMMENT="# Synology Serviio Package"
TEMP_FOLDER="`find / -maxdepth 2 -path '/volume?/@tmp' | head -n 1`"
PUBLIC_FOLDER="`synoshare --get public | sed -r "/Path/!d;s/^.*\[(.*)\].*$/\1/"`"
PLUGINS_PATH="${PUBLIC_FOLDER}/serviio"
source /etc/profile


pre_checks ()
{
  #These checks are called from preinst and from preupgrade functions to prevent failures resulting in a partially upgraded package
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. " >> $SYNOPKG_TEMP_LOGFILE
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5" >> $SYNOPKG_TEMP_LOGFILE
    exit 1
  fi

  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. " >> $SYNOPKG_TEMP_LOGFILE
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5" >> $SYNOPKG_TEMP_LOGFILE
    exit 1
  fi

  JAVA_VER=`java -version 2>&1 | sed -r "/^.* version/!d;s/^.* version \"[0-9]\.([0-9]).*$/\1/"`
  if [ ${JAVA_VER} -lt 8 ]; then
    echo "This version of Serviio requires Java 8 or newer. Please update your Java package. " >> $SYNOPKG_TEMP_LOGFILE
    exit 1
  fi

  if [ -z ${PUBLIC_FOLDER} ]; then
    echo "A shared folder called 'public' could not be found - note this name is case-sensitive. " >> $SYNOPKG_TEMP_LOGFILE
    echo "Please create this using the Shared Folder DSM Control Panel and try again." >> $SYNOPKG_TEMP_LOGFILE
    exit 1
  fi
}


preinst ()
{
  pre_checks
  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    #this will allow serviio.org to track the number of downloads from Synology users
    WGET_URL=`echo ${WGET_URL} | sed -e "s/^ua/--user-agent=Synology --referer=http:\/\/pcloadletter.co.uk\/2012\/01\/25\/serviio-syno-package /"`
    wget ${WGET_URL}
    if [ $? != 0 ]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, " >> $SYNOPKG_TEMP_LOGFILE
        echo "which was \"${WGET_URL}\" " >> $SYNOPKG_TEMP_LOGFILE
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. " >> $SYNOPKG_TEMP_LOGFILE
        exit 1
      fi
    fi
  done

  exit 0
}


postinst ()
{
  #extract the downloaded Serviio archive
  cd ${TEMP_FOLDER}
  tar xzf ${TEMP_FOLDER}/${DOWNLOAD_FILE}
  rm ${TEMP_FOLDER}/${DOWNLOAD_FILE}
  cp -R ${TEMP_FOLDER}/${EXTRACTED_FOLDER}/* ${SYNOPKG_PKGDEST}
  if [ ! -z "${EXTRACTED_FOLDER}" ]; then
    rm -r ${TEMP_FOLDER}/${EXTRACTED_FOLDER}
  fi
  if [ ! -d "${PLUGINS_PATH}/plugins" ]; then
    mkdir -p ${PLUGINS_PATH}/plugins
  fi
  if [ ! -d "${PLUGINS_PATH}/fonts" ]; then
    mkdir -p ${PLUGINS_PATH}/fonts
  fi

  #extract CPU-specific additional binaries
  cd ${SYNOPKG_PKGDEST}/lib
  tar xJf ${TEMP_FOLDER}/${NATIVE_BINS_FILE} && rm ${TEMP_FOLDER}/${NATIVE_BINS_FILE}
  mv ${SYNOPKG_PKGDEST}/lib/ffmpeg ${SYNOPKG_PKGDEST}/bin
  mv ${SYNOPKG_PKGDEST}/lib/fc-cache ${SYNOPKG_PKGDEST}/bin
  [ -e ${SYNOPKG_PKGDEST}/lib/omxregister-bellagio ] mv ${SYNOPKG_PKGDEST}/lib/omxregister-bellagio ${SYNOPKG_PKGDEST}/bin

  #remove legacy package font versions
  [ -d ${PLUGINS_PATH}/fonts/dejavu-fonts-ttf-2.33/ ] && rm -rf ${PLUGINS_PATH}/fonts/dejavu-fonts-ttf-2.33/
  [ -d ${PLUGINS_PATH}/fonts/dejavu-fonts-ttf-2.34/ ] && rm -rf ${PLUGINS_PATH}/fonts/dejavu-fonts-ttf-2.34/
  [ -d ${PLUGINS_PATH}/fonts/dejavu-fonts-ttf-2.35/ ] && rm -rf ${PLUGINS_PATH}/fonts/dejavu-fonts-ttf-2.35/

  #extract open source font package for subtitle support during transcoding
  cd ${PLUGINS_PATH}/fonts
  tar xvjf ${TEMP_FOLDER}/${FONTS_FILE} && rm ${TEMP_FOLDER}/${FONTS_FILE}
  sed -i "s|WINDOWSFONTDIR|${PLUGINS_PATH}/fonts|" ${SYNOPKG_PKGDEST}/config/fonts/fonts.conf
  sed -i "s|WINDOWSTEMPDIR_FONTCONFIG_CACHE|~/.fontconfig.cache|" ${SYNOPKG_PKGDEST}/config/fonts/fonts.conf

  #wrapper script can be useful for testing different encoder options
  if [ -e "${SYNOPKG_PKGDEST}/bin/ffmpeg-wrapper-${SYNO_CPU_ARCH}.sh" ]; then
    #we need to use the wrapper to make FFmpeg use libshine on ARM systems and to use hardware decode/encode on Intel Evansport systems
    FFMPEG_PATH="\$SERVIIO_HOME/bin/ffmpeg-wrapper-${SYNO_CPU_ARCH}.sh"
  else
    FFMPEG_PATH="\$SERVIIO_HOME/bin/ffmpeg"
  fi

  #modifications to device profiles (evansport hardware transcoding)
  if [ -e "${SYNOPKG_PKGDEST}/config/profiles-${SYNO_CPU_ARCH}.xml" ]; then
    mv "${SYNOPKG_PKGDEST}/config/profiles.xml" "${SYNOPKG_PKGDEST}/config/profiles-orig.xml"
    mv "${SYNOPKG_PKGDEST}/config/profiles-${SYNO_CPU_ARCH}.xml" "${SYNOPKG_PKGDEST}/config/profiles.xml"
  fi

  #modifications to application profiles (evansport hardware transcoding)
  #removed owing to frame drops in FlowPlayer because encoder does not produce 100% valid streams, evansport has sufficient power for flv encoding
  #if [ -e "${SYNOPKG_PKGDEST}/config/application-profiles-${SYNO_CPU_ARCH}.xml" ]; then
  #  mv "${SYNOPKG_PKGDEST}/config/application-profiles-${SYNO_CPU_ARCH}.xml" "${SYNOPKG_PKGDEST}/config/application-profiles.xml"
  #fi

  #make changes to Serviio launcher script so that pid file is created for Java process
  sed -r -i "s%Execute the JVM in the foreground%Execute the JVM in the background%" ${SYNOPKG_PKGDEST}/bin/serviio.sh
  sed -r -i "s%^(exec \"$JAVA.*)$%\1 \&%" ${SYNOPKG_PKGDEST}/bin/serviio.sh
  echo "echo \$! > ${PID_FILE}" >> ${SYNOPKG_PKGDEST}/bin/serviio.sh

  #set some additional Serviio system properties (temp folder, FFmpeg path, plugins folder)
  #http://www.serviio.org/index.php?option=com_content&view=article&id=43
  EXTRA_OPTS="-Dserviio\.defaultTranscodeFolder=${TEMP_FOLDER} -Dffmpeg\.location=${FFMPEG_PATH} -Dplugins\.location=${PLUGINS_PATH}"
  #fix Java prefs checking which was preventing NAS hibernation http://forum.serviio.org/viewtopic.php?f=5&t=6878
  EXTRA_OPTS="${EXTRA_OPTS} -Djava.util.prefs.syncInterval=86400"
  if [ "${SYNO_CPU_ARCH}" == "armv5tel" ]; then
    #use integer math (not floating point) Dolby AC-3 encoder for better performance on ARM CPUs
    #http://ffmpeg.org/ffmpeg.html#ac3-and-ac3_005ffixed
    EXTRA_OPTS="${EXTRA_OPTS} -Dserviio\.fixedPointEncoders"
  fi
  sed -r -i "s% -Dffmpeg\.location=ffmpeg%%; s%^(JAVA_OPTS=.*)\"$%\1 ${EXTRA_OPTS}\"%" ${SYNOPKG_PKGDEST}/bin/serviio.sh

  #create log file to allow package start errors to be captured
  [ -e ${SYNOPKG_PKGDEST}/log ] || mkdir ${SYNOPKG_PKGDEST}/log
  [ -e ${SYNOPKG_PKGDEST}/log/serviio.log ] || touch ${SYNOPKG_PKGDEST}/log/serviio.log

  #add firewall config
  /usr/syno/bin/servicetool --install-configure-file --package /var/packages/${SYNOPKG_PKGNAME}/conf/${SYNOPKG_PKGNAME}.sc > /dev/null

  exit 0
}


preuninst ()
{
  `dirname $0`/stop-start-status stop

  exit 0
}


postuninst ()
{
  #remove fontconfig configuration
  sed -i "/${COMMENT}/d" /root/.profile

  #remove firewall config
  if [ "${SYNOPKG_PKG_STATUS}" == "UNINSTALL" ]; then
    /usr/syno/bin/servicetool --remove-configure-file --package ${SYNOPKG_PKGNAME}.sc > /dev/null
  fi

  #remove legacy daemon user and homedir
  [ -e /var/services/homes/serviio ] && synouser --del serviio
  [ -e /var/services/homes/serviio ] && rm -r /var/services/homes/serviio

  exit 0
}


preupgrade ()
{
  `dirname $0`/stop-start-status stop
  pre_checks
  #if a media database exists we need to preserve it
  if [ -d ${SYNOPKG_PKGDEST}/library/db ]; then
    mkdir ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_db_migration
    mv ${SYNOPKG_PKGDEST}/library/db ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_db_migration
  fi

  exit 0
}


postupgrade ()
{
  #use the backed up media database from the previous version
  if [ -d ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_db_migration/db ]; then
    mv ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_db_migration/db ${SYNOPKG_PKGDEST}/library
    rmdir ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_db_migration
  fi
  chown -R root:root ${SYNOPKG_PKGDEST}

  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------SERVIIO start-stop-status script
#--------package maintained at pcloadletter.co.uk

PKG_FOLDER="/var/packages/Serviio"
ENGINE_CFG="${PKG_FOLDER}/target/bin/serviio.sh"
ENGINE_SCRIPT="${PKG_FOLDER}/target/bin/serviio.sh"
PID_FILE="${PKG_FOLDER}/target/serviio.pid"
DNAME="Serviio"
DLOG="${PKG_FOLDER}/target/log/serviio.log"
COMMENT="# Synology Serviio Package"
TIMESTAMP="`date "+%F %X,000"`"
source /etc/profile
source /root/.profile


EnvCheck ()
#updates to DSM will reset these changes so check them each startup
{
  #/root/.profile should contain 2 lines added by this package tagged with trailing comments
  COUNT=`grep -c "$COMMENT$" /root/.profile`
  if [ $COUNT != 2 ]; then

    #remove any existing mods
    sed -i "/${COMMENT}/d" /root/.profile

    #add required environment variables
    echo "export FONTCONFIG_FILE=fonts.conf ${COMMENT}" >> /root/.profile
    echo "export FONTCONFIG_PATH=${PKG_FOLDER}/target/config/fonts ${COMMENT}" >> /root/.profile
  fi
}

start_daemon ()
{
  EnvCheck
  source /root/.profile

  #create/refresh fontconfig cache - prevents delay the first time that FFmpeg renders hard subs
  #FC_DEBUG=128 fc-cache --verbose
  ${PKG_FOLDER}/target/bin/fc-cache

  #refresh hostname in Serviio instance name - recently DSM has modified the hosts file and broken localhost name resolution
  sed -i -r "s%<FriendlyName>Serviio \((\{computerName\})\)%<FriendlyName>Serviio \(`hostname`\)%" $PKG_FOLDER/target/config/profiles.xml

  #set appropriate Java max heap size
  RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
  if [ $RAM -le 128 ]; then
    JAVA_MAX_HEAP=80
  elif [ $RAM -le 256 ]; then
    JAVA_MAX_HEAP=192
  elif [ $RAM -le 512 ]; then
    JAVA_MAX_HEAP=384
  #Serviio's default max heap is 512MB
  elif [ $RAM -gt 512 ]; then
    JAVA_MAX_HEAP=512
  fi
  sed -i -r "s/(^..JAVA.) -Xmx[0-9]+[mM] (.*$)/\1 -Xmx${JAVA_MAX_HEAP}m \2/" "${ENGINE_CFG}"
  echo "${TIMESTAMP} Starting ${DNAME}" >> ${DLOG}
  ${ENGINE_SCRIPT} > /dev/null 2>> ${DLOG}
  if [ -z ${SYNOPKG_PKGDEST} ]; then
    #script was manually invoked, need this to show status change in Package Center
    [ -e ${PKG_FOLDER}/enabled ] || touch ${PKG_FOLDER}/enabled
  fi
}

stop_daemon ()
{
  echo "${TIMESTAMP} Stopping ${DNAME}" >> ${DLOG}
  kill `cat ${PID_FILE}`
  wait_for_status 1 20 || kill -9 `cat ${PID_FILE}`
  rm -f ${PID_FILE}
  if [ -z ${SYNOPKG_PKGDEST} ]; then
    #script was manually invoked, need this to show status change in Package Center
    [ -e ${PKG_FOLDER}/enabled ] && rm ${PKG_FOLDER}/enabled
  fi
}

daemon_status ()
{
  if [ -f ${PID_FILE} ] && kill -0 `cat ${PID_FILE}` > /dev/null 2>&1; then
    return
  fi
  rm -f ${PID_FILE}
  return 1
}

wait_for_status ()
{
  counter=$2
  while [ ${counter} -gt 0 ]; do
    daemon_status
    [ $? -eq $1 ] && return
    let counter=counter-1
    sleep 1
  done
  return 1
}

case $1 in
  start)
    if daemon_status; then
      echo ${DNAME} is already running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo Starting ${DNAME} ...
      start_daemon
      exit $?
    fi
  ;;

  stop)
    if daemon_status; then
      echo Stopping ${DNAME} ...
      stop_daemon
      exit $?
    else
      echo ${DNAME} is not running
      exit 0
    fi
  ;;

  restart)
    stop_daemon
    start_daemon
    exit $?
  ;;

  status)
    if daemon_status; then
      echo ${DNAME} is running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo ${DNAME} is not running
      exit 1
    fi
  ;;

  log)
    echo "${DLOG}"
    exit 0
  ;;

  *)
    echo "Usage: $0 {start|stop|status|restart}" >&2
    exit 1
  ;;

esac
 

ffmpeg-wrapper-i686evansport.sh

#!/bin/sh

#FFmpeg wrapper script to enable hardware decoding and encoding on Intel Evansport CPUs
#wrapper originally posted here: http://forum.serviio.org/viewtopic.php?f=7&t=6458

FOLDER="`dirname $0`"
PARAMS=""
COMMANDLINE=$@
FFMPEG_BIN="ffmpeg"
INPUT=0
for PARAM in "$@"; do
  if [ ${INPUT} = 1 ]; then
    #the FFmpeg input filename/URL needs quotes adding back on
    #because it may contain spaces, and the shell has removed them
    PARAMS="${PARAMS} \"${PARAM}\""    
    INPUT=0
  elif [ ${INPUT} = 2 ]; then
    #the FFmpeg subtitles filename needs quotes adding back on
    #because it may contain spaces, and the shell has removed them
    PARAM=${PARAM/\[*.*\]subtitles=filename=/subtitles=filename=\"}
    PARAM=${PARAM/:original_size/\":original_size}
    PARAMS="${PARAMS} ${PARAM}"    
    INPUT=0
  elif [ "${PARAM}" == "-filter_complex" ]; then
    #next parameter will be subtitles filtergraph including subtitle filename
    #which needs special treatment, so set a flag
    INPUT=2
    PARAMS="${PARAMS} ${PARAM}"
  elif [ "${PARAM}" == "-i" ]; then
    #next parameter will be input filename which needs special treatment, so set a flag    
    INPUT=1
    #enable hardware decoding by default, but not when Serviio is either:
    # gathering media metadata,
    # remuxing video,
    # generating thumbnails,
    # or rendering subtitles into the video stream (pipeline mode not supported for subs),
    if [ $# -lt 3 ] || \
       [ "${COMMANDLINE}" != "${COMMANDLINE/ -c:v copy /}" ] || \
       [ "${COMMANDLINE}" != "${COMMANDLINE/ pipe:/}" ] || \
       [ "${COMMANDLINE}" != "${COMMANDLINE/ -filter_complex \[*:*\]subtitles=/}" ]; then
      PARAMS="${PARAMS} -i"
    # or gathering online stream metadata 
    elif [ $# -eq 4 ] && [ "${COMMANDLINE}" != "${COMMANDLINE/-analyzeduration /}" ]; then
      PARAMS="${PARAMS} -i"
    else
      PARAMS="${PARAMS} -prefer_smd -i"
    fi
  else
    PARAMS="${PARAMS} ${PARAM}"
  fi
done

#older Evansport FFmpeg version 2.7.1 needs "-strict -2" to use experimental aac encoder
if [ "${PARAMS}" != "${PARAMS/ -c:a:* aac /}" ]; then
  PARAMS="${PARAMS/ aac / aac -strict -2 }"
fi

#Is this FFmpeg commandline a candidate for the hardware encoder?
HWENC=0
if [ "${PARAMS}" != "${PARAMS/ -c:v libx264 /}" ]; then
  #check whether H.264 encoder is already busy with another Serviio/VideoStation session
  PID_HWENC=`cat /tmp/VideoStation/enabled 2> /dev/null | sed -r "s/.*\"PID\":([0-9]+),\"hardware_transcode.*$/\1/;s/\[//;s/\]//"`
  #is there an indicated PID?
  if [ -n "${PID_HWENC}" ]; then
    #there is an indicated PID - is it running? 
    if ! kill -0 ${PID_HWENC}; then
      #the indicated PID is not in fact running, HW encoder is therefore available
      rm /tmp/VideoStation/enabled
      #make hardware H.264 encoder substitution to replace libx264
      HWENC=1
    fi 
  else
    #there is no indicated PID, HW encoder is therefore available
    HWENC=1
  fi  
fi

if [ ${HWENC} = 1 ]; then
  #make hardware H.264 encoder substitution to replace libx264
  PARAMS=${PARAMS/ libx264 / h264_smd }

  #by default Serviio's libx264 transcoding is intended for speed over quality, with SMD we can opt for better quality
  PARAMS="`echo ${PARAMS} | sed -r "s/ -crf [0-9]+ / /; s/ -g [0-9]+ / /"`"
  PARAMS=${PARAMS/ -preset:v veryfast / }
  PARAMS=${PARAMS/ -profile:v baseline / -profile:v high }
  PARAMS=${PARAMS/ -level 3 / -level 4.1 }
fi

#invoke FFmpeg
#log commandline, except media probes
#[ $# -gt 2 ] && echo "${FOLDER}/${FFMPEG_BIN} ${PARAMS}" >> ${FOLDER}/../log/ffmpeg-wrapper.log

#need to use eval here otherwise the quotes aren't handled properly
#http://fvue.nl/wiki/Bash:_Why_use_eval_with_variable_expansion%3F
eval ${FOLDER}/${FFMPEG_BIN} ${PARAMS}

#return FFmpeg status
exit $?
 

Changelog:

  • 0037 05/Jun/19 – Updated to Serviio 2.0, clear your web browser cache to fix Serviio Console issues
  • 0036 17/Apr/19 – Updated to Serviio 1.10.1, fixed missing Serviio console icon in DSM 6.2 main menu, added support for DS119j
  • 0035 14/May/18 – Updated to Serviio 1.9.2
  • 0034 01/Mar/18 – Updated to Serviio 1.9.1, added ARMv8 support
  • 0033 02/Aug/17 – Updated to Serviio 1.9
  • 0032 21/Jan/17 – Fixed Evansport FFmpeg wrapper script to allow transcoding to aac since it’s an older FFmpeg version with slightly different command syntax
  • 0031 07/Jan/17 – Updated to Serviio 1.8
    Updated FFmpeg to 3.2.2
    Updated Intel Evansport hardware transcoding build of FFmpeg from 2.0.2 to 2.7.1 – now detects H.265 codec correctly
    Fixed Intel Evansport FFmpeg support for transcoding DTS audio
    FFmpeg now uses hard float ABI on systems with Armada 370/XP SoC (DSM 6.0 change)
    FFmpeg now uses NEON on systems with Armada 375 SoC (DSM 6.0 change)
    Disabled FFmpeg wrapper script to enable libshine for ARMv5 systems now that Serviio handles this automatically
    Added Ogg Vorbis audio support to FFmpeg
    Updated Deja-Vu fonts to 2.37
    DSM 6.0 is now required due to greatly simplify FFmpeg compilation
  • 0030 21/May/16 – Updated to Serviio 1.6.1
  • 0029 28/Jan/16 – Updated to Serviio 1.6, integrated the new built-in web console into DSM, fixed stupid wrapper bug which had broken evansport hardware transcoding on DS214play and DS415play, fixed online content when using evansport build. You must have already upgraded to version 1.5.2-0028 or else Package Center will report a TCP port conflict and will fail to upgrade the package. If you missed the 0028 update, install it manually in Package Center using the .spk file: (DS214play/DS415play), (all other models).
  • 0028 08/Jan/16 – Serviio instance uses hostname rather than localhost, more accurate detection of DSM 6.0 on Intel CPUs which need x64 FFmpeg build, added DS216play support, fixed missing libgmp on DS3612xs running DSM 5.0, change to allow easy package upgrade to forthcoming Serviio 1.6
  • 0027 12/Nov/15 – Substantial package re-write hence the long delay since the last release
    Completely refreshed documentation, please re-read
    Updated to Serviio version 1.5.2
    Java 8 is now required – this is a Serviio imposed requirement
    DSM 5.0 or newer is now required
    Updated to FFmpeg 2.7.1 and many libs e.g. x264 at latest versions from late July 2015
    Added optional hardware transcoding support for Intel Evansport systems (DS214play, DS415play) built using Synology-modified sources for FFmpeg 2.0.2
    FFmpeg is now compiled with https support via GNUTLS library (useful for online streams via plugins)
    Switched to using root account which was necessary for Evansport hardware encoding – no more adding account permissions, package upgrades will no longer break this
    Added support for Mindspeed Comcerto 2000 CPU (comcerto2k – ARM Cortex-A9 with NEON) in DS414j
    Added support for Intel Atom C2538 (avoton) CPU in DS415+
    Fontconfig fc-cache utility now included to allow troubleshooting issues with adding new fonts for additional subtitle language support
    Fontconfig variables checked on each package startup so DSM upgrades no longer break hard subs support
    Updated DejaVu subtitle font to version 2.35
    DSM Firewall application definitions added
    Tested with DSM Task Scheduler to allow package to start/stop at certain times of day, saving RAM when not needed
    Daemon init script now uses a proper PID file instead of the unreliable method of using grep on the output of ps
    Daemon init script can be run from the command line
    Improved accuracy of temp folder and plugins folder detection
    Switched to .tar.xz compression for native binaries to reduce web hosting storage footprint
    Package is now signed with repository private key
    User Agent customization while downloading Serviio binaries from serviio.org to allow download stats gathering
  • 0026 25/Mar/14 – Updated to Serviio 1.4.1.2
  • 0025 18/Mar/14 – Updated to Serviio 1.4.1.1
  • 0024 30/Jan/14 – Updated to Serviio 1.4
  • 0023 07/Nov/13 – Updated to FFmpeg 2.1 stable release
  • 0022 30/Oct/13 – Added support for Intel Atom Evansport and Armada XP CPUs in new DSx14 products
  • 0021 18/Oct/13 – Fixed zlib linking issue in QorIQ and Intel builds of FFmpeg
  • 0020 05/Sep/13 – Updated to Serviio 1.3.1
  • 0019 25/May/13 – Added support for Armada370 SoC used in DS213j (ARMv7 CPU with FPU)
  • 0018 29/Mar/13 – Updated to Serviio 1.2.1, recompiled FFmpeg to use the libRTMP source code from serviio.org for consistency
  • 0017 01/Mar/13 – Updated to Serviio 1.2
  • 0016 23/Jan/13 – Fixes for DSM 4.2
  • 015 30/Dec/13 – Updated to Serviio 1.1, fixed garbled ac3 encoding issue on ARM CPUs by compiling FFmpeg natively, rather than cross compiling. Remember to update your computer’s Serviio Console to 1.1 before connecting to manage.
  • 014 28/Oct/12 – Added support for Freescale QorIQ PowerPC CPUs used in some Synology x13 series products, switched to shared library compile of FFmpeg with some minor changes, ARM build now uses libshine encoder which allows realtime audio transcoding to MP3, edited FlowPlayer config to play AAC audio natively to make MediaBrowser more usable with online feed items.
  • 013 Fixed a bug in the plugins folder creation, fixed deletion of home directories after user deletion (DSM 4.1)
  • 012 Reduced Java prefs checking interval to once every 24 hours to allow the NAS to hibernate (was 30 seconds by default – remember to increase the time between library refreshes if you want hibernation)
  • 011 Updated Serviio to 1.0.1, slightly altered how the plugins folder path is determined
  • 010 Updated to Serviio 1.0, removed WebUI, FFmpeg wrapper no longer needed on ARM, plugins folder moved to /volume1/public/serviio/plugins to make adding/updating plugins easier
  • 009 Installation fails unless User Home service is enabled, unified the installer scripts, merged ARM and Intel packages into one which downloads the FFmpeg binary separately, used integer maths Dolby AC-3 encoder on ARM systems (no floating point)
  • 008 Fixed DST timezone support, installer no longer assumes /volume1 is primary storage volume
  • 007 Updated Serviio to 0.6.2, kairoh’s WebUI to 0.6.2c, changed package to download Serviio from the official website during installation, temp path is set by default to /volume1/@tmp on first run, and finally some CSS improvements
  • 006 Updated to Serviio 0.6.1, switched to kairoh’s Java WebUI, added timezone support, removed DSM icon when Serviio is not running, adjusted Java max heap size for systems with low RAM, specified FFmpeg path directly rather than creating a symlink in /bin so as not to interfere with other packages which may use different versions of FFmpeg
  • 005 Added Web Station dependency, EULA dialog, and links for Web UI and user forum in More Info
  • 004 Test for package repo to allow update notification
  • 0.6.0.1 v3 Hopefully fixed an issue with the Serviio DSM icon in DSM 3.2 on NAS units without Optware installed
  • 0.6.0.1 v2 Inclusion of a modified version of the PHP Web UI
  • 0.6.0.1 v1 New minor Serviio release 0.6.0.1 with some updated international translations – will upgrade 0.6 preserving the media library
  • 0.6 v1 23/Sep/11 – First public release – Serviio 0.6
  • 0.6b4 v2 Fixed permissions on transcode temporary folder following upgrade, avoiding the need for a reboot
  • 0.6b4 v1 Updated to Serviio 0.6 beta 4, added upgrade scripts to allow media database migration (install future packages over the top of this one – database is preserved)
  • 0.6b3c v2 Fixed HOME env var for serviio user (to fix librtmp issues with BBC iPlayer) and removed the need to edit the passwd file to change the shell (safer)
  • 0.6b3c v1 Initial spk test release – Serviio 0.6 beta 3

 
 

Minecraft package for Synology NAS

UPDATE – The instructions and notes on this page apply to both the Minecraft and the CraftBukkit packages hosted on my repo. Now works on QorIQ CPU Synology models!

Minecraft is an intriguing game. I only recently bought it after a friend had recommended it. There’s a lot to like: the striking retro visual style, the mellow sporadic music, exploring the procedurally generated world, the logical way items are crafted, multiplayer collaboration, and emergent gameplay. Notch its creator certainly deserves the fortune he’s now sitting on. I thoroughly recommend you watch this short documentary all about it.

Minecraft Package Info

Right after I released the Java package for Synology a couple of people mentioned that they wanted to use it for installing a Minecraft server. Now that I have the game, I decided to try that too. I had assumed that the ARM CPUs would not be capable of running it, and the first tests seemed to confirm this. I decided it would be worth creating a package nonetheless since the Intel CPU NAS units would be ok, especially with their extra RAM. However, with a little performance tuning it actually runs acceptably (no lag when mining blocks) on my ARM powered DS111 which only has 256MB. The CPU load is 100% for a while at first, but soon after playing it settles down to around 60%. I have briefly tested with two players connected and after a bit of lag at first (one second delay to mine a block) it does seem to settle down.

Minecraft running showing draw distance and server load

Minecraft showing draw distance and server load running on a Synology DS111

 

Installation

  • This package is not CPU specific. If you have Java it will run. At the moment that means ARM, Intel and QorIQ PowerPC processors, but not the older PowerPCs.
  • In the User Control Panel in DSM, enable the User Homes service.
  • Install the package directly from Package Center in DSM. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk. You will need to install either one of my Java SE for Embedded packages first (Java 6 or 7).

Notes

  • The package fetches the minecraft server jar file from Mojang as it is installed. I am complying with their wish that no one redistributes it.
  • The server daemon script checks the amount of system RAM and scales the Java heap size appropriately. It also applies a few performance tweaks to Java to try to reduce garbage collection latency.
  • The first time you run the server it will create a new world, which can take a few minutes. You cannot interrupt this but you can check on progress by repeatedly viewing the Log tab.
  • Because the time investments in playing Minecraft can be so considerable, when you uninstall the package it will back up the world folder and settings to /volume1/public/minecraftworld.todaysdate.bak to prevent accidental deletion.
  • The package supports upgrades to future versions while preserving the world folder and server settings.
  • If you want to transplant an existing world folder into the server, copy it to /volume1/@appstore/Minecraft. You will also need to run chown -R minecraft /volume1/@appstore/Minecraft to grant ownership of the files to the daemon user.
  • The first time you run the package, the server config file /volume1/@appstore/Minecraft/server.properties is generated. The next time it is launched, my script reduces the default draw distance for ARM CPUs from 10 chunks to 7. This was appropriate for my DS111 to prevent latency when mining blocks, but you may wish to reduce this further on the J series NAS units which have less RAM and slower CPUs. Other server files (white-list.txt etc.) are found in the same folder. For CraftBukkit the files are in the folder /volume1/@appstore/Craftbukkit/server.properties (note the capitalization – the CraftBukkit project seemed to change this after I had already created the package with a lower case ‘b’).
  • The simplest way to edit these config files if you’re not really confident with Linux is to install Merty’s Config File Editor package, which requires the official Synology Perl package to be installed too (since DSM 4.2). Load Config File Editor, then in the dropdown menu edit Config File Editor’s own config (it’s the last in the list) and add the lines:
    /volume1/@appstore/Minecraft/server.properties,Minecraft-properties
    /volume1/@appstore/Minecraft/white-list.txt,Minecraft-whitelist
    /volume1/@appstore/Minecraft/ops.txt,Minecraft-ops

    Make sure to add an extra blank line underneath, save, then relaunch CFE and you’ll have entries for Minecraft in the dropdown. You’ll need to restart the Minecraft package for any changes to take effect.
  • It was a bit tricky to get the server to shut down gracefully without just killing the Java process. We need it to shutdown properly so it saves the active chunks to disk first. Most of the guides on the Net use the screen binary which isn’t included with Synology DSM, and I didn’t want to have to make a version of the package for each CPU architecture. I found that I could use tail to send the last line of the file /tmp/stdin.minecraft to the server (/tmp/stdin.craftbukkit for the CraftBukkit package). This is how the stop command is issued. You could send your own commands, for instance echo say Hello players >> /tmp/stdin.minecraft. You can verify that the command was received by looking at the server log in Package Center.
  • The server runs on the default TCP port for Minecraft (25565) so you will need to port forward this on your router if you want it to be publicly accessible.
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology Package wiki.

installer.sh

#!/bin/sh

#--------MINECRAFT/CRAFTBUKKIT installer script
#--------package maintained at pcloadletter.co.uk

if [ "${SYNOPKG_PKGNAME}" == "Minecraft" ]; then
  DOWNLOAD_PATH="http://s3.amazonaws.com/MinecraftDownload/launcher"
  DOWNLOAD_FILE="minecraft_server.jar"
  UPGRADE_FILES="server.properties *.txt world"
fi
if [ "${SYNOPKG_PKGNAME}" == "Craftbukkit" ]; then
  DOWNLOAD_PATH="http://cbukk.it"
  DOWNLOAD_FILE="craftbukkit-beta.jar"
  UPGRADE_FILES="server.properties *.txt *.yml world world_nether world_the_end plugins bukkit_update"
fi

DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
DAEMON_PASS="`openssl rand 12 -base64 2>/dev/null`"
MIGRATION_FOLDER="${DAEMON_USER}_data_mig"
ENGINE_SCRIPT="/var/packages/${SYNOPKG_PKGNAME}/scripts/launcher.sh"
INSTALL_FILES="${DOWNLOAD_URL}"
source /etc/profile
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
PRIMARY_VOLUME="/`echo $TEMP_FOLDER | cut -f2 -d'/'`"
WORLD_BACKUP="${PRIMARY_VOLUME}/public/${DAEMON_USER}world.`date +\"%d-%b\"`.bak"

preinst ()
{
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  #is the User Home service enabled?
  UH_SERVICE=maybe
  synouser --add userhometest Testing123 "User Home test user" 0 "" ""
  UHT_HOMEDIR=`cat /etc/passwd | sed -r '/User Home test user/!d;s/^.*:User Home test user:(.*):.*$/\1/'`
  if echo $UHT_HOMEDIR | grep '/var/services/homes/' > /dev/null; then
    if [ ! -d $UHT_HOMEDIR ]; then
      UH_SERVICE=false
    fi
  fi
  synouser --del userhometest
  #remove home directory (needed since DSM 4.1)
  [ -e /var/services/homes/userhometest ] && rm -r /var/services/homes/userhometest
  if [ ${UH_SERVICE} == "false" ]; then
    echo "The User Home service is not enabled. Please enable this feature in the User control panel in DSM."
    exit 1
  fi

  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, "
        echo "which was \"${WGET_URL}\" "
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. "
        exit 1
      fi
    fi
  done
  
  exit 0
}


postinst ()
{
  #create daemon user
  synouser --add ${DAEMON_USER} ${DAEMON_PASS} "${DAEMON_ID}" 0 "" ""
  
  mv ${TEMP_FOLDER}/${DAEMON_USER}*.jar ${SYNOPKG_PKGDEST}/${DAEMON_USER}.jar
  
  #determine the daemon user homedir and save that variable in the user's profile
  #this is needed because new users seem to inherit a HOME value of /root which they have no permissions for
  DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
  su - ${DAEMON_USER} -s /bin/sh -c "echo export HOME=\'${DAEMON_HOME}\' >> .profile"
  
  #change owner of folder tree
  chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
  
  exit 0
}


preuninst ()
{
  #make sure server is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} stop ${SYNOPKG_PKGNAME} ${SYNOPKG_PKGDEST}"
  sleep 10
  
  #if a world exists, back it up to the public folder, just in case...
  if [ -d ${SYNOPKG_PKGDEST}/world ]; then
    if [ ! -d ${WORLD_BACKUP} ]; then
      mkdir -p ${WORLD_BACKUP}
    fi
    for ITEM in ${UPGRADE_FILES}; do
      mv ${SYNOPKG_PKGDEST}/${ITEM} ${WORLD_BACKUP}
    done
  fi
  
  exit 0
}


postuninst ()
{
  #remove daemon user
  synouser --del ${DAEMON_USER}
  
  #remove daemon user's home directory (needed since DSM 4.1)
  [ -e /var/services/homes/${DAEMON_USER} ] && rm -r /var/services/homes/${DAEMON_USER}
  
  exit 0
}


preupgrade ()
{
  #make sure the server is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} stop ${SYNOPKG_PKGNAME} ${SYNOPKG_PKGDEST}"
  sleep 10
  
  #if a world exists, back it up
  if [ -d ${SYNOPKG_PKGDEST}/world ]; then
    mkdir ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}
    for ITEM in ${UPGRADE_FILES}; do
      if [ -e ${SYNOPKG_PKGDEST}/${ITEM} ]; then
        mv ${SYNOPKG_PKGDEST}/${ITEM} ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}
      fi
    done
  fi
  
  exit 0
}


postupgrade ()
{
  #use the migrated data files from the previous version
  if [ -d ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}/world ]; then
    for ITEM in ${UPGRADE_FILES}; do
      if [ -e ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}/${ITEM} ]; then
        mv ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}/${ITEM} ${SYNOPKG_PKGDEST}
      fi
    done
    rmdir ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}
    
    #daemon user has been deleted and recreated so we need to reset ownership (new UID)
    chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
  fi
  	
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------MINECRAFT/CRAFTBUKKIT start-stop-status script
#--------package maintained at pcloadletter.co.uk

DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
ENGINE_SCRIPT="/var/packages/${SYNOPKG_PKGNAME}/scripts/launcher.sh"
DAEMON_USER_SHORT=`echo ${DAEMON_USER} | cut -c 1-8`

daemon_status ()
{
    ps | grep "^ *[0-9]* ${DAEMON_USER_SHORT} .*java" > /dev/null
}

case $1 in
  start)
    DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
    
    #set the current timezone for Java so that log timestamps are accurate
    #we need to use the modern timezone names so that Java can figure out DST
    SYNO_TZ=`cat /etc/synoinfo.conf | grep timezone | cut -f2 -d'"'`
    SYNO_TZ=`grep "^${SYNO_TZ}" /usr/share/zoneinfo/Timezone/tzname | sed -e "s/^.*= //"`
    grep "^export TZ" ${DAEMON_HOME}/.profile > /dev/null \
     && sed -i "s%^export TZ=.*$%export TZ='${SYNO_TZ}'%" ${DAEMON_HOME}/.profile \
     || echo export TZ=\'${SYNO_TZ}\' >> ${DAEMON_HOME}/.profile
    
    su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} start ${DAEMON_USER} ${SYNOPKG_PKGDEST} &"
    exit 0
  ;;
  
  stop)
    su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} stop ${DAEMON_USER} ${SYNOPKG_PKGDEST}"
    exit 0
  ;;
  
  status)
    if daemon_status ; then
      exit 0
    else
      exit 1
    fi
  ;;
  
  log)
    echo "${SYNOPKG_PKGDEST}/server.log"
    exit 0
  ;;
esac
 

launcher.sh

#!/bin/sh

#--------MINECRAFT/CRAFTBUKKIT server launcher script
#--------package maintained at pcloadletter.co.uk
 
#--------Allows graceful shutdown of server without CPU-specific binaries
#--------You can send commands to the running server like so:
#--------    echo say Hello players >> /tmp/stdin.minecraft
#--------    echo say Hello players >> /tmp/stdin.craftbukkit

DAEMON_USER=$2
SYNOPKG_PKGDEST=$3
DAEMON_USER_SHORT=`echo ${DAEMON_USER} | cut -c 1-8`
JAR_FILE=${SYNOPKG_PKGDEST}/$2.jar

case $1 in
  start)
    if [ -f /tmp/stdin.${DAEMON_USER} ]; then
      rm /tmp/stdin.${DAEMON_USER}
    fi
    touch /tmp/stdin.${DAEMON_USER}
    cd ${SYNOPKG_PKGDEST}
    if [ ! -f syno-marker.txt ]; then
      if [ -f server.properties ]; then
        sed -i "s/A Minecraft Server/A Synology Minecraft Server/" server.properties
  
        #ARM CPU lags a lot, so reduce drawing distance from 10 chunks to 6
        cat /proc/cpuinfo | grep "CPU architecture: 5TE" > /dev/null \
         && sed -i "s/^view-distance=10/view-distance=6/" server.properties
  
        #record that these mods have been made
        echo config updated > syno-marker.txt
      fi
    fi
    JAVA_OPTS='-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalPacing -XX:+AggressiveOpts'
    RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
    if [ $RAM -le 128 ]; then
      JAVA_MAX_HEAP=80M
    elif [ $RAM -le 256 ]; then
      JAVA_MAX_HEAP=192M			
    elif [ $RAM -le 512 ]; then
      JAVA_MAX_HEAP=448M
    elif [ $RAM -le 1024 ]; then
      JAVA_MAX_HEAP=896M
    elif [ $RAM -le 2048 ]; then
      JAVA_MAX_HEAP=1792M
    elif [ $RAM -gt 2048 ]; then
      JAVA_MAX_HEAP=2048M
    fi
    JAVA_START_HEAP=${JAVA_MAX_HEAP}
    tail -n 0 -f /tmp/stdin.${DAEMON_USER} | java -Xmx${JAVA_START_HEAP} -Xms${JAVA_MAX_HEAP} ${JAVA_OPTS} -jar ${JAR_FILE} nogui
  ;;

  stop)
    echo say shutting down.. >> /tmp/stdin.${DAEMON_USER}
    sleep 5
    echo stop >> /tmp/stdin.${DAEMON_USER}
    sleep 10
    kill -9 `ps | grep "^ *[0-9]* ${DAEMON_USER_SHORT}.*tail -n 0 -f /tmp/stdin.${DAEMON_USER}" | sed -e "s/^ *\([0-9]*\).*$/\1/"`
    if [ -f /tmp/stdin.${DAEMON_USER} ]; then
      rm /tmp/stdin.${DAEMON_USER}
    fi
  ;;
esac
 

Changelog:

  • 0015 updated to Minecraft 1.5.2, CraftBukkit beta 1.5.2-R0.1
  • 0014 updated to Minecraft 1.4.7, CraftBukkit beta 1.4.7-R0.1, and fixes for DSM 4.2
  • 013 updated to Minecraft 1.4.6, and CraftBukkit beta 1.4.6-R0.3
  • 012 updated to Minecraft 1.4.5, and CraftBukkit 1.3.2-R1.0
  • 011 updated to Minecraft 1.4.2
  • 010 updated to Minecraft 1.3.2, and CraftBukkit 1.3.1-R2.0
  • 009 package scripts fully re-written to unify the Minecraft and CraftBukkit packages
  • 008 updated to Minecraft 1.3.1, unified most scripts into a single installer script, and incorporated minor enhancements from my other packages
  • 007 updated to Minecraft 1.2.3
  • 006 reduced Java max heap to 80MB on 128MB systems, fixed timezone support each server start
  • 005 fixed Java max heap size behaviour on systems with more than 2GB RAM
  • 004 fixed wget SSL problem preventing jar download on some systems
  • 003 updated to Minecraft 1.1
  • 002 server config files are also migrated during version upgrades, and backed up during uninstall. My script’s edits to server.properties are made only once, rather than every startup
  • 001 intial public release