Category Archives: Linux

Cross-compiling nano with UTF-8 support for Synology

export CROSS_PREFIX=i686-pc-linux-gnu
export TARGET=i686-pc-linux-gnu
export TOOLCHAIN=/usr/local/evansport-pc-linux-gnu
export AR=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ar
export AS=${TOOLCHAIN}/bin/${CROSS_PREFIX}-as
export CC=${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc
export CXX=${TOOLCHAIN}/bin/${CROSS_PREFIX}-g++
export LD=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ld
export LDSHARED="${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc -shared "
export RANLIB=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ranlib
export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"
export LDFLAGS="-L${TOOLCHAIN}/lib"
export PKG_CONFIG_PATH="${TOOLCHAIN}/lib/pkgconfig"

#DSM 5.0 already includes libncursesw so we can link the nano binary to this
#fetch the source which was extracted from the Synology DSM 5.0 source tarball
wget https://dl.dropboxusercontent.com/u/1188556/ncurses-5.x.zip
unzip ncurses-5.x.zip
cd ncurses-5.x
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --with-shared --without-normal --without-progs --without-debug --enable-widec
make
make install
cd ..
wget http://www.nano-editor.org/dist/v2.2/nano-2.2.6.tar.gz
tar xvzf nano-2.2.6.tar.gz
cd nano-2.2.6
#src/nano.h:92:20: fatal error: curses.h: No such file or directory
#http://blog.csdn.net/alienspy/article/details/24135761
wget http://dl.dropboxusercontent.com/u/1188556/nano.h.patch
patch nano.h < nano.h.patch
CFLAGS="${CFLAGS} -I${TOOLCHAIN}/include/ncurses" ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-utf8 --disable-nls --enable-color --enable-extra --enable-multibuffer --enable-nanorc
#sudo apt-get install texinfo
sed -i -e "s%^CPPFLAGS = -I/usr/include/ncursesw%CPPFLAGS = -I${TOOLCHAIN}/include/ncurses%" Makefile
sed -i -e "s%^CPPFLAGS = -I/usr/include/ncursesw%CPPFLAGS = -I${TOOLCHAIN}/include/ncurses%" src/Makefile
make
make install
#now install it to /usr/local/bin
 

nano.h.patch

--- src/nano.h.old      2014-09-17 12:16:15.006206100 +0100
+++ src/nano.h  2014-09-17 12:19:01.226206100 +0100
@@ -86,10 +86,10 @@
 #define KEY_IC SL_KEY_IC
 /* Ncurses support. */
 #elif defined(HAVE_NCURSES_H)
-#include <ncurses.h>
+#include <ncursesw/ncurses.h>
 #else
 /* Curses support. */
-#include <curses.h>
+#include <ncursesw/curses.h>
 #endif /* CURSES_H */

 #ifdef ENABLE_NLS
 

~/.nanorc

##############################################################################
#
# Syntax highlighting for XML files
#
# Author:  Josef 'Jupp' Schugt, jupp(a)rubyforge.org
# License: GPL 2  or later
#
# Version: 2004-02-25
#
##############################################################################
syntax "ml" ".*\.([jrs]?html?|xml|sgml?)$"
color white "^.+$"
color green  start="<" end=">"
color cyan   "<[^> ]+"
color cyan   ">"
color yellow start="<!DOCTYPE" end="[/]?>"
color yellow start="<!--" end="-->"
color red    "&[^;]*;"

##############################################################################
#
# Syntax highlighting for HTTP codes
#
# Author:  Josef 'Jupp' Schugt, jupp(a)rubyforge.org
# License: GPL 2  or later
#
# Version: 2004-02-25
#
##############################################################################
syntax "urls"
color brightmagenta   "^.*$"
color cyan      "^(1[0-9][0-9]|20[256]|30[45]).*$"
color green      "^20[03].*$"
color brightyellow   "^(201|30[0-37]).*$"
color brightred      "^(204|[45][0-9][0-9]|666).*$"

##############################################################################
#
# Syntax highlighting for CSS files
#
# Author:  Simon Rupf, simon.rupf(a)int-ag.ch
# License: GPL 2  or later
#
# Version: 2005-02-14
#
##############################################################################
syntax "css" "\.css$"
color brightred     "."
color brightyellow  start="\{" end="\}"
color brightwhite           start=":" end="[;^\{]"
color brightblue    ":active|:focus|:hover|:link|:visited|:link|:after|:before|$"
color brightblue    start="\/\*" end="\\*/"
color green         ";|:|\{|\}"

syntax "sh" "\.sh$"
#icolor brightgreen "^[0-9A-Z_]+\(\)"
#color green "\b(case|do|done|elif|else|esac|exit|fi|for|function|if|in|local|read|return|select|shift|then|time|until|while)\b"
#color green "(\{|\}|\(|\)|\;|\]|\[|`|\\|\$|<|>|!|=|&|\|)"
#color green "-[Ldefgruwx]\b"
#color green "-(eq|ne|gt|lt|ge|le|s|n|z)\b"
#color brightblue "\b(cat|cd|chmod|chown|cp|echo|env|export|grep|install|let|ln|make|mkdir|mv|rm|sed|set|tar|touch|umask|unset)\b"
icolor brightwhite "\$\{?[0-9A-Z_!@#$*?-]+\}?"
color brightcyan "(^|[[:space:]])#.*$"
#color brightyellow ""(\\.|[^"])*"" "'(\\.|[^'])*'"
#color cyan ""(\\.|[^"])*"" "'(\\.|[^'])*'"
#color ,green "[[:space:]]+$"

Cross-compiling FFmpeg for Serviio with shared libraries on Synology NAS (Intel, ARMv5, ARMv7, and QorIQ CPUs)

Update 3 – Updated the guide for building FFmpeg 2.1, with libx264 which is required for Serviio 1.3. The patch for the libshine encoder is no longer needed now that it has been merged into FFmpeg. I’m proud to have suggested and tested that feature and contributed in some small way to FFmpeg!

Update 2 – Updated to allow build for ARMv7 CPU in Synology products using Armada370 SoC (currently the DS213j).

Update – I discovered that the cross-compiled ARMv5 build of FFmpeg cannot encode AC3 audio without severe artifacts. After a very long time spent recompiling with different options, trying static builds etc. I finally found that the Synology cross toolchain (GCC 4.2.1) appears to cause the problem. Compiling natively on a bootstrapped ARMv5 Synology system with GCC 4.2.3 produces a working binary. However, I had to spend a long time re-working the method, particularly since you can’t run automake to build libshine on the Synology due to a dependency on a threaded version of Perl, which is missing from the Optware repo, and in turn is very tricky to compile. I have appended the ARM shared build guide to the end of this post.

This method will build FFmpeg with portable shared libraries (linked with relative paths) using a Ubuntu Desktop 12 VM. The >10MB size of the static executable is kind of getting out of hand especially on embedded systems with very limited RAM. Using shared libs means many concurrent instances of FFmpeg can use broadly the same memory footprint, and package distribution binaries can be smaller – our target systems already have libmp3lame, libz, libssl, libcrypto, libfreetype, libexpat, and libpthread which add up to several megabytes. I’m also guessing that the OS will decide when to unload them from RAM which could help when FFmpeg is being launched repeatedly in a short period of time.

Although the unmodified source code will compile successfully for QorIQ CPUs, the FFmpeg binary will core dump when running any command on a video file. I contacted Synology Support for help with this issue. Their developers mentioned that they had needed to make numerous patches to the source of the version they used for DSM, and they guessed that the following one in particular was likely to be needed. This produced a working binary.

--- ffmpeg/libavcodec/ppc/dsputil_ppc.c	2012-02-17 18:20:07.000000000 +0000
+++ ffmpeg-patched/libavcodec/ppc/dsputil_ppc.c	2012-10-18 22:15:50.740992688 +0100
@@ -116,7 +116,11 @@
 
     /* below the constraint "b" seems to mean "Address base register"
        in gcc-3.3 / RS/6000 speaks. seems to avoid using r0, so.... */
+    #if 1
+    __asm__ volatile("dcbz %0, %1" : : "r" (fakedata_middle), "r" (zero));
+    #else
     __asm__ volatile("dcbzl %0, %1" : : "b" (fakedata_middle), "r" (zero));
+    #endif
 
     for (i = 0; i < 1024 ; i ++) {
         if (fakedata[i] == (char)0)
 
#-----set up Synology toolchain
cd ~/Downloads
export DL_PATH="http://sourceforge.net/projects/dsgpl/files/DSM%204.1%20Tool%20Chains"

#----------------------------------------------------
#-----NOW PASTE ONE OF THE FOLLOWING FOUR BLOCKS TO THE TERMINAL, DEPENDING ON YOUR TARGET CPU TYPE




#-----Marvell Kirkwood mv6281/mv6282 CPU is based on the ARMv5TE core which has DSP and thumb instruction support
#-----http://www.arm.com/products/processors/technologies/dsp-simd.php
#-----http://www.marvell.com/embedded-processors/kirkwood/assets/88f6282-3_pb.pdf
wget "${DL_PATH}/Marvell%2088F628x%20Linux%202.6.32/gcc421_glibc25_88f6281-GPL.tgz"
tar xvzf gcc421_glibc25_88f6281-GPL.tgz
export CROSS_PREFIX=arm-none-linux-gnueabi
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
export TARGET=armle-unknown-linux
export MARCH="-march=armv5te"
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
export SSL_CONFIG="./Configure.syno --prefix=${TOOLCHAIN} threads shared linux-elf-armle"
#-----libx264:
#-----http://forum.doom9.org/showthread.php?t=160584
#-----https://mailman.videolan.org/pipermail/x264-devel/2010-December/008093.html
#-----x264 builds with NEON by default because x264 is so slow without NEON (and on any non-NEON chip) as to be useless.
#-----You can of course compile with --disable-asm on such chips, but we don't do it by default because
#-----we don't feel it's necessary to actively support chips on which x264 would be basically useless
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --enable-shared --disable-opencl --disable-asm --enable-strip"
#-----FFmpeg:
#-----ARMv5TE CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
#-----zbmv decoder won't cross compile for some reason so it is disabled: https://ffmpeg.org/trac/ffmpeg/ticket/1794
#-----something changed in the FFmpeg configure script and now vfp and neon optimizations have to be explicitly disabled for ARMv5TE, which doesn't have those features in any case
export FF_CONFIG="./configure --arch=arm --cpu=armv5te --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-vfp --disable-neon --disable-debug --enable-pthreads --enable-libshine --enable-librtmp --enable-libass --enable-libx264 --enable-gpl --disable-encoder=zmbv --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"
#-----sudo apt-get automake
#-----sudo apt-get libtool
wget https://github.com/savonet/shine/zipball/master
unzip master
cd savonet-shine-*
./bootstrap
cd ..




#-----Marvell Armada 370 CPU is based on a dual issue ARMv7 core with VFPv3-16, but no NEON vector unit
#-----http://www.marvell.com/embedded-processors/armada-300/assets/Marvell_ARMADA_370_SoC.pdf
#-----http://www.arm.com/products/processors/technologies/vector-floating-point.php
wget "${DL_PATH}/Marvell%20armada%20370%20Linux%203.2.30/gcc464_glibc215_hard_armada370-GPL.tgz"
tar xvzf gcc464_glibc215_hard_armada370-GPL.tgz
#-----correct some missing header file links in the toolchain 
cp /usr/local/arm-marvell-linux-gnueabi/arm-marvell-linux-gnueabi/libc/thumb2/usr/include/asm/errno.h /usr/local/arm-marvell-linux-gnueabi/arm-marvell-linux-gnueabi/libc/usr/include/asm/errno.h
cp /usr/local/arm-marvell-linux-gnueabi/arm-marvell-linux-gnueabi/libc/thumb2/usr/include/asm/ioctl.h /usr/local/arm-marvell-linux-gnueabi/arm-marvell-linux-gnueabi/libc/usr/include/asm/ioctl.h
cp /usr/local/arm-marvell-linux-gnueabi/arm-marvell-linux-gnueabi/libc/thumb2/usr/include/asm/bitsperlong.h /usr/local/arm-marvell-linux-gnueabi/arm-marvell-linux-gnueabi/libc/usr/include/asm/bitsperlong.h
export CROSS_PREFIX=arm-marvell-linux-gnueabi
export TARGET=armle-unknown-linux
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
export MARCH="-mhard-float -mfpu=vfpv3-d16"
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
export SSL_CONFIG="./Configure.syno --prefix=${TOOLCHAIN} threads shared no-asm linux-elf-armle"
#-----libx264:
#-----http://forum.doom9.org/showthread.php?t=160584
#-----https://mailman.videolan.org/pipermail/x264-devel/2010-December/008093.html
#-----x264 builds with NEON by default because x264 is so slow without NEON (and on any non-NEON chip) as to be useless.
#-----You can of course compile with --disable-asm on such chips, but we don't do it by default because
#-----we don't feel it's necessary to actively support chips on which x264 would be basically useless
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-opencl --disable-asm --enable-pic --enable-strip"
#-----FFmpeg:
#-----ARMv7 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=arm --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-neon --enable-thumb --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libx264 --enable-gpl --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----Intel CPUs used in Synology products are all 64bit and SSSE3 capable but existing DSM libs are 32bit, so use the i686 toolchain rather than x86_64
#-----libx264 requires yasm 1.20 which is not on Ubuntu apt-get
#-----https://trac.ffmpeg.org/wiki/UbuntuCompilationGuide
sudo apt-get remove yasm
wget http://www.tortall.net/projects/yasm/releases/yasm-1.2.0.tar.gz
tar xvzf yasm-1.2.0.tar.gz
cd yasm-1.2.0
./configure --prefix=/
make
sudo make install
cd..
wget "${DL_PATH}/Intel%20x86%20Linux%203.2.11%20%28Pineview%29/gcc421_glibc236_x86-GPL.tgz"
tar xvzf gcc421_glibc236_x86-GPL.tgz
export CROSS_PREFIX=i686-linux-gnu
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
export TARGET=i686-linux-gnu
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder --enable-nasm"
export SSL_CONFIG="./Configure.syno --prefix=${TOOLCHAIN} threads shared linux-generic32"
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-opencl --enable-pic --enable-strip"
#-----FFmpeg:
#-----Intel CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=x86 --enable-ssse3 --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libx264 --enable-gpl --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#-----QorIQ P1022 CPU is based on a dual PowerPC e500v2 core with Signal Processing Engine (SPE) which is not a classic FPU design, but no AltiVec vector unit 
#-----Some QorIQ models have e500mc cores with true FPUs but these are not used in 2013 series Synology NAS
#-----http://en.wikipedia.org/wiki/QorIQ
#-----http://cache.freescale.com/files/32bit/doc/fact_sheet/QP1022FS.pdf?fpsp=1
wget "${DL_PATH}/PowerPC%20QorIQ%20Linux%202.6.32/gcc4374_eglibc2874_qoriq-GPL.tgz"
tar xvzf gcc4374_eglibc2874_qoriq-GPL.tgz
export CROSS_PREFIX=powerpc-none-linux-gnuspe
export TOOLCHAIN=/usr/local/${CROSS_PREFIX}
export TARGET=powerpc-unknown-linux
export MARCH="-mcpu=8548 -mhard-float -mfloat-gprs=double"
export LAME_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static --disable-decoder"
export SSL_CONFIG="./Configure.syno --prefix=${TOOLCHAIN} threads shared linux-ppc"
#-----asm disabled since QorIQ has no AltiVec
export X264_CONFIG="./configure --prefix=${TOOLCHAIN} --host=${TARGET} --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --enable-shared --disable-opencl --disable-asm --enable-pic --enable-strip"
#-----FFmpeg:#-----PowerPC e500v2 CPU-specific options (based on inspecting 'configure' script https://github.com/FFmpeg/FFmpeg/blob/master/configure)
#-----pkg-config needs to be defined because with cross-prefix it assumes ${CROSS_PREFIX}-pkg-config which doesn't exist, and then librtmp won't be detected
export FF_CONFIG="./configure --arch=ppc --cpu=e500v2 --enable-cross-compile --cross-prefix=${TOOLCHAIN}/bin/${CROSS_PREFIX}- --target-os=linux --prefix=${TOOLCHAIN} --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-debug --enable-pthreads --enable-libmp3lame --enable-librtmp --enable-libass --enable-libx264 --enable-gpl --pkg-config=pkg-config --extra-version=compiled_by_patters_for_Serviio"




#----------------------------------------------------
#-----script continues

sudo mv ${CROSS_PREFIX} /usr/local
export AR=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ar
export CC=${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc
export CXX=${TOOLCHAIN}/bin/${CROSS_PREFIX}-g++
export LD=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ld
export LDSHARED="${TOOLCHAIN}/bin/${CROSS_PREFIX}-gcc -shared "
export RANLIB=${TOOLCHAIN}/bin/${CROSS_PREFIX}-ranlib
export CFLAGS="-I${TOOLCHAIN}/include -O3 ${MARCH}"
export LDFLAGS="-L${TOOLCHAIN}/lib"
export PKG_CONFIG_PATH="${TOOLCHAIN}/lib/pkgconfig"


#-----fetch Synology's DSM sources for certain libraries
wget https://dl.dropboxusercontent.com/u/1188556/DSM-source.tgz
tar xvzf DSM-source.tgz
cd DSM-source

#-----libz shared lib (toolchain only has static lib)
cd zlib-1.2
./configure --prefix=${TOOLCHAIN}
#-----owing to a bug in the configure script, libz cross compiles without a SONAME (check it with objdump -x)
#-----http://forums.gentoo.org/viewtopic-p-7156392.html
sed -i "s/\(^LDSHARED=.*$\)/\1 -Wl,-soname,libz.so.1,--version-script,zlib.map/" Makefile
make
make install
cd ..

#-----LAME shared lib
cd lame-398
$LAME_CONFIG
make
make install
cd ..

#-----shine shared lib for ARMv5
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && mv ../savonet-shine-* .
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && cd savonet-shine-*
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && ./configure --prefix=${TOOLCHAIN} --host=${CROSS_PREFIX} --build=x86_64-linux-gnu --enable-shared --disable-static
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && make
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && make install
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && cd ..

#-----openssl shared lib (Synology seem to have tweaked the source for each CPU)
cd openssl-1.0.x-${CROSS_PREFIX}
$SSL_CONFIG
make
make install
cd ..

#-----expat shared lib
cd expat-2.x
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..

#-----freetype shared lib
cd freetype-2.3.7
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..

#-----finished building DSM-bundled libs
cd ..


#-----fribidi shared lib
wget http://fribidi.org/download/fribidi-0.19.5.tar.bz2
tar xvjf fribidi-0.19.5.tar.bz2
cd fribidi-0.19.5
#-----patch fribidi source since it won't compile - some sort of page size function is no longer available
#-----error on ARMv5 and Intel: fribidi-run.c:70: error: 'PAGE_SIZE' undeclared (first use in this function)
#-----error on QorIQ: include/asm/kdump.h:41: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'void'
#-----clues found here:
#-----http://armbedded.eu/node/56
#-----http://lists.debian.org/debian-glibc/2006/10/msg00169.html
#-----http://my.safaribooksonline.com/book/operating-systems-and-server-administration/linux/0596009585/advanced-file-i-o/mapping_files_into_memory
[ "${CROSS_PREFIX}" == "arm-marvell-linux-gnueabi" ] || sed -i "s|include <asm/page.h>|include <unistd.h>\n#   define PAGE_SIZE sysconf(_SC_PAGESIZE)|" lib/common.h
./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
make
make install
cd ..

#-----fontconfig shared lib
wget http://freedesktop.org/software/fontconfig/release/fontconfig-2.10.91.tar.gz
tar xvzf fontconfig-2.10.91.tar.gz
cd fontconfig-2.10.91
LDFLAGS=-Wl,-rpath,\'\$\$ORIGIN\' ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
cd ..

#-----libass shared lib
wget http://libass.googlecode.com/files/libass-0.10.1.tar.gz
tar xvzf libass-0.10.1.tar.gz
cd libass-0.10.1
LDFLAGS=-Wl,-rpath,\'\$\$ORIGIN\' ./configure --prefix=${TOOLCHAIN} --host=${TARGET} --build=x86_64-linux-gnu --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
make
make install
#-----Although libass would link correctly to older FFmpeg versions, this mod is now necessary:
#-----http://ffmpeg.org/pipermail/ffmpeg-user/2012-November/010906.html
sed -i -r "s/(^Libs: .*$)/\1 -lfontconfig -lfribidi -lfreetype -lexpat/" $TOOLCHAIN/lib/pkgconfig/libass.pc
cd ..

#-----librtmp shared lib
wget http://download.serviio.org/opensource/rtmpdump.tar.gz
tar xvzf rtmpdump.tar.gz
cd rtmpdump
make CROSS_COMPILE=${TOOLCHAIN}/bin/${CROSS_PREFIX}- SYS=posix prefix=${TOOLCHAIN} INC=-I${TOOLCHAIN}/include XLDFLAGS=-L${TOOLCHAIN}/lib
make install prefix=${TOOLCHAIN}
cd ..

#-----libx264 shared lib
wget http://download.serviio.org/opensource/last_x264.tar.bz2
tar xvfj last_x264.tar.bz2
cd x264-snapshot-20130529-2245
$X264_CONFIG
make
make install
cd ..

#-----FFmpeg
wget http://ffmpeg.org/releases/ffmpeg-2.1.tar.gz
tar xvzf ffmpeg-2.1.tar.gz
cd ffmpeg-2.1

#-----ARM-specific fixed point maths modification for mpeg audio encoder
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && sed -i "s/^#define USE_FLOATS//" libavcodec/mpegaudioenc.c

#-----QorIQ-specific fix provided by Synology Support 
[ "${CROSS_PREFIX}" == "powerpc-none-linux-gnuspe" ] && wget https://dl.dropboxusercontent.com/u/1188556/ffmpeg-synology-qoriq.patch
[ "${CROSS_PREFIX}" == "powerpc-none-linux-gnuspe" ] && patch --verbose -p1 < ffmpeg-synology-qoriq.patch

#-----set library linker to use relative library search path (../lib and .)
#-----tricky shell escaping of $ORIGIN is required http://itee.uq.edu.au/~daniel/using_origin/
#-----http://stackoverflow.com/questions/6562403/i-dont-understand-wl-rpath-wl
#-----the use of -rpath-link appears to be necessary for compilation using the Armada370 toolchain
LDFLAGS=-Wl,-rpath,\'\$\$ORIGIN/../lib:\$\$ORIGIN\',-rpath-link,${TOOLCHAIN}/lib ${FF_CONFIG}

#-----external libs (librtmp, libass) don't seem to link correctly as shared libs
#-----on all toolchains except Marvell Kirkwood without this fix (check compiled binary with objdump -x)
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] || sed -r -i 's%(^LDFLAGS.*)\$\(LDFLAGS\)%\1-Wl,-rpath=z888z\$\$ORIGIN/../lib:\$\$ORIGINz888z%' common.mak; sed -i "s/z888z/'/g" common.mak

make
make install
cd ..

mkdir native-${CROSS_PREFIX}
cd native-${CROSS_PREFIX}
cp ${TOOLCHAIN}/bin/ffmpeg .
cp ${TOOLCHAIN}/lib/libass.so.4 .
cp ${TOOLCHAIN}/lib/libavcodec.so.55 .
cp ${TOOLCHAIN}/lib/libavdevice.so.55 .
cp ${TOOLCHAIN}/lib/libavfilter.so.3 .
cp ${TOOLCHAIN}/lib/libavformat.so.55 .
cp ${TOOLCHAIN}/lib/libavutil.so.52 .
cp ${TOOLCHAIN}/lib/libfontconfig.so.1 .
cp ${TOOLCHAIN}/lib/libfribidi.so.0 .
cp ${TOOLCHAIN}/lib/libpostproc.so.52 .
cp ${TOOLCHAIN}/lib/librtmp.so.0 .
cp ${TOOLCHAIN}/lib/libswresample.so.0 .
cp ${TOOLCHAIN}/lib/libswscale.so.2 .
cp ${TOOLCHAIN}/lib/libx264.so.133 .
[ "${CROSS_PREFIX}" == "arm-none-linux-gnueabi" ] && cp ${TOOLCHAIN}/lib/libshine.so.3 .
sudo chown root:root *
tar -cvzf ffmpeg-syno-${CROSS_PREFIX}.tgz *
mkdir oldDSM
cd oldDSM
cp ${TOOLCHAIN}/lib/libz.so.1 .
cp ${TOOLCHAIN}/lib/libcrypto.so.1.0.0 .
cp ${TOOLCHAIN}/lib/libssl.so.1.0.0 .
cp ${TOOLCHAIN}/lib/libmp3lame.so.0 .
cp ${TOOLCHAIN}/lib/libfreetype.so.6 .
cp ${TOOLCHAIN}/lib/libexpat.so.1 .
sudo chown root:root *
tar -cvzf ffmpeg-syno-${CROSS_PREFIX}-oldDSM.tgz *
cd ..
 

Native compile for ARMv5 systems

Here is the method to natively compile FFmpeg with libshine and libass, with shared libraries rather than static. It transpires that the cross compiled build has a broken AC3 encoder which produces crackling and muffled audio streams. The guide assumes a bootstrapped NAS with the optware-devel tools installed. See my much earlier blog post for details of how to set that up, since there are some pitfalls. I haven’t run through and tested the whole sequence – I have assembled this from hundreds of lines of notes I made while working – but I’m publishing it here to help others, and to refer to in future when I need to build newer FFmpeg revisions for future Serviio releases.

#-----on a bootstrapped system that's since had its DSM upgraded the path has been reset
#export PATH=/opt/bin:/opt/sbin:$PATH

#-----fetch headers for zlib, OpenSSL, LAME, freetype2, and expat libraries obtained by compiling the DSM 4.1 included versions from source, plus ncurses headers from Synology ARMv5 cross toolchain
cd /tmp
mkdir include
cd include
wget --no-check-certificate https://dl.dropboxusercontent.com/u/1188556/synology-armv5te-headers.tgz
tar xvzf synology-armv5te-headers.tgz

#-----this archive also contains pkgconfig definitions for DSM's existing versions of libcrypto, libssl and freetype2, edited to fit requirements
#-----this will allow librtmp to link to DSM's existing OpenSSL libraries, and libass to link to DSM's bundled freetype2 and expat libraries
cp *.pc /opt/lib/pkgconfig
cd /volume1/@tmp

#-----fetch savonet-shine source which has already had the bootstrap script run on a Linux system with automake
#-----on the Linux system you would need to run:
#sudo apt-get automake
#sudo apt-get libtool
#wget https://github.com/savonet/shine/zipball/master
#unzip master
#cd savonet-shine-*
#./bootstrap
#cd ..
#-----but here's one I made earlier:
wget --no-check-certificate https://dl.dropboxusercontent.com/u/1188556/savonet-shine-bc9e3b6.tgz
tar xvzf savonet-shine-bc9e3b6.tgz
cd savonet-shine-bc9e3b6
./configure --prefix=/opt --enable-shared --disable-static
make
#-----don't worry about the repeated message #warning NO MULT FILE FOR ARCHITECTURE - USING GENERIC MATH
#-----the include for mult_sarm_gcc.h (optimized ARM maths in assembler) is commented out in src/lib/types.h
#-----the comment mentions that fixing this is on the to-do list, so it can't be helped
make install
cd ..

#-----fetch libfdk-aac fixed point math AAC encoder
#-----binaries linked to it cannot be redistributed under GPL
#-----so it is excluded from the FFmpeg I built for the Serviio package
wget http://sourceforge.net/projects/opencore-amr/files/fdk-aac/fdk-aac-0.1.2.tar.gz
tar xvzf fdk-aac-0.1.2.tar.gz
cd fdk-aac-0.1.2
./configure --prefix=/opt --disable-static --enable-shared
make
make install
cd ..

#-----fetch libRTMP source
wget http://download.serviio.org/opensource/rtmpdump.tar.gz
tar xvzf rtmpdump.tar.gz
cd rtmpdump
#-----the following options will force it to use the existing OpenSSL libs thanks to the imported headers
make SYS=posix prefix=/opt INC=-I/tmp/include XLDFLAGS=-L/lib
make install prefix=/opt
cp -R /opt/include/librtmp /tmp/include
#-----remove unsupported URL line from pkgconfig/librtmp.pc
#-----pkg-config --exists --print-errors librtmp
sed -i -e '/^URL/d' /opt/lib/pkgconfig/librtmp.pc
cd ..

#-----fetch fribidi source
wget http://fribidi.org/download/fribidi-0.19.5.tar.bz2
tar xvjf fribidi-0.19.5.tar.bz2
cd fribidi-0.19.5
#-----patch fribidi source since it won't compile on ARMv5
#-----fribidi-run.c:70: error: 'PAGE_SIZE' undeclared (first use in this function)
#-----clues found here:
#-----http://armbedded.eu/node/56
#-----http://lists.debian.org/debian-glibc/2006/10/msg00169.html
#-----http://my.safaribooksonline.com/book/operating-systems-and-server-administration/linux/0596009585/advanced-file-i-o/mapping_files_into_memory
sed -i "s|include <asm/page.h>|include <unistd.h>\n#   define PAGE_SIZE sysconf(_SC_PAGESIZE)|" lib/common.h
./configure --prefix=/opt --enable-shared --disable-static
make
make install
cd ..

#-----fetch fontconfig source
wget http://freedesktop.org/software/fontconfig/release/fontconfig-2.10.91.tar.gz
tar xvzf fontconfig-2.10.91.tar.gz
cd fontconfig-2.10.91
LDFLAGS=-Wl,-rpath='$$ORIGIN' ./configure --prefix=/opt --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
sed -i "s|^sys_lib_search_path_spec="/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3 /opt/arm-none-linux-gnueabi/lib /opt/lib /lib /usr/lib"|sys_lib_search_path_spec="/lib /usr/lib /opt/lib/gcc/arm-none-linux-gnueabi/4.2.3 /opt/arm-none-linux-gnueabi/lib /opt/lib"|" libtool
make
make install
cd ..

#-----fetch libass source
wget http://libass.googlecode.com/files/libass-0.10.1.tar.gz
tar xvzf libass-0.10.1.tar.gz
cd libass-0.10.1
export CFLAGS=-I/tmp/include
LDFLAGS=-Wl,-rpath='$$ORIGIN' ./configure --prefix=/opt --enable-shared --disable-static
#-----correct some non-standard library linking behaviour
sed -i "s/^hardcode_into_libs=yes/hardcode_into_libs=no/" libtool
sed -i "s|^sys_lib_search_path_spec="/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3 /opt/arm-none-linux-gnueabi/lib /opt/lib /lib /usr/lib"|sys_lib_search_path_spec="/lib /usr/lib /opt/lib/gcc/arm-none-linux-gnueabi/4.2.3 /opt/arm-none-linux-gnueabi/lib /opt/lib"|" libtool
make
make install
cd ..

#-----libx264 shared lib
wget http://download.serviio.org/opensource/last_x264.tar.bz2
tar xvfj last_x264.tar.bz2
cd x264-snapshot-20130529-2245
sed -i "s/^\${SRCPATH}\/version\.sh/bash \${SRCPATH}\/version\.sh/" ./configure
#-----http://forum.doom9.org/showthread.php?t=160584
#-----https://mailman.videolan.org/pipermail/x264-devel/2010-December/008093.html
#-----x264 builds with NEON by default because x264 is so slow without NEON (and on any non-NEON chip) as to be useless.
#-----You can of course compile with --disable-asm on such chips, but we don't do it by default because
#-----we don't feel it's necessary to actively support chips on which x264 would be basically useless
bash ./configure --prefix=/opt --enable-shared --disable-opencl --disable-asm
make
make install
cd ..

#-----FFmpeg
wget http://ffmpeg.org/releases/ffmpeg-2.1.tar.gz
tar xvzf ffmpeg-2.1.tar.gz
cd ffmpeg-2.1

#-----ARMv5-specific fixed point maths modification for mpeg audio encoder, mentioned in same ticket
sed -i "s/^#define USE_FLOATS//" libavcodec/mpegaudioenc.c

#-----FFmpeg configure parameters for native compile with shared libraries
#-----something changed in the FFmpeg configure script and now vfp and neon optimizations have to be explicitly disabled for ARMv5TE, which doesn't have those features in any case
export FF_CONFIG="./configure --arch=arm --cpu=armv5te --prefix=/opt --extra-cflags=-I/tmp/include --enable-shared --disable-static --enable-pic --disable-ffplay --disable-ffserver --disable-vfp --disable-neon --disable-debug --enable-pthreads --enable-libshine --enable-libfdk_aac --enable-librtmp --enable-libass --enable-libx264 --enable-gpl --enable-nonfree --extra-version=compiled_by_patters_for_Serviio"

#-----set library linker to use relative library search path (../lib:.)
#-----tricky shell escaping of $ORIGIN is required http://itee.uq.edu.au/~daniel/using_origin/
#-----http://stackoverflow.com/questions/6562403/i-dont-understand-wl-rpath-wl
LDFLAGS=-Wl,-rpath=\'\$\$ORIGIN/../lib:\$\$ORIGIN\' ${FF_CONFIG}
make
make install

mkdir /volume1/@tmp/binaries
cd /volume1/@tmp/binaries
cp /opt/bin/ffmpeg .
cp /opt/lib/libass.so.4 .
cp /opt/lib/libavcodec.so.55 .
cp /opt/lib/libavdevice.so.55 .
cp /opt/lib/libavfilter.so.3 .
cp /opt/lib/libavformat.so.55 .
cp /opt/lib/libavutil.so.52 .
cp /opt/lib/libfontconfig.so.1 .
cp /opt/lib/libfribidi.so.0 .
cp /opt/lib/libpostproc.so.52 .
cp /opt/lib/librtmp.so.0 .
cp /opt/lib/libshine.so.3 .
cp /opt/lib/libfdk-aac.so.0 .
cp /opt/lib/libswresample.so.0 .
cp /opt/lib/libswscale.so.2 .
cp /opt/lib/libx264.so.133 .
tar -cvzf ffmpeg-syno-armv5tel.tgz *
mkdir oldDSM
cd oldDSM
cp /lib/libz.so.1 .
cp /lib/libcrypto.so.1.0.0 .
cp /lib/libssl.so.1.0.0 .
cp /lib/libmp3lame.so.0 .
cp /lib/libfreetype.so.6 .
cp /lib/libexpat.so.1 .
tar -cvzf ffmpeg-syno-armv5tel-oldDSM.tgz *
 

Here’s how the library dependencies look. There are no links to any libraries in /opt, and DSM-bundled libraries in /lib are linked to where possible. All the other new libraries are kept within the Serviio lib folder (../lib relative to the ffmpeg binary).

/$ ldd /volume1/@appstore/Serviio/bin/ffmpeg
        libavdevice.so.55 => /volume1/@appstore/Serviio/bin/../lib/libavdevice.so.55 (0x40026000)
        libavfilter.so.3 => /volume1/@appstore/Serviio/bin/../lib/libavfilter.so.3 (0x40039000)
        libavformat.so.55 => /volume1/@appstore/Serviio/bin/../lib/libavformat.so.55 (0x40112000)
        libavcodec.so.55 => /volume1/@appstore/Serviio/bin/../lib/libavcodec.so.55 (0x40271000)
        libpostproc.so.52 => /volume1/@appstore/Serviio/bin/../lib/libpostproc.so.52 (0x4110c000)
        libswresample.so.0 => /volume1/@appstore/Serviio/bin/../lib/libswresample.so.0 (0x4111e000)
        libswscale.so.2 => /volume1/@appstore/Serviio/bin/../lib/libswscale.so.2 (0x4113b000)
        libavutil.so.52 => /volume1/@appstore/Serviio/bin/../lib/libavutil.so.52 (0x4118c000)
        libx264.so.133 => /volume1/@appstore/Serviio/bin/../lib/libx264.so.133 (0x411f3000)
        libm.so.6 => /lib/libm.so.6 (0x41346000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x413f5000)
        libc.so.6 => /lib/libc.so.6 (0x41415000)
        librt.so.1 => /lib/librt.so.1 (0x41544000)
        libass.so.4 => /volume1/@appstore/Serviio/bin/../lib/libass.so.4 (0x41553000)
        librtmp.so.0 => /volume1/@appstore/Serviio/bin/../lib/librtmp.so.0 (0x41585000)
        libz.so.1 => /lib/libz.so.1 (0x415a5000)
        libshine.so.3 => /volume1/@appstore/Serviio/bin/../lib/libshine.so.3 (0x415c2000)
        libfdk-aac.so.0 => /volume1/@appstore/Serviio/bin/../lib/libfdk-aac.so.0 (0x415d4000)
        /lib/ld-linux.so.3 (0x40000000)
        libfreetype.so.6 => /lib/libfreetype.so.6 (0x41678000)
        libfribidi.so.0 => /volume1/@appstore/Serviio/bin/../lib/libfribidi.so.0 (0x41737000)
        libfontconfig.so.1 => /volume1/@appstore/Serviio/bin/../lib/libfontconfig.so.1 (0x41756000)
        libssl.so.1.0.0 => /lib/libssl.so.1.0.0 (0x4178d000)
        libcrypto.so.1.0.0 => /lib/libcrypto.so.1.0.0 (0x417db000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x4196f000)
        libexpat.so.1 => /lib/libexpat.so.1 (0x41983000)
        libdl.so.2 => /lib/libdl.so.2 (0x419aa000)
/$

Bliss album art manager package for Synology NAS

bliss-UI

 

Bliss is a Java application written by Dan Gravell which can manage file names, tags and album art for your music collection and which can also enforce their consistency. It is designed to be left running once installed so that albums you add later will have these same policies applied to them automatically. It supports a wide range of music formats, and effortlessly deals with very large collections – apparently even ones containing quite obscure recordings. My own collection didn’t really put this to the test, since it doesn’t contain bootlegs, live sets or rarities plus I had also already obtained cover art for it (from back when CoverFlow first graced the screen of my iPhone 2G).

I could see from referrals to this blog that people were asking Dan for a Synology package which he didn’t have the time to investigate, and so I thought that it would make an interesting little project, especially since a NAS is the ideal device to run Bliss on. Although there was already a Howto post on the Synology forums, that guide only really covered getting the basic functionality of Bliss up and running – it was missing the best bits: the filesystem watching and audio fingerprinting features. These depend on natively compiled binaries, but Bliss doesn’t support running on ARM or PowerPC CPUs, and its Intel binaries don’t work on Synology either (the usual problem – glibc too old). Getting these working provided precisely the sort of challenge I like. Not only were they difficult to compile, but getting them integrated into the various OSGi ‘bundles’ that make up Bliss was quite involved too.

Bliss uses an open source library called Chromaprint, itself part of the wider Acoustid project. The aim is to scan an audio file, to produce a fingerprint of the sound, and then to compare this against an open online database such as MusicBrainz.org to identify music regardless of compression codec used. Its author Lukáš Lalinský explains how it works. It turns out that Chromaprint uses some complex maths functions that FFmpeg can provide (specifically Fourier Transform), and FFmpeg’s shared libraries are already included with Synology DSM. I was able to download the sources that were used to build DSM, which then allowed me to build Chromaprint linked to those existing libraries – resulting in a minuscule 84KB build of fpcalc, rather than the statically compiled ones for various OS and CPU architectures included with Bliss, which weigh in at several megabytes. I think I’m finally ‘getting’ what open source is all about, which is nice since that was my objective in experimenting with my NAS. To prevent fpcalc building and linking to its dynamic library libchromaprint.so and to get it to detect FFmpeg properly I had to carefully inspect the Makefiles to find the correct build syntax:
FFMPEG_DIR=/usr/local/i686-linux-gnu cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_EXAMPLES=ON -DBUILD_SHARED_LIBS=NO .
FFMPEG_DIR is the base folder from which CMake will look for lib/libavcodec.so and include/libavcodec/avfft.h.

It’s a bit of a nuisance that on ARM you first have to compile CMake because it doesn’t exist on the Optware ipkg repository, but it was straightforward enough to build. To build the Intel binaries I cross-compiled on a Ubuntu Desktop 12 virtual machine using the Synology toolchain (see the Synology DSM 3rd party developer guide for details).

For watching the filesystem Bliss uses JNotify to hook into the Linux kernel’s inotify subsystem. Getting this compiled was tricky. It seems no one has reported successfully compiling it for an ARM CPU, and JNotify’s author Omry Yadan wasn’t aware of anyone doing this either. The problem is that compilation halts with these errors:

In file included from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
../inotify-syscalls.h:35:1: error: "__NR_inotify_init" redefined
In file included from /opt/include/sys/syscall.h:25,
                 from ../inotify-syscalls.h:4,
                 from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
/opt/include/asm/unistd.h:344:1: error: this is the location of the previous definition
In file included from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
../inotify-syscalls.h:36:1: error: "__NR_inotify_add_watch" redefined
In file included from /opt/include/sys/syscall.h:25,
                 from ../inotify-syscalls.h:4,
                 from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
/opt/include/asm/unistd.h:345:1: error: this is the location of the previous definition
In file included from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
../inotify-syscalls.h:37:1: error: "__NR_inotify_rm_watch" redefined
In file included from /opt/include/sys/syscall.h:25,
                 from ../inotify-syscalls.h:4,
                 from ../net_contentobjects_jnotify_linux_JNotify_linux.c:43:
/opt/include/asm/unistd.h:346:1: error: this is the location of the previous definition

By searching in a very generic way for a solution I found this post on stackoverflow.com which helped me to patch the header inotify-syscalls.h to get around the problem. Since there is no JDK for ARM embedded systems I included the headers from OpenJDK6 which I took from a Ubuntu VM.

Compiling on Intel also required a fix. I was getting the same error as featured in this post on the JNotify user forum on SourceForge.net:
expected specifier-qualifier-list before ‘pid_t’

Despite some people’s apparent success simply rearranging the order of the includes in net_contentobjects_jnotify_linux_JNotify_linux.c this didn’t help me. I’m not sure quite how I stumbled upon it, but I found the solution staring at me on this page:
The size_t, ssize_t, uid_t, gid_t, off_t and pid_t types are defined as described in sys/types.h

I inserted an additional include in for sys/types.h and it compiled ok. It’s worth pointing out that, although all Intel Synology units are x86-64 architecture, the Oracle JRE for Embedded is x86 (32 bit), so I used the i686 toolchain. Synology DSM’s FFmpeg shared libraries are also 32 bit so my build of fpcalc needed to comply with this. Bliss nonetheless expects the binary to be called fpcalc_linux64 since Bliss is detecting the underlying Linux CPU architecture.

Once I had got past the obstacle of compiling the native code, I needed to liaise back and forth with Dan to understand how Bliss was dealing with its libraries and how I could replace the built-in versions. Originally this was quite a kludge but Dan has since abstracted out the native binaries into their own OSGI bundle fragments which makes things a lot easier, and should allow Bliss to survive an in-app update. The Synology package provides the following architecture-specific jar files (with corresponding edits to their manifest). Thank you to Dan for all the quick answers!

  • net.contentobjects.jnotify.linux.amd64-0.94.0.jar
  • net.contentobjects.jnotify.linux.ARM_le-0.94.0.jar
  • net.contentobjects.jnotify.linux.PowerPC-0.94.0.jar
  • com.elsten.bliss.policy.tag.auto.linux.amd64-1.0.0.jar – contains fpcalc.
  • com.elsten.bliss.policy.tag.auto.linux.ARM_le-1.0.0.jar
  • com.elsten.bliss.policy.tag.auto.linux.PowerPC-1.0.0.jar

Here are my fixes to JNotify to get the native library part compiled, since they may help someone else:

 

Patch for native compile of JNotify 0.94 on ARM Synology

diff -crB jnotify-vanilla/inotify-syscalls.h jnotify-arm/inotify-syscalls.h
*** jnotify-vanilla/inotify-syscalls.h	2005-11-30 15:07:56.000000000 +0000
--- jnotify-arm/inotify-syscalls.h	2012-09-14 02:43:44.032130098 +0100
***************
*** 32,40 ****
  # define __NR_inotify_add_watch	152
  # define __NR_inotify_rm_watch	156
  #elif defined (__arm__)
! # define __NR_inotify_init	316
! # define __NR_inotify_add_watch	317
! # define __NR_inotify_rm_watch	318
  #elif defined (__SH4__)
  # define __NR_inotify_init	290
  # define __NR_inotify_add_watch	291
--- 32,46 ----
  # define __NR_inotify_add_watch	152
  # define __NR_inotify_rm_watch	156
  #elif defined (__arm__)
! # ifndef __NR_inotify_init
! #  define __NR_inotify_init     316
! # endif
! # ifndef __NR_inotify_add_watch
! #  define __NR_inotify_add_watch 317
! # endif
! # ifndef __NR_inotify_rm_watch
! #  define __NR_inotify_rm_watch 318
! # endif
  #elif defined (__SH4__)
  # define __NR_inotify_init	290
  # define __NR_inotify_add_watch	291
diff -crB jnotify-vanilla/Release/subdir.mk jnotify-arm/Release/subdir.mk
*** jnotify-vanilla/Release/subdir.mk	2011-02-28 18:07:20.000000000 +0000
--- jnotify-arm/Release/subdir.mk	2012-09-14 02:29:00.000000000 +0100
***************
*** 17,23 ****
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	gcc -I/usr/lib/jvm/java-6-sun/include -I/usr/lib/jvm/java-6-sun/include/linux -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
  
--- 17,23 ----
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	gcc -I/volume1/public/temp/jdk_include/ -I/volume1/public/temp/jdk_include/linux -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
 

Patch for cross compile of JNotify 0.94 for other Synology CPU architectures, using Ubuntu Desktop 12

diff -crB jnotify-vanilla/net_contentobjects_jnotify_linux_JNotify_linux.c jnotify-i686/net_contentobjects_jnotify_linux_JNotify_linux.c
*** jnotify-vanilla/net_contentobjects_jnotify_linux_JNotify_linux.c	2011-02-28 18:07:20.000000000 +0000
--- jnotify-xcomp/net_contentobjects_jnotify_linux_JNotify_linux.c	2012-09-14 00:41:53.455010206 +0100
***************
*** 36,41 ****
--- 36,42 ----
  #include <sys/time.h>
  #include <sys/select.h>
  #include <sys/ioctl.h>
+ #include <sys/types.h>
  #include <errno.h>
  #include <stdio.h>
  #include <unistd.h>
diff -crB jnotify-vanilla/Release/makefile jnotify-i686/Release/makefile
*** jnotify-vanilla/Release/makefile	2011-02-28 18:07:20.000000000 +0000
--- jnotify-xcomp/Release/makefile	2012-09-14 00:37:56.475007855 +0100
***************
*** 28,34 ****
  libjnotify.so: $(OBJS) $(USER_OBJS)
  	@echo 'Building target: $@'
  	@echo 'Invoking: GCC C Linker'
! 	gcc -shared -o"libjnotify.so" $(OBJS) $(USER_OBJS) $(LIBS)
  	@echo 'Finished building target: $@'
  	@echo ' '
  
--- 28,34 ----
  libjnotify.so: $(OBJS) $(USER_OBJS)
  	@echo 'Building target: $@'
  	@echo 'Invoking: GCC C Linker'
! 	$(CC) -shared -o"libjnotify.so" $(OBJS) $(USER_OBJS) $(LIBS)
  	@echo 'Finished building target: $@'
  	@echo ' '
  
diff -crB jnotify-vanilla/Release/subdir.mk jnotify-i686/Release/subdir.mk
*** jnotify-vanilla/Release/subdir.mk	2011-02-28 18:07:20.000000000 +0000
--- jnotify-xcomp/Release/subdir.mk	2012-09-14 00:37:33.835007731 +0100
***************
*** 17,23 ****
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	gcc -I/usr/lib/jvm/java-6-sun/include -I/usr/lib/jvm/java-6-sun/include/linux -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
  
--- 17,23 ----
  %.o: ../%.c
  	@echo 'Building file: $<'
  	@echo 'Invoking: GCC C Compiler'
! 	$(CC) -I$(TOOLCHAIN)/include -O3 -Wall -Werror -c -fmessage-length=0 -fPIC -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@:%.o=%.d)" -o"$@" "$<"
  	@echo 'Finished building: $<'
  	@echo ' '
 
 

Package instructions

The package can be installed on an unmodified NAS – no hacking or bootstrapping is required. Only Marvell Kirkwood, Intel, and Freescale QorIQ PowerPC CPUs are supported, so please check which CPU your NAS has.

  • Firstly, in the DSM User control panel enable the User Home service.
  • Open the DSM Package Center. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk
  • Install either one of my Java SE for Embedded Synology packages (Java 6 or Java 7). Only Marvell Kirkwood, Freescale QorIQ, or Intel CPUs are supported. Please read all the notes and instructions on that page carefully.
  • Install Bliss using DSM’s Package Center.
  • The rest is self-explanatory.
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology Package wiki.

installer.sh

#!/bin/sh

#--------BLISS installer script
#--------package maintained at pcloadletter.co.uk

DOWNLOAD_FILE="latest-linux-version"
DOWNLOAD_PATH="http://www.blisshq.com/app"
DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_PASS="`openssl rand 12 -base64 2>nul`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
ENGINE_SCRIPT="bliss.sh"
MIGRATION_FOLDER="${DAEMON_USER}_data_mig"
SYNO_CPU_ARCH="`uname -m`"
NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/bliss-native-${SYNO_CPU_ARCH}.tgz"   
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
INSTALL_FILES="${DOWNLOAD_URL}"
source /etc/profile

TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
APP_TEMP="${TEMP_FOLDER}/${SYNOPKG_PKGNAME}"
DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
PRIMARY_VOLUME="/`echo $TEMP_FOLDER | cut -f2 -d'/'`"


preinst ()
{
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  #is the User Home service enabled?
  UH_SERVICE=maybe
  synouser --add userhometest Testing123 "User Home test user" 0 "" ""
  UHT_HOMEDIR=`cat /etc/passwd | sed -r '/User Home test user/!d;s/^.*:User Home test user:(.*):.*$/\1/'`
  if echo $UHT_HOMEDIR | grep '/var/services/homes/' > /dev/null; then
    if [ ! -d $UHT_HOMEDIR ]; then
      UH_SERVICE=false
    fi
  fi
  synouser --del userhometest
  #remove home directory (needed since DSM 4.1)
  [ -e /var/services/homes/userhometest ] && rm -r /var/services/homes/userhometest
  if [ ${UH_SERVICE} == "false" ]; then
    echo "The User Home service is not enabled. Please enable this feature in the User control panel in DSM."
    exit 1
  fi
  #fetch the URL for the latest installer version from blisshq.com  
  _download
  DOWNLOAD_URL=`cat ${TEMP_FOLDER}/${DOWNLOAD_FILE}`
  INSTALL_FILES="${DOWNLOAD_URL} ${NATIVE_BINS_URL}"
  #now fetch the installer
  _download
  exit 0
}

_download ()
{
  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE} ]; then
        cp ${PUBLIC_FOLDER}/${DOWNLOAD_FILE} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, "
        echo "which was \"${WGET_URL}\" "
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. "
        exit 1
      fi
    fi
  done
}


postinst ()
{
  #create daemon user
  synouser --add ${DAEMON_USER} ${DAEMON_PASS} "${DAEMON_ID}" 0 "" ""
  
  #determine the daemon user homedir and save that variable in the user's profile
  #this is needed because new users seem to inherit a HOME value of /root which they have no permissions for
  DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
  su - ${DAEMON_USER} -s /bin/sh -c "echo export HOME=\'${DAEMON_HOME}\' >> .profile"

  #set working folder (/tmp is very small on Synology)
  #and reduce Java prefs system polling frequency to allow hibernation
  su - ${DAEMON_USER} -s /bin/sh -c "echo export VMARGS=\'-Djava.io.tmpdir=${APP_TEMP} -Djava.util.prefs.syncInterval=86400\' >> .profile"
  
  #once again fetch the URL for the latest installer version from blisshq.com
  #and use it to determine the filename we downloaded during preinst
  _download
  DOWNLOAD_FILE=`cat ${TEMP_FOLDER}/${DOWNLOAD_FILE} | sed -r "s%^.*/(.*)%\1%"`

  #run the installer
  cd ${TEMP_FOLDER}
  echo INSTALL_PATH=${SYNOPKG_PKGDEST} > ${TEMP_FOLDER}/bliss-synology.properties
  java -jar ${TEMP_FOLDER}/${DOWNLOAD_FILE} -options ${TEMP_FOLDER}/bliss-synology.properties
  rm ${TEMP_FOLDER}/bliss-synology.properties
  rm ${TEMP_FOLDER}/${DOWNLOAD_FILE}
  sed -i "s%^#!/bin/bash%#!/bin/sh%" ${SYNOPKG_PKGDEST}/bin/${ENGINE_SCRIPT}
  
  #stow jar files containing Synology versions of native code
  #(libjnotify, and fpcalc from the Chromaprint audio fingerprinting library)
  mkdir ${SYNOPKG_PKGDEST}/syno-native
  cd ${SYNOPKG_PKGDEST}/syno-native
  tar xzf ${TEMP_FOLDER}/${NATIVE_BINS_FILE}

  #change owner of folder tree and temp files
  chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
  chown -R ${DAEMON_USER} ${DAEMON_HOME} 
  [ -d ${APP_TEMP} ] && chown -R ${DAEMON_USER} ${APP_TEMP}

  exit 0
}


preuninst ()
{
  #make sure daemon is stopped
  /var/packages/${SYNOPKG_PKGNAME}/scripts/start-stop-status stop
  sleep 6
  
  exit 0
}


postuninst ()
{
  #remove daemon user
  synouser --del ${DAEMON_USER}
  
  #clean up temp
  [ -d ${TEMP_FOLDER}/Bliss ] && rm -r ${TEMP_FOLDER}/Bliss

  exit 0
}


preupgrade ()
{
  #make sure daemon is stopped
  /var/packages/${SYNOPKG_PKGNAME}/scripts/start-stop-status stop
  sleep 6
  
  #if config data exists back it up
  if [ -d ${DAEMON_HOME}/.bliss ]; then
    mkdir ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_migration
    mv ${DAEMON_HOME}/.bliss ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_migration
    mv ${DAEMON_HOME}/.java ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_migration
  fi

  exit 0
}


postupgrade ()
{
  #use the migrated config data from the previous version
  if [ -d ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_migration/.bliss ]; then
    mv ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_migration/.bliss ${DAEMON_HOME}
    mv ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_migration/.java ${DAEMON_HOME}
    rmdir ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_migration
    
    #daemon user has been deleted and recreated so we need to reset ownership (new UID)
    chown -R ${DAEMON_USER} ${DAEMON_HOME}
  fi
  
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------BLISS start-stop-status script
#--------package maintained at pcloadletter.co.uk

DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
ENGINE_SCRIPT="bliss.sh"
PIDFILE="${SYNOPKG_PKGDEST}/${SYNOPKG_PKGNAME}.pid"
SYNO_CPU_ARCH="`uname -m`"

_findpid ()
{
  ps -w | grep "^ *[0-9]* ${DAEMON_USER} .*java.*-Djava\.io\.tmpdir=/volume1/@tmp/Bliss" | grep -v "grep" | awk '{ print $1 }'
}


case $1 in
  start)
    PID=`_findpid`
    if [ -n "$PID" ]; then
      echo daemon is already running with pid $PID
      exit 1;
    fi
    DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
    
    #set the current timezone for Java so that log timestamps are accurate
    #we need to use the modern timezone names so that Java can figure out DST
    SYNO_TZ=`cat /etc/synoinfo.conf | grep timezone | cut -f2 -d'"'`
    SYNO_TZ=`grep "^${SYNO_TZ}" /usr/share/zoneinfo/Timezone/tzname | sed -e "s/^.*= //"`
    grep "^export TZ" ${DAEMON_HOME}/.profile > /dev/null \
     && sed -i "s%^export TZ=.*$%export TZ='${SYNO_TZ}'%" ${DAEMON_HOME}/.profile \
     || echo export TZ=\'${SYNO_TZ}\' >> ${DAEMON_HOME}/.profile
    
    #update the package version number in case of an in-app update
    #find must write out to a file because vars cannot be defined in subshells
    #http://mywiki.wooledge.org/BashFAQ/024
    find ${SYNOPKG_PKGDEST}/felix-cache -name "*.info" > /tmp/bliss-v-check.txt
    while IFS="" read -r FILE_TO_PARSE; do
      if [ -e ${FILE_TO_PARSE} ]; then
        if grep "com.elsten.bliss.bundle" ${FILE_TO_PARSE} > /dev/null; then
          BLISS_BUNDLE_DIR="`dirname ${FILE_TO_PARSE}`"
        fi
      fi
    done < /tmp/bliss-v-check.txt
    #read current version
    BLISS_VERSION=`grep "version" /var/packages/${SYNOPKG_PKGNAME}/INFO | sed -r "s/^.*([0-9]{8}).*$/\1/"`
    if [ -d ${BLISS_BUNDLE_DIR} ]; then
      find ${BLISS_BUNDLE_DIR} -name *.jar > /tmp/bliss-v-check.txt
      while IFS="" read -r FILE_TO_PARSE; do
        if [ -e ${FILE_TO_PARSE} ]; then
          DETECTED_VERSION=`unzip -p ${FILE_TO_PARSE} META-INF/MANIFEST.MF | grep Bundle-Version | cut -f4 -d'.' |  cut -c1-8`
        fi
        if [ ${DETECTED_VERSION} -gt ${BLISS_VERSION} ]; then
          BLISS_VERSION=${DETECTED_VERSION}
        fi
      done < /tmp/bliss-v-check.txt
    fi
    rm /tmp/bliss-v-check.txt
    sed -r -i "s/^version=\"[0-9]{8}/version=\"${BLISS_VERSION}/" /var/packages/Bliss/INFO
    
    #update the CPU-specific repository customizations (in case of an in-app update)
    if [ ${SYNO_CPU_ARCH} == "armv5tel" ]; then
      sed -i "s/jnotify\.linux\.x86/jnotify\.linux\.ARM_le/g" ${SYNOPKG_PKGDEST}/bliss-bundle/repository.xml
      sed -i "s/policy\.tag\.auto\.linux\.x86/policy\.tag\.auto\.linux\.ARM_le/g" ${SYNOPKG_PKGDEST}/bliss-bundle/repository.xml
    fi
    if [ ${SYNO_CPU_ARCH} == "ppc" ]; then
      sed -i "s/jnotify\.linux\.x86/jnotify\.linux\.PowerPC/g" ${SYNOPKG_PKGDEST}/bliss-bundle/repository.xml
      sed -i "s/policy\.tag\.auto\.linux\.x86/policy\.tag\.auto\.linux\.PowerPC/g" ${SYNOPKG_PKGDEST}/bliss-bundle/repository.xml
    fi
    
    #overwrite native lib bundles with syno versions (in case of an in-app update)
    cp ${SYNOPKG_PKGDEST}/syno-native/* ${SYNOPKG_PKGDEST}/bliss-bundle

    #start daemon in background mode
    echo "Starting ${SYNOPKG_PKGNAME} ... "
    su - ${DAEMON_USER} -s /bin/sh -c "cd ${SYNOPKG_PKGDEST}/bin && ./${ENGINE_SCRIPT} > ${SYNOPKG_PKGDEST}/bliss.out 2>&1 &"
    sleep 2
    PID=`_findpid`
    if [ -n "$PID" ]; then
      echo $PID > $PIDFILE
      echo "OK"
    else
      echo "FAIL"
      exit 1
    fi

    #set up symlink for the DSM GUI
    if [ -d /usr/syno/synoman/webman/3rdparty ]; then
      ln -s ${SYNOPKG_PKGDEST}/DSM/${SYNOPKG_PKGNAME} /usr/syno/synoman/webman/3rdparty/${SYNOPKG_PKGNAME}
    fi
   
    exit 0
  ;;

  stop)
    echo -n "Stopping daemon ... "
    if [ -f $PIDFILE ] ; then
      kill `cat $PIDFILE`
      sleep 10
    fi
    PID=`_findpid`
    if [ -n "$PID" ]; then
      echo "Still running, killing PID=$PID ... "
      kill -9 $PID
    fi
    rm -f $PIDFILE
    echo "OK"
    
    #remove DSM icon symlink
    if [ -e /usr/syno/synoman/webman/3rdparty/${SYNOPKG_PKGNAME} ]; then
      rm /usr/syno/synoman/webman/3rdparty/${SYNOPKG_PKGNAME}
    fi

    exit 0
  ;;

  status)
    PID=`_findpid`
    if [ -n "$PID" ]; then
      exit 0
    else
      exit 1
    fi
  ;;

  log)
    echo "${SYNOPKG_PKGDEST}/bliss.out"
    exit 0
  ;;
esac
 

Changelog:

  • 20130213-0009 Updated to Bliss 20130213, and will correctly report version in Package Center after an in-app update
  • 20130131-0008 Updated to Bliss 20130131
  • 20121112-0007 Fixes for DSM 4.2
  • 20121112-006 Updated to Bliss 20121112
  • 20121019-005 Updated to Bliss 20121019
  • 20121002-004 Updated to Bliss 20121002
  • 20120830-003 Added support for Freescale QorIQ PowerPC CPUs used in some Synology x13 series products, PowerPC processors in previous Synology generations with older glibc versions are not supported
  • 20120830-002 Hopefully fixed Java prefs polling issue that prevented NAS hibernation
  • 20120830-001 initial public release

 
 

CrashPlan PROe Server package for Synology NAS

CrashPlan is a popular online backup solution, with most people using it to protect their data in the Cloud. However, by licensing CrashPlan PROe server you can be that Cloud and act as the storage provider for other client machines running CrashPlan PROe.

I was recently contracted to implement this on Synology hardware for North Bay Technologies, an IT services company in San Francisco. Instead of undertaking a careful manual install that would be difficult to maintain in future, I decided to go one better – to build a package which integrates properly with Synology DSM. I then back-merged most of the changes into the existing CrashPlan client package scripts so everything is as consistent as possible. It was agreed that I would also publish this to the community, so here it is!

CrashPlan-PROe-dashboard

I should stress at this point that although this package could technically install on Synology products with ARM or QorIQ processors, you should only use this on a model with an Intel CPU. Ideally you should equip it with more than 1GB of RAM too, because the application requires 1GB all for itself. The package repo will not advertise it to ARM systems, because they have far too little available RAM.

As with the CrashPlan client packages I have made, I have been careful to comply with Code 42 Software’s stipulation that no one redistributes their work.

CrashPlan-PROe-server-info

 

Installation

  • This package is for Intel CPUs only. It will work on an unmodified NAS, no hacking or bootstrapping required.
  • More than 1GB of RAM is recommended.
  • You will need to install my Java SE for Embedded package first. Read the instructions on that page carefully too.
  • In the User Control Panel in DSM, enable the User Homes service.
  • Purchase your Master License Key and licences (or obtain a trial key) from crashplan.com, download the PROe Server installer for Linux, and save in the public shared folder on your NAS. You should have created this folder when you installed the Java package.
  • Install the CrashPlan PROe server Synology package directly from Package Center in DSM. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk.
 

Notes

  • The package expects the end user to have separately downloaded the CrashPlan PROe Server installer for Linux, and it presents them with the official EULA during installation.
  • For details of TCP ports used, to help you set up firewalling and/or port forwarding please consult the requirements document.
  • Once running, CrashPlan PROe server is configured by a web dashboard on https://yourNasIP:4285/console/ This link can be found in the package info screen in Package Center in Synology DSM when the package is running.
  • Full support documentation is available here.
  • The DSM Package Center upgrade functionality allows you to move between my package versions without losing settings or data, but if you’re moving to a new CrashPlan PROe Server version you will need to do that manually via the admin web app, using the Linux downloads (with file extension .upgrade) available from Code 42 Software. Depending on how up to date your version is, you may need to update incrementally through several versions. Before you apply a Code 42 update package, you should manually install the latest Synology package for the specific PROe Server version you’re currently runnning. This will ensure the update scripts are handled correctly. So for example if you’re upgrading from 3.5.4 to 3.5.5 you should manually install cpproeserver3.5.4-merged-020.spk over the top first. You can find up to date package versions for each PROe Server build here:
    cpproeserver3.2.1.2-merged-0020.spk
    cpproeserver3.3.0.2-merged-0020.spk
    cpproeserver3.3.0.3-merged-0020.spk
    cpproeserver3.3.0.4-merged-0020.spk
    cpproeserver3.4.1-merged-0020.spk
    cpproeserver3.4.1.5-merged-0020.spk
    cpproeserver3.5.1.1-merged-0020.spk
    cpproeserver3.5.3.2-merged-0020.spk
    cpproeserver3.5.4-merged-0020.spk
    cpproeserver3.5.5-merged-0020.spk
  • The engine daemon script checks the amount of system RAM and scales the Java heap size appropriately (up to the default maximum of 1024MB). This can be overridden in a persistent way if you are backing up very large backup sets by editing /volume1/@appstore/cpproeserver/syno_package.vars.
  • As with my other syno packages, this user account password is randomized when it is created using a perl script called passgen (I followed the example of the Transmission package). DSM Package Center runs as the root user so my script starts the package using an su command. This means that you can change the password yourself and CrashPlan will still work.
  • The default location for saving backup data is set to /volume1/cpproeserver (where /volume1 is you primary storage volume) to eliminate the chance of them being destroyed accidentally by uninstalling the package.
  • The package supports upgrading to future versions while preserving the machine identity, logs, login details, and cache.
  • The log which is displayed in the package’s Log tab is actually the activity history. If you’re trying to troubleshoot an issue you may need to use an SSH session to inspect the more detailed log files which are stored in /volume1/cpproeserver/log
  • I’m not sure if it works for the PROe products, but I would really appreciate it if you could use this affiliate link when purchasing your licences (you may need to browse to the PROe section of the website using the links in the footer of that page). If this package saves you the several days worth of work I put into making it, please also consider donating using the PayPal button on the right hand side of the page. Thanks!
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology Package wiki.

installer.sh

#!/bin/sh

#--------CRASHPLAN PROe server installer script
#--------package maintained at pcloadletter.co.uk

DOWNLOAD_PATH="http://download.crashplan.com/installs/proserver/CP_VER"
DOWNLOAD_FILE="CrashPlanPROServer_CP_VER_Linux.tgz"
DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
TGZ_FILE="CrashPlanPROServer.tgz"
#remove file extension
DOWNLOAD_FILE="`echo ${DOWNLOAD_FILE} | sed -e 's/.tgz$//'`"
EXTRACTED_FOLDER="${DOWNLOAD_FILE}"
DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_PASS="`openssl rand 12 -base64 2>/dev/null`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
DAEMON_HOME="/var/services/homes/${DAEMON_USER}"
OPTDIR="${SYNOPKG_PKGDEST}/proserver/server"
VARS_FILE="${OPTDIR}/.install.vars"
ENGINE_SCRIPT="proserver"
SYNO_CPU_ARCH="`uname -m`"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="i686"
NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/crashplan-native-${SYNO_CPU_ARCH}.tgz"   
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
INSTALL_FILES="${NATIVE_BINS_URL}"
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
#this is where the user data will go, so it persists after a package uninstall
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/${DAEMON_USER}"
VARLOGDIR=${MANIFEST_FOLDER}/log
LOG_FILE="${VARLOGDIR}/com_backup42_app.log.0"
UPGRADE_FILES=""
UPGRADE_FOLDERS="activemq-data conf db keys"

source /etc/profile
PUBLIC_FOLDER="`cat /usr/syno/etc/smb.conf | sed -r '/\/public$/!d;s/^.*path=(\/volume[0-9]{1,4}\/public).*$/\1/'`"

  
preinst ()
{
  if [ -z ${PUBLIC_FOLDER} ]; then
    echo "A shared folder called 'public' could not be found - note this name is case-sensitive. "
    echo "Please create this using the Shared Folder DSM Control Panel and try again."
    exit 1
  fi
  
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  #is the User Home service enabled?
  UH_SERVICE=maybe
  synouser --add userhometest Testing123 "User Home test user" 0 "" ""
  UHT_HOMEDIR=`cat /etc/passwd | sed -r '/User Home test user/!d;s/^.*:User Home test user:(.*):.*$/\1/'`
  if echo $UHT_HOMEDIR | grep '/var/services/homes/' > /dev/null; then
    if [ ! -d $UHT_HOMEDIR ]; then
      UH_SERVICE=false
    fi
  fi
  synouser --del userhometest
  #remove home directory (needed since DSM 4.1)
  [ -e /var/services/homes/userhometest ] && rm -r /var/services/homes/userhometest
  if [ "${UH_SERVICE}" == "false" ]; then
    echo "The User Home service is not enabled. Please enable this feature in the User control panel in DSM."
    exit 1
  fi
  
  CP_BINARY_FOUND=
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tgz ] && CP_BINARY_FOUND=true
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tar ] && CP_BINARY_FOUND=true
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tar.tar ] && CP_BINARY_FOUND=true
  [ -f ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.gz ] && CP_BINARY_FOUND=true
  
  if [ -z ${CP_BINARY_FOUND} ]; then
    echo "CrashPlan PROe server Linux installer not found. "
    echo "I was expecting the file ${PUBLIC_FOLDER}/${DOWNLOAD_FILE}.tgz "
    echo "Please visit crashplan.com, download the installer from ${DOWNLOAD_URL} "
    echo "and place it in the 'public' shared folder on your NAS."
    exit 1
  fi

  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, "
        echo "which was \"${WGET_URL}\" "
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. "
        exit 1
      fi
    fi
  done
  
  exit 0
}


postinst ()
{
  VAROPTDIR=${MANIFEST_FOLDER}/data
  VARLOGDIR=${MANIFEST_FOLDER}/log
  ETCDIR=${OPTDIR}/bin
  INITDIR=${OPTDIR}/bin
  RUNLVLDIR=${OPTDIR}/bin
  
  #create daemon user
  synouser --add ${DAEMON_USER} ${DAEMON_PASS} "${DAEMON_ID}" 0 "" ""
  
  #save the daemon user's homedir as variable in that user's profile
  #this is needed because new users seem to inherit a HOME value of /root which they have no permissions for.
  su - ${DAEMON_USER} -s /bin/sh -c "echo export HOME=\'${DAEMON_HOME}\' >> .profile"

  #extract CPU-specific additional binaries
  mkdir ${SYNOPKG_PKGDEST}/bin
  cd ${SYNOPKG_PKGDEST}/bin
  tar xzf ${TEMP_FOLDER}/${NATIVE_BINS_FILE} && rm ${TEMP_FOLDER}/${NATIVE_BINS_FILE}

  mkdir -p ${OPTDIR}
  mkdir -p ${INITDIR}
  mkdir -p ${RUNLVLDIR}
  mkdir -p ${VAROPTDIR}
  mkdir -p ${VARLOGDIR}
  cd ${PUBLIC_FOLDER}
  
  #extract CrashPlan Linux installer (Web browsers love to interfere with .tar.gz files)
  if [ -f ${DOWNLOAD_FILE}.tgz ]; then
    #Firefox seems to be the only browser that leaves it alone
    tar xzf ${DOWNLOAD_FILE}.tgz
  elif [ -f ${DOWNLOAD_FILE}.gz ]; then
    #Chrome
    tar xzf ${DOWNLOAD_FILE}.gz
  elif [ -f ${DOWNLOAD_FILE}.tar ]; then
    #Safari
    tar xf ${DOWNLOAD_FILE}.tar
  elif [ -f ${DOWNLOAD_FILE}.tar.tar ]; then
    #Internet Explorer
    tar xzf ${DOWNLOAD_FILE}.tar.tar
  fi
  
  mkdir -p ${OPTDIR}/content-custom
  mkdir -p ${OPTDIR}/installs
  mkdir ${VAROPTDIR}/backupArchives
  mkdir ${VAROPTDIR}/backupCache
  mkdir ${VAROPTDIR}/dumps
  chown -R ${DAEMON_USER} ${MANIFEST_FOLDER}
  
  #extract nested tgz archive
  cd ${OPTDIR}
  tar xozf "${PUBLIC_FOLDER}/${EXTRACTED_FOLDER}/${TGZ_FILE}"
  
  echo "#uncomment to expand Java max heap size beyond prescribed value of 1024M (will survive upgrades)" > ${SYNOPKG_PKGDEST}/syno_package.vars
  echo "#USR_MAX_HEAP=1024M" >> ${SYNOPKG_PKGDEST}/syno_package.vars
  echo >> ${SYNOPKG_PKGDEST}/syno_package.vars
  
  #create a valid identity file if there is no existing GUID
  GUID=
  if [ -f ${VAROPTDIR}/.identity ] ; then
    . ${VAROPTDIR}/.identity
  fi
  if [ "x${GUID}" == "x" ]; then
    echo -n "GUID=" > ${VAROPTDIR}/.identity
    java -cp "${OPTDIR}/lib/com.backup42.app.jar" com.code42.utils.UniqueId >> ${VAROPTDIR}/.identity
    . ${VAROPTDIR}/.identity
    if [ "x${GUID}" == "x" ] ; then
      echo "Failed to create valid server identity. Identity Path: ${VAROPTDIR}/.identity"
      exit 1
    fi
  fi
 
  #amend entries in default server config file
  sed -i "s%<OPT>%${OPTDIR}%" ${OPTDIR}/conf/conf_proe.properties
  sed -i "s%<VAROPT>%${VAROPTDIR}%" ${OPTDIR}/conf/conf_proe.properties
  sed -i "s%<VARLOGDIR>%${VARLOGDIR}%" ${OPTDIR}/conf/conf_proe.properties
  
  #save install variables which Crashplan expects its own installer script to create
  echo "" > ${VARS_FILE}
  echo "OPTDIR=${OPTDIR}" >> ${VARS_FILE}
  echo "VAROPTDIR=${VAROPTDIR}" >> ${VARS_FILE}
  echo "VARLOGDIR=${VARLOGDIR}" >> ${VARS_FILE}
  echo "ETCDIR=${ETCDIR}" >> ${VARS_FILE}
  echo "INITDIR=${INITDIR}" >> ${VARS_FILE}
  echo "RUNLVLD=${RUNLVLDIR}" >> ${VARS_FILE}
  echo "INSTALLDATE=`date +%Y%m%d`" >> ${VARS_FILE}
  echo "JAVACOMMON=\${JAVA_HOME}/bin/java" >> ${VARS_FILE}

  #remove temp files
  rm -r ${PUBLIC_FOLDER}/${EXTRACTED_FOLDER}
  
  #change owner of CrashPlan folder tree
  chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}

  echo "CrashPlan PROe Server has been installed. When you start the package a few moments of first-time initialization "
  echo "are needed before the management application will be available in your web browser. You can check the Log tab "
  echo "to discover when it has fully started. "
  echo "http://localhost:4280/console "
  echo "https://localhost:4285/console "
  echo "Please note that your clients will communicate with the server on TCP port 4282."
  
  exit 0
}


preuninst ()
{
  #make sure engine is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} stop"
  sleep 2
  
  exit 0
}


postuninst ()
{
  if [ -f ${SYNOPKG_PKGDEST}/syno_package.vars ]; then
    source ${SYNOPKG_PKGDEST}/syno_package.vars
  fi
  
  if [ "${LIBFFI_SYMLINK}" == "YES" ]; then
    rm /lib/libffi.so.5
  fi

  #if it doesn't exist, but is still a link then it's a broken link and should also be deleted
  if [ ! -e /lib/libffi.so.5 ]; then
    [ -L /lib/libffi.so.5 ] && rm /lib/libffi.so.5
  fi

  #remove daemon user
  synouser --del ${DAEMON_USER}
  
  #remove daemon user's home directory (needed since DSM 4.1)
  [ -e /var/services/homes/${DAEMON_USER} ] && rm -r /var/services/homes/${DAEMON_USER}
  
  exit 0
}

preupgrade ()
{
  #make sure engine is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} stop"
  sleep 2
  
  #if config data exists back it up
  if [ -d ${OPTDIR}/keys ]; then
    mkdir -p ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
      if [ -d ${OPTDIR}/${FOLDER_TO_MIGRATE} ]; then
        mv ${OPTDIR}/${FOLDER_TO_MIGRATE} ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig
      fi
    done
  fi

  exit 0
}


postupgrade ()
{
  #use the migrated config data from the previous version
  if [ -d ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig/keys ]; then
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
    if [ -d ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig/${FOLDER_TO_MIGRATE} ]; then
      cp -R ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig/${FOLDER_TO_MIGRATE} ${OPTDIR}
    fi
    done
    rmdir ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_data_mig

    #make log entry
    TIMESTAMP="`date +%x` `date +%I:%M%p`"
    echo "I ${TIMESTAMP} Synology Package Center updated ${SYNOPKG_PKGNAME} to version ${SYNOPKG_PKGVER}" >> ${LOG_FILE}
    
    #daemon user has been deleted and recreated so we need to reset ownership (new UID)
    chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
  fi
  
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------CRASHPLAN PROe server start-stop-status script
#--------package maintained at pcloadletter.co.uk

DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_HOME="/var/services/homes/${DAEMON_USER}"
OPTDIR="${SYNOPKG_PKGDEST}/proserver/server"
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/${DAEMON_USER}"
VARLOGDIR=${MANIFEST_FOLDER}/log
LOG_FILE="${VARLOGDIR}/history.log.0"
ENGINE_SCRIPT="proserver"
APP_NAME="CPServer"
SCRIPTS_TO_EDIT="${ENGINE_SCRIPT} proservermonitor"
ENGINE_CFG="${ENGINE_SCRIPT}"
LIBFFI_SO_NAMES="5 6" #armada370 build of libjnidispatch.so is newer, and uses libffi.so.6
CFG_PARAM="JAVA_MEM_ARGS"
#note that the vars in the next two string values are escaped for evaluation later on
JAVA_MEM_ARGS="-Xss128k -Xms\${JAVA_MIN_HEAP}m -Xmx\${JAVA_MAX_HEAP}m"
JAVA_GC_ARGS="-XX:+DisableExplicitGC -XX:+UseAdaptiveGCBoundary -XX:PermSize=\${JAVA_MIN_HEAP}m -XX:MaxPermSize=\${JAVA_MIN_HEAP}m"
source ${OPTDIR}/.install.vars

JAVA_MIN_HEAP=`grep "^${CFG_PARAM}=" "${OPTDIR}/bin/${ENGINE_CFG}" | sed -r "s/^.*-Xms([0-9]+)[Mm] .*$/\1/"` 
SYNO_CPU_ARCH="`uname -m`"

case $1 in
  start)
    #set the current timezone for Java so that log timestamps are accurate
    #we need to use the modern timezone names so that Java can figure out DST 
    SYNO_TZ=`cat /etc/synoinfo.conf | grep timezone | cut -f2 -d'"'`
    SYNO_TZ=`grep "^${SYNO_TZ}" /usr/share/zoneinfo/Timezone/tzname | sed -e "s/^.*= //"`
    grep "^export TZ" ${DAEMON_HOME}/.profile > /dev/null \
     && sed -i "s%^export TZ=.*$%export TZ='${SYNO_TZ}'%" ${DAEMON_HOME}/.profile \
     || echo export TZ=\'${SYNO_TZ}\' >> ${DAEMON_HOME}/.profile
    #check persistent variables from syno_package.vars
    USR_MAX_HEAP=0
    if [ -f ${SYNOPKG_PKGDEST}/syno_package.vars ]; then
      source ${SYNOPKG_PKGDEST}/syno_package.vars
    fi
    USR_MAX_HEAP=`echo $USR_MAX_HEAP | sed -e "s/[mM]//"`

    #create or repair libffi symlink if a DSM upgrade has removed it
    for FFI_VER in ${LIBFFI_SO_NAMES}; do 
      if [ -e ${OPTDIR}/lib/libffi.so.${FFI_VER} ]; then
        if [ ! -e /lib/libffi.so.${FFI_VER} ]; then
          #if it doesn't exist, but is still a link then it's a broken link and should be deleted
          [ -L /lib/libffi.so.${FFI_VER} ] && rm /lib/libffi.so.${FFI_VER}
          ln -s ${OPTDIR}/lib/libffi.so.${FFI_VER} /lib/libffi.so.${FFI_VER}
        fi
      fi
    done

    #fix up some of the binary paths and fix some command syntax for busybox 
    #moved this to start-stop-status from installer.sh because Code42 push updates and these
    #new scripts will need this treatment too
    FIND_TARGETS=
    for TARGET in ${SCRIPTS_TO_EDIT}; do
      FIND_TARGETS="${FIND_TARGETS} -o -name ${TARGET}"
    done
    find ${OPTDIR} \( -name \*.sh ${FIND_TARGETS} \) | while IFS="" read -r FILE_TO_EDIT; do
      if [ -e ${FILE_TO_EDIT} ]; then
        #this list of substitutions will probably need expanding as new CrashPlan updates are released
        sed -i "s%^#!/bin/bash%#!${SYNOPKG_PKGDEST}/bin/bash%" "${FILE_TO_EDIT}"
        sed -i -r "s%(^\s*)nice -n%\1${SYNOPKG_PKGDEST}/bin/nice -n%" "${FILE_TO_EDIT}"
        sed -i -r "s%(^\s*)(/bin/ps|ps) [^\|]*\|%\1/bin/ps w \|%" "${FILE_TO_EDIT}"
        sed -i -r "s%\`ps [^\|]*\|%\`ps w \|%" "${FILE_TO_EDIT}"
        sed -i "s/rm -fv/rm -f/" "${FILE_TO_EDIT}"
        sed -i "s/mv -fv/mv -f/" "${FILE_TO_EDIT}"
      fi
    done

    #an upgrade script that has been launched via the web app will usually have failed until the above
    #changes are made so we need to find it and start it, if it exists
    UPGRADE_SCRIPT=`find ${OPTDIR}/upgrade -name "upgrade.sh"`
    if [ -n "${UPGRADE_SCRIPT}" ]; then
      rm ${OPTDIR}/${ENGINE_SCRIPT}.pid
      SCRIPT_HOME=`dirname $UPGRADE_SCRIPT`

      #make CrashPlan log entry
      TIMESTAMP="`date +%x` `date +%I:%M%p`"
      echo "I ${TIMESTAMP} Synology repairing upgrade in ${SCRIPT_HOME}" >> ${LOG_FILE}

      mv ${SCRIPT_HOME}/upgrade.log ${SCRIPT_HOME}/upgrade.log.old
      chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
      su - ${DAEMON_USER} -s /bin/sh -c "cd ${SCRIPT_HOME} ; . upgrade.sh"
      mv ${SCRIPT_HOME}/upgrade.sh ${SCRIPT_HOME}/upgrade.sh.old
      exit 0
    fi

    #updates may also overwrite our native binaries
    if [ "${SYNO_CPU_ARCH}" != "x86_64" ]; then
      cp -f ${SYNOPKG_PKGDEST}/bin/libjtux.so ${OPTDIR}
      cp -f ${SYNOPKG_PKGDEST}/bin/jna-3.2.5.jar ${OPTDIR}/lib
      cp -f ${SYNOPKG_PKGDEST}/bin/libffi.so.* ${OPTDIR}/lib
    fi

    #set appropriate Java max heap size
    RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
    if [ $RAM -le 128 ]; then
      JAVA_MAX_HEAP=80
    elif [ $RAM -le 256 ]; then
      JAVA_MAX_HEAP=192
    elif [ $RAM -le 512 ]; then
      JAVA_MAX_HEAP=384
    elif [ $RAM -le 1024 ]; then
      JAVA_MAX_HEAP=896
    #CrashPlan PROe server's default max heap is 1GB
    elif [ $RAM -gt 1024 ]; then
      JAVA_MAX_HEAP=1024
    fi
    if [ $USR_MAX_HEAP -gt $JAVA_MAX_HEAP ]; then
      JAVA_MAX_HEAP=${USR_MAX_HEAP}
    fi
    if [ $JAVA_MAX_HEAP -lt $JAVA_MIN_HEAP ]; then
      #can't have a max heap lower than min heap (ARM low RAM systems)
      JAVA_MIN_HEAP=${JAVA_MAX_HEAP}
    fi

    #reset ownership of all files to daemon user, so that manual edits to config files won't cause problems
    chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
    chown -R ${DAEMON_USER} ${DAEMON_HOME}    
    
    #CrashPlan PROe server will read customized vars from a separate file
    eval echo "JAVA_GC_ARGS='\"'${JAVA_GC_ARGS}'\"'" > ${OPTDIR}/.proserverrc
    eval echo "JAVA_MEM_ARGS='\"'${JAVA_MEM_ARGS}'\"'" >> ${OPTDIR}/.proserverrc
    su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} start"
    exit 0
  ;;

  stop)
    su - ${DAEMON_USER} -s /bin/sh -c "${OPTDIR}/bin/${ENGINE_SCRIPT} stop"
    exit 0
  ;;

  status)
    PID=`/bin/ps w| grep "app=${APP_NAME}" | grep -v grep | awk '{ print $1 }'`
    if [[ -n "$PID" ]]; then
      exit 0
    else
      exit 1
    fi
  ;;

  log)
    echo "${LOG_FILE}"
    exit 0
  ;;
esac
 

Changelog:

  • 020 added support for Intel Atom Evansport CPU in some new DSx14 products
  • 019 update to CrashPlan PROe Server 3.5.5, improved update handling
  • 018 update to CrashPlan PROe Server 3.4.1.5, improved update handling, fixes for DSM 4.2
  • 017 update to CrashPlan PROe Server 3.4.1, improved update handling
  • 016 update to CrashPlan PROe Server 3.3.0.4
  • 015 further fixes for the update mechanism
  • 014 created a wrapper for the ps command, and a symlink for /bin/bash which should hopefully allow server upgrade scripts from Code 42 to run
  • 013 fixed a timezone detection bug
  • 012 fixed a bug with the script editing logic which amends Code 42′s scripts to work with busybox shell tools
  • 011 updated to CrashPlan PROe Server 3.3.0.3
  • 010 intial public release
 
 

CrashPlan packages for Synology NAS

UPDATE – The instructions and notes on this page apply to all three versions of the package hosted on my repo: CrashPlan, CrashPlan PRO, and CrashPlan PROe.

CrashPlan is a popular online backup solution which supports continuous syncing. With this your NAS can become even more resilient – it could even get stolen or destroyed and you would still have your data. Whilst you can pay a small monthly charge for a storage allocation in the Cloud, one neat feature CrashPlan offers is for individuals to collaboratively backup their important data to each other – for free! You could install CrashPlan on your laptop and have it continuously protecting your documents to your NAS, even whilst away from home.

CrashPlan-Windows

CrashPlan is a Java application, and one that’s typically difficult to install on a NAS – therefore an obvious candidate for me to simplify into a package, given that I’ve made a few others. I tried and failed a few months ago, getting stuck at compiling the Jtux library for ARM CPUs (the Oracle Java for Embedded doesn’t come with any headers).

I noticed a few CrashPlan setup guides linking to my Java package, and decided to try again based on these: Kenneth Larsen’s blog post, the Vincesoft blog article for installing on ARM processor Iomega NAS units, and this handy PDF document which is a digest of all of them, complete with download links for the additional compiled ARM libraries. I used the PowerPC binaries Christophe had compiled on his chreggy.fr blog, so thanks go to him. I wanted make sure the package didn’t require the NAS to be bootstrapped, so I picked out the few generic binaries that were needed (bash, nice and cpio) directly from the Optware repo.

UPDATE – For version 3.2 I also had to identify and then figure out how to compile Tim Macinta’s fast MD5 library, to fix the supplied libmd5.so on ARM systems (CrashPlan only distributes libraries for x86). I’m documenting that process here in case more libs are required in future versions. I identified it from the error message in log/engine_error.log and by running objdump -x libmd5.so. I could see that the same Java_com_twmacinta_util_MD5_Transform_1native function mentioned in the error was present in the x86 lib but not in my compiled libmd5.so from W3C Libwww. I took the headers from an install of OpenJDK on a regular Ubuntu desktop. I then used the Linux x86 source from the download bundle on Tim’s website – the closest match – and compiled it directly on the syno using the command line from a comment in another version of that source:
gcc -O3 -shared -I/tmp/jdk_headers/include /tmp/fast-md5/src/lib/arch/linux_x86/MD5.c -o libmd5.so

Aside from the challenges of getting the library dependencies fixed for ARM and QorIQ PowerPC systems, there was also the matter of compliance – Code 42 Software’s EULA prohibits redistribution of their work. I had to make the syno package download CrashPlan for Linux (after the end user agrees their EULA), then I had to write my own script to extract this archive and mimic their installer, since their installer is interactive. It took a lot of slow testing, but I managed it!

CPPROe package info

My most recent package version introduces handling of the automatic updates which Code 42 sometimes publish to the clients. This has proved to be quite a challenge to get working as testing was very laborious. I can confirm that it worked with the update from CrashPlan PRO 3.2 to 3.2.1 , and from CrashPlan 3.2.1 to 3.4.1:

CrashPlan-update-repair

 

Installation

  • This package is for Marvell Kirkwood, Marvell Armada 370/XP, Intel and Freescale QorIQ/PowerQUICC PowerPC CPUs only, so please check which CPU your NAS has. It will work on an unmodified NAS, no hacking or bootstrapping required. It will only work on older PowerQUICC PowerPC models that are running DSM 5.0. It is technically possible to run CrashPlan on older DSM versions, but it requires chroot-ing to a Debian install. Christophe from chreggy.fr has recently released packages to automate this.
  • In the User Control Panel in DSM, enable the User Homes service.
  • Install the package directly from Package Center in DSM. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk.
  • You will need to install either one of my Java SE Embedded packages first (Java 6 or 7). Read the instructions on that page carefully too.
  • If you previously installed CrashPlan manually using the Synology Wiki, you can find uninstall instructions here.
 

Notes

  • The package downloads the CrashPlan installer directly from Code 42 Software, following acceptance of their EULA. I am complying with their wish that no one redistributes it.
  • CrashPlan is installed in headless mode – backup engine only. This is configured by a desktop client, but operates independently of it.
  • The engine daemon script checks the amount of system RAM and scales the Java heap size appropriately (up to the default maximum of 512MB). This can be overridden in a persistent way if you are backing up very large backup sets by editing /volume1/@appstore/CrashPlan/syno_package.vars. If you’re considering buying a NAS purely to use CrashPlan and intend to back up more than a few hundred GB then I strongly advise buying one of the Intel models which come with 1GB RAM and can be upgraded to 3GB very cheaply. RAM is very limited on the ARM ones. 128MB RAM on the J series means CrashPlan is running with only one fifth of the recommended heap size, so I doubt it’s viable for backing up very much at all. My DS111 has 256MB of RAM and currently backs up around 60GB with no issues. I have found that a 512MB heap was insufficient to back up more than 2TB of files on a Windows server. It kept restarting the backup engine every few minutes until I increased the heap to 1024MB.
  • As with my other syno packages, the daemon user account password is randomized when it is created using the openssl binary. DSM Package Center runs as the root user so my script starts the package using an su command. This means that you can change the password yourself and CrashPlan will still work.
  • The default location for saving friends’ backups is set to /volume1/crashplan/backupArchives (where /volume1 is you primary storage volume) to eliminate the chance of them being destroyed accidentally by uninstalling the package.
  • The first time you run the server you will need to stop it and restart it before you can connect the client. This is because a config file that’s only created on first run needs to be edited by one of my scripts. The engine is then configured to listen on all interfaces on the default port 4243.
  • Once the engine is running, you can manage it by installing CrashPlan on another computer, and editing the file conf/ui.properties on that computer so that this line:
    #serviceHost=127.0.0.1
    is uncommented (by removing the hash symbol) and set to the IP address of your NAS, e.g.:
    serviceHost=192.168.1.210
    On Windows you can also disable the CrashPlan service if you will only use the client.
  • If you need to manage CrashPlan from a remote location, I suggest you do so using SSH tunnelling as per this support document.
  • The package supports upgrading to future versions while preserving the machine identity, logs, login details, and cache. Upgrades can now take place without requiring a login from the client afterwards.
  • If you remove the package completely and re-install it later, you can re-attach to previous backups. When you log in to the Desktop Client with your existing account after a re-install, you can select “adopt computer” to merge the records, and preserve your existing backups. I haven’t tested whether this also re-attaches links to friends’ CrashPlan computers and backup sets, though the latter does seem possible in the Friends section of the GUI. It’s probably a good idea to test that this survives a package reinstall before you start relying on it. Sometimes, particularly with CrashPlan PRO I think, the adopt option is not offered. In this case you can log into CrashPlan Central and retrieve your computer’s GUID. On the CrashPlan client, double-click on the logo in the top right and you’ll enter a command line mode. You can use the GUID command to change the system’s GUID to the one you just retrieved from your account.
  • The log which is displayed in the package’s Log tab is actually the activity history. If you’re trying to troubleshoot an issue you will need to use an SSH session to inspect the two engine log files which are:
    /volume1/@appstore/CrashPlan/log/engine_output.log
    /volume1/@appstore/CrashPlan/log/engine_error.log
  • When CrashPlan downloads and attempts to run an automatic update, the script will most likely fail and stop the package. This is typically caused by syntax differences with the Synology versions of certain Linux shell commands (like rm, mv, or ps). You will need to wait several minutes in the event of this happening before you take action, because the update script tries to restart CrashPlan 10 times at 10 second intervals. After this, you simply start the package again in Package Center and my scripts will fix the update, then run it. One final package restart is required before you can connect with the CrashPlan Desktop client (remember to update that too).
  • After their backup is seeded some users may wish to schedule the CrashPlan engine using cron so that it only runs at certain times. This is particularly useful on ARM systems because CrashPlan currently prevents hibernation while it is running (unresolved issue, reported to Code 42). To schedule, edit /etc/crontab and add the following entries for starting and stopping CrashPlan:
    55 2 * * * root /var/packages/CrashPlan/scripts/start-stop-status start
    0  4 * * * root /var/packages/CrashPlan/scripts/start-stop-status stop

    This example would configure CrashPlan to run daily between 02:55 and 04:00am. CrashPlan by default will scan the whole backup selection for changes at 3:00am so this is ideal. The simplest way to edit crontab if you’re not really confident with Linux is to install Merty’s Config File Editor package, which requires the official Synology Perl package to be installed too (since DSM 4.2). After editing crontab you will need to restart the cron daemon for the changes to take effect:
    /usr/syno/etc.defaults/rc.d/S04crond.sh stop
    /usr/syno/etc.defaults/rc.d/S04crond.sh start

    It is vitally important that you do not improvise your own startup commands or use a different account because this will most likely break the permissions on the config files, causing additional problems. The package scripts are designed to be run as root, and they will in turn invoke the CrashPlan engine using its own dedicated user account.
  • If you update DSM later, you will need to re-install the Java package or else UTF-8 and locale support will be broken by the update.
  • If you decide to sign up for one of CrashPlan’s paid backup services as a result of my work on this, I would really appreciate it if you could use this affiliate link, or consider donating using the PayPal button on the right.
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology Package wiki.

installer.sh

#!/bin/sh

#--------CRASHPLAN installer script
#--------package maintained at pcloadletter.co.uk


DOWNLOAD_PATH="http://download.crashplan.com/installs/linux/install/${SYNOPKG_PKGNAME}"
[ "${SYNOPKG_PKGNAME}" == "CrashPlan" ] && DOWNLOAD_FILE="CrashPlan_3.6.4_Linux.tgz"
[ "${SYNOPKG_PKGNAME}" == "CrashPlanPRO" ] && DOWNLOAD_FILE="CrashPlanPRO_3.6.4_Linux.tgz"
if [ "${SYNOPKG_PKGNAME}" == "CrashPlanPROe" ]; then
  [ "${WIZARD_VER_364}" == "true" ] && CPPROE_VER="3.6.4"
  [ "${WIZARD_VER_363}" == "true" ] && CPPROE_VER="3.6.3"
  [ "${WIZARD_VER_3614}" == "true" ] && CPPROE_VER="3.6.1.4"
  [ "${WIZARD_VER_353}" == "true" ] && CPPROE_VER="3.5.3"
  [ "${WIZARD_VER_341}" == "true" ] && CPPROE_VER="3.4.1"
  [ "${WIZARD_VER_33}" == "true" ] && CPPROE_VER="3.3"
  DOWNLOAD_FILE="CrashPlanPROe_${CPPROE_VER}_Linux.tgz"
fi
DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
CPI_FILE="${SYNOPKG_PKGNAME}_*.cpi"
EXTRACTED_FOLDER="${SYNOPKG_PKGNAME}-install"
OPTDIR="${SYNOPKG_PKGDEST}"
VARS_FILE="${OPTDIR}/install.vars"
SYNO_CPU_ARCH="`uname -m`"
[ "${SYNO_CPU_ARCH}" == "armv5tel" ] && SYNO_CPU_ARCH="armle"
[ "${SYNO_CPU_ARCH}" == "armv7l" ] && SYNO_CPU_ARCH="armle"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="i686"
cat /proc/cpuinfo | grep "Comcerto 2000" > /dev/null && SYNO_CPU_ARCH="armhf"
NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/crashplan-native-${SYNO_CPU_ARCH}.tar.xz"   
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
INSTALL_FILES="${DOWNLOAD_URL} ${NATIVE_BINS_URL}"
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
#the Manifest folder is where friends' backup data is stored
#we set it outside the app folder so it persists after a package uninstall
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/crashplan"
LOG_FILE="${SYNOPKG_PKGDEST}/log/history.log.0"
UPGRADE_FILES="syno_package.vars conf/my.service.xml conf/service.login conf/service.model"
UPGRADE_FOLDERS="log cache"
PUBLIC_FOLDER="`synoshare --get public | sed -r "/Path/!d;s/^.*\[(.*)\].*$/\1/"`"
source /etc/profile


preinst ()
{
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi

  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi

  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, "
        echo "which was \"${WGET_URL}\" "
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. "
        exit 1
      fi
    fi
  done
 
  exit 0
}


postinst ()
{
  #extract CPU-specific additional binaries
  mkdir ${SYNOPKG_PKGDEST}/bin
  cd ${SYNOPKG_PKGDEST}/bin
  tar xJf ${TEMP_FOLDER}/${NATIVE_BINS_FILE} && rm ${TEMP_FOLDER}/${NATIVE_BINS_FILE}

  #extract main archive
  cd ${TEMP_FOLDER}
  tar xzf ${TEMP_FOLDER}/${DOWNLOAD_FILE} && rm ${TEMP_FOLDER}/${DOWNLOAD_FILE} 
  
  #extract cpio archive
  cd ${SYNOPKG_PKGDEST}
  cat "${TEMP_FOLDER}/${EXTRACTED_FOLDER}"/${CPI_FILE} | gzip -d -c | ${SYNOPKG_PKGDEST}/bin/cpio -i --no-preserve-owner
  
  echo "#uncomment to expand Java max heap size beyond prescribed value (will survive upgrades)" > ${SYNOPKG_PKGDEST}/syno_package.vars
  echo "#you probably only want more than the recommended 1024M if you're backing up extremely large volumes of files" >> ${SYNOPKG_PKGDEST}/syno_package.vars
  echo "#USR_MAX_HEAP=1024M" >> ${SYNOPKG_PKGDEST}/syno_package.vars
  echo >> ${SYNOPKG_PKGDEST}/syno_package.vars

  cp ${TEMP_FOLDER}/${EXTRACTED_FOLDER}/scripts/CrashPlanEngine ${OPTDIR}/bin
  cp ${TEMP_FOLDER}/${EXTRACTED_FOLDER}/scripts/run.conf ${OPTDIR}/bin
  mkdir -p ${MANIFEST_FOLDER}/backupArchives    
  
  #save install variables which Crashplan expects its own installer script to create
  echo TARGETDIR=${SYNOPKG_PKGDEST} > ${VARS_FILE}
  echo BINSDIR=/bin >> ${VARS_FILE}
  echo MANIFESTDIR=${MANIFEST_FOLDER}/backupArchives >> ${VARS_FILE}
  #leave these ones out which should help upgrades from Code42 to work (based on examining an upgrade script)
  #echo INITDIR=/etc/init.d >> ${VARS_FILE}
  #echo RUNLVLDIR=/usr/syno/etc/rc.d >> ${VARS_FILE}
  echo INSTALLDATE=`date +%Y%m%d` >> ${VARS_FILE}
  echo JAVACOMMON=\${JAVA_HOME}/bin/java >> ${VARS_FILE}
  cat ${TEMP_FOLDER}/${EXTRACTED_FOLDER}/install.defaults >> ${VARS_FILE}
  
  #remove temp files
  rm -r ${TEMP_FOLDER}/${EXTRACTED_FOLDER}
  
  #add firewall config
  /usr/syno/bin/servicetool --install-configure-file --package /var/packages/${SYNOPKG_PKGNAME}/scripts/${SYNOPKG_PKGNAME}.sc > /dev/null
  
  #amend CrashPlanPROe client version
  [ "${SYNOPKG_PKGNAME}" == "CrashPlanPROe" ] && sed -i -r "s/^version=\".*(-.*$)/version=\"${CPPROE_VER}\1/" /var/packages/${SYNOPKG_PKGNAME}/INFO

  exit 0
}


preuninst ()
{
  #make sure engine is stopped
  /var/packages/${SYNOPKG_PKGNAME}/scripts/start-stop-status stop
  
  exit 0
}


postuninst ()
{
  if [ -f ${SYNOPKG_PKGDEST}/syno_package.vars ]; then
    source ${SYNOPKG_PKGDEST}/syno_package.vars
  fi

  if [ "${LIBFFI_SYMLINK}" == "YES" ]; then
    sed -i "/^LIBFFI_SYMLINK/d" ${SYNOPKG_PKGDEST}/syno_package.vars
  fi
  [ -e ${OPTDIR}/lib/libffi.so.* ] && rm ${OPTDIR}/lib/libffi.so.*
  
  #delete symlinks that no longer resolve
  if [ ! -e /lib/libffi.so.5 ]; then
    [ -L /lib/libffi.so.5 ] && rm /lib/libffi.so.5
  fi
  if [ ! -e /lib/libffi.so.6 ]; then
    [ -L /lib/libffi.so.6 ] && rm /lib/libffi.so.6
    #repair libffi.so.6 symlink on DSM 5.0 now that it's included by default
    ln -s libffi.so.6.0.1 /lib/libffi.so.6
  fi
    
  #remove firewall config
  if [ "${SYNOPKG_PKG_STATUS}" == "UNINSTALL" ]; then
    /usr/syno/bin/servicetool --remove-configure-file --package ${SYNOPKG_PKGNAME}.sc > /dev/null
  fi

  #remove legacy daemon user and homedir
  DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
  synouser --del ${DAEMON_USER}
  [ -e /var/services/homes/${DAEMON_USER} ] && rm -r /var/services/homes/${DAEMON_USER}
  
 exit 0
}


preupgrade ()
{
  #make sure engine is stopped
  /var/packages/${SYNOPKG_PKGNAME}/scripts/start-stop-status stop
  CONFIG_MIGRATION=

  #if identity from legacy package version exists migrate it
  DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
  DAEMON_HOME="/var/services/homes/${DAEMON_USER}"
  if [ -d ${DAEMON_HOME}/.crashplan ]; then
    mkdir -p ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/conf
    mkdir -p /var/lib/crashplan
    mv ${DAEMON_HOME}/.crashplan/.identity /var/lib/crashplan/
    chown -R root:root /var/lib/crashplan/
    CONFIG_MIGRATION="true"
  fi

  #if identity exists back up config
  if [ -f /var/lib/crashplan/.identity ]; then
    mkdir -p ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/conf
    CONFIG_MIGRATION="true"
  fi

  #if config data exists back it up  
  if [ "${CONFIG_MIGRATION}" == "true" ]; then
    for FILE_TO_MIGRATE in ${UPGRADE_FILES}; do
      if [ -f ${OPTDIR}/${FILE_TO_MIGRATE} ]; then
        cp ${OPTDIR}/${FILE_TO_MIGRATE} ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FILE_TO_MIGRATE}
      fi
    done
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
      if [ -d ${OPTDIR}/${FOLDER_TO_MIGRATE} ]; then
        mv ${OPTDIR}/${FOLDER_TO_MIGRATE} ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig
      fi
    done
  fi

  exit 0
}


postupgrade ()
{
  #use the migrated identity and config data from the previous version
  if [ -f ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/conf/my.service.xml ]; then
    for FILE_TO_MIGRATE in ${UPGRADE_FILES}; do
      if [ -f ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FILE_TO_MIGRATE} ]; then
        mv ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FILE_TO_MIGRATE} ${OPTDIR}/${FILE_TO_MIGRATE}
      fi
    done
    for FOLDER_TO_MIGRATE in ${UPGRADE_FOLDERS}; do
    if [ -d ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FOLDER_TO_MIGRATE} ]; then
      mv ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/${FOLDER_TO_MIGRATE} ${OPTDIR}
    fi
    done
    rmdir ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig/conf
    rmdir ${SYNOPKG_PKGDEST}/../${SYNOPKG_PKGNAME}_data_mig
    
    #make CrashPlan log entry
    TIMESTAMP="`date "+%D %I:%M%p"`"
    echo "I ${TIMESTAMP} Synology Package Center updated ${SYNOPKG_PKGNAME} to version ${SYNOPKG_PKGVER}" >> ${LOG_FILE}

    #fix permissions on migrated settings and manifest from legacy package versions which used a daemon user account
    chown -R root:root ${SYNOPKG_PKGDEST}
    if [ -f ${SYNOPKG_PKGDEST}/conf/my.service.xml ]; then
      MANIFEST_FOLDER=`cat ${SYNOPKG_PKGDEST}/conf/my.service.xml | grep "<manifestPath>" | cut -f2 -d'>' | cut -f1 -d'<'`
      chown -R root:root ${MANIFEST_FOLDER}
    fi    

    #tidy up some legacy settings
    sed -i "/^CRON_SYNOPKG_PKGNAME/d" ${SYNOPKG_PKGDEST}/syno_package.vars
    sed -i "/^CRON_SYNOPKG_PKGDEST/d" ${SYNOPKG_PKGDEST}/syno_package.vars
  fi
  
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------CRASHPLAN start-stop-status script
#--------package maintained at pcloadletter.co.uk


TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
MANIFEST_FOLDER="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/crashplan" 
ENGINE_CFG="run.conf"
PKG_FOLDER="`dirname $0 | cut -f1-4 -d'/'`"
DNAME="`dirname $0 | cut -f4 -d'/'`"
OPTDIR="${PKG_FOLDER}/target"
PID_FILE="${OPTDIR}/${DNAME}.pid"
DLOG="${OPTDIR}/log/history.log.0"
CFG_PARAM="SRV_JAVA_OPTS"
JAVA_MIN_HEAP=`grep "^${CFG_PARAM}=" "${OPTDIR}/bin/${ENGINE_CFG}" | sed -r "s/^.*-Xms([0-9]+)[Mm] .*$/\1/"` 
SYNO_CPU_ARCH="`uname -m`"
TIMESTAMP="`date "+%D %I:%M%p"`"
FULL_CP="${OPTDIR}/lib/com.backup42.desktop.jar:${OPTDIR}/lang"
source ${OPTDIR}/install.vars
source /etc/profile
source /root/.profile


start_daemon ()
{
  #check persistent variables from syno_package.vars
  USR_MAX_HEAP=0
  if [ -f ${OPTDIR}/syno_package.vars ]; then
    source ${OPTDIR}/syno_package.vars
  fi
  USR_MAX_HEAP=`echo $USR_MAX_HEAP | sed -e "s/[mM]//"`

  #do we need to restore the identity file - has a DSM upgrade scrubbed /var/lib/crashplan?
  if [ ! -e /var/lib/crashplan ]; then
    mkdir /var/lib/crashplan
    [ -e ${OPTDIR}/conf/var-backup/.identity ] && cp ${OPTDIR}/conf/var-backup/.identity /var/lib/crashplan/
  fi

  #fix up some of the binary paths and fix some command syntax for busybox 
  #moved this to start-stop-status.sh from installer.sh because Code42 push updates and these
  #new scripts will need this treatment too
  find ${OPTDIR}/ -name "*.sh" | while IFS="" read -r FILE_TO_EDIT; do
    if [ -e ${FILE_TO_EDIT} ]; then
      #this list of substitutions will probably need expanding as new CrashPlan updates are released
      sed -i "s%^#!/bin/bash%#!$/bin/sh%" "${FILE_TO_EDIT}"
      sed -i -r "s%(^\s*)(/bin/ps|ps) [^w][^\|]*\|%\1/bin/ps w \|%" "${FILE_TO_EDIT}"
      sed -i -r "s%\`ps [^w][^\|]*\|%\`ps w \|%" "${FILE_TO_EDIT}"
      sed -i -r "s%^ps [^w][^\|]*\|%ps w \|%" "${FILE_TO_EDIT}"
      sed -i "s/rm -fv/rm -f/" "${FILE_TO_EDIT}"
      sed -i "s/mv -fv/mv -f/" "${FILE_TO_EDIT}"
    fi
  done

  #use this daemon init script rather than the unreliable Code42 stock one which greps the ps output
  sed -i "s%^ENGINE_SCRIPT=.*$%ENGINE_SCRIPT=$0%" ${OPTDIR}/bin/restartLinux.sh

  #any downloaded upgrade script will usually have failed until the above changes are made so we need to
  #find it and start it, if it exists
  UPGRADE_SCRIPT=`find ${OPTDIR}/upgrade -name "upgrade.sh"`
  if [ -n "${UPGRADE_SCRIPT}" ]; then
    rm ${OPTDIR}/*.pid
    SCRIPT_HOME=`dirname $UPGRADE_SCRIPT`
 
    #make CrashPlan log entry
    echo "I ${TIMESTAMP} Synology repairing upgrade in ${SCRIPT_HOME}" >> ${DLOG}

    mv ${SCRIPT_HOME}/upgrade.log ${SCRIPT_HOME}/upgrade.log.old
    cd ${SCRIPT_HOME}
    source upgrade.sh
    mv ${SCRIPT_HOME}/upgrade.sh ${SCRIPT_HOME}/upgrade.sh.old
    exit 0
  fi

  #updates may also overwrite our native binaries
  [ -e ${OPTDIR}/bin/libjtux.so ] && cp -f ${OPTDIR}/bin/libjtux.so ${OPTDIR}/
  cp -f ${OPTDIR}/bin/jna-3.2.5.jar ${OPTDIR}/lib

  #set appropriate Java max heap size
  RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
  if [ $RAM -le 128 ]; then
    JAVA_MAX_HEAP=80
  elif [ $RAM -le 256 ]; then
    JAVA_MAX_HEAP=192
  elif [ $RAM -le 512 ]; then
    JAVA_MAX_HEAP=384
  elif [ $RAM -le 1024 ]; then
    JAVA_MAX_HEAP=512
  elif [ $RAM -gt 1024 ]; then
    JAVA_MAX_HEAP=1024
  fi
  if [ $USR_MAX_HEAP -gt $JAVA_MAX_HEAP ]; then
    JAVA_MAX_HEAP=${USR_MAX_HEAP}
  fi   
  if [ $JAVA_MAX_HEAP -lt $JAVA_MIN_HEAP ]; then
    #can't have a max heap lower than min heap (ARM low RAM systems)
    $JAVA_MAX_HEAP=$JAVA_MIN_HEAP
  fi
  sed -i -r "s/(^${CFG_PARAM}=.*) -Xmx[0-9]+[mM] (.*$)/\1 -Xmx${JAVA_MAX_HEAP}m \2/" "${OPTDIR}/bin/${ENGINE_CFG}"
  
  #disable the use of the x86-optimized external Fast MD5 library if running on ARM and PPC CPUs
  #seems to be the default behaviour now but that may change again
  if [ "${SYNO_CPU_ARCH}" != "x86_64" ]; then
    grep "^${CFG_PARAM}=.*c42\.native\.md5\.enabled" "${OPTDIR}/bin/${ENGINE_CFG}" > /dev/null \
     || sed -i -r "s/(^${CFG_PARAM}=\".*)\"$/\1 -Dc42.native.md5.enabled=false\"/" "${OPTDIR}/bin/${ENGINE_CFG}"
  fi

  #move the Java temp directory from the default of /tmp
  grep "^${CFG_PARAM}=.*Djava\.io\.tmpdir" "${OPTDIR}/bin/${ENGINE_CFG}" > /dev/null \
   || sed -i -r "s%(^${CFG_PARAM}=\".*)\"$%\1 -Djava.io.tmpdir=${TEMP_FOLDER}\"%" "${OPTDIR}/bin/${ENGINE_CFG}"

  #now edit the XML config file, which only exists after first run
  if [ -f ${OPTDIR}/conf/my.service.xml ]; then

    #allow direct connections from CrashPlan Desktop client on remote systems
    #you must edit the value of serviceHost in conf/ui.properties on the client you connect with
    #users report that this value is sometimes reset so now it's set every service startup 
    sed -i "s/<serviceHost>127\.0\.0\.1<\/serviceHost>/<serviceHost>0\.0\.0\.0<\/serviceHost>/" "${OPTDIR}/conf/my.service.xml"
     
    #this change is made only once in case you want to customize the friends' backup location
    if [ "${MANIFEST_PATH_SET}" != "True" ]; then

      #keep friends' backup data outside the application folder to make accidental deletion less likely 
      sed -i "s%<manifestPath>.*</manifestPath>%<manifestPath>${MANIFEST_FOLDER}/backupArchives/</manifestPath>%" "${OPTDIR}/conf/my.service.xml"
      echo "MANIFEST_PATH_SET=True" >> ${OPTDIR}/syno_package.vars
    fi

    #since CrashPlan version 3.5.3 the value javaMemoryHeapMax also needs setting to match that used in bin/run.conf
    sed -i -r "s%(<javaMemoryHeapMax>)[0-9]+[mM](</javaMemoryHeapMax>)%\1${JAVA_MAX_HEAP}m\2%" "${OPTDIR}/conf/my.service.xml"
  else
    echo "Check the package log to ensure the package has started successfully, then stop and restart the package to allow desktop client connections." > "${SYNOPKG_TEMP_LOGFILE}"
  fi

  #increase the system-wide maximum number of open files from Synology default of 24466
  echo "65536" > /proc/sys/fs/file-max

  #raise the maximum open file count from the Synology default of 1024 - thanks Casper K. for figuring this out
  #http://support.code42.com/Administrator/3.6_And_4.0/Troubleshooting/Too_Many_Open_Files
  ulimit -n 65536

  source ${OPTDIR}/bin/run.conf
  source ${OPTDIR}/install.vars
  cd ${OPTDIR}
  $JAVACOMMON $SRV_JAVA_OPTS -classpath $FULL_CP com.backup42.service.CPService > ${OPTDIR}/log/engine_output.log 2> ${OPTDIR}/log/engine_error.log &
  if [ $! -gt 0 ]; then
    echo $! > $PID_FILE
    renice 19 $!
    if [ -z "${SYNOPKG_PKGDEST}" ]; then
      #script was manually invoked, need this to show status change in Package Center      
      [ -e ${PKG_FOLDER}/enabled ] || touch ${PKG_FOLDER}/enabled
    fi
  else
    echo "${DNAME} failed to start, check ${OPTDIR}/log/engine_error.log" > "${SYNOPKG_TEMP_LOGFILE}"
    echo "${DNAME} failed to start, check ${OPTDIR}/log/engine_error.log" >&2
    exit 1
  fi
}

stop_daemon ()
{
  echo "I ${TIMESTAMP} Stopping ${DNAME}" >> ${DLOG}
  kill `cat ${PID_FILE}`
  wait_for_status 1 20 || kill -9 `cat ${PID_FILE}`
  rm -f ${PID_FILE}
  if [ -z ${SYNOPKG_PKGDEST} ]; then
    #script was manually invoked, need this to show status change in Package Center
    [ -e ${PKG_FOLDER}/enabled ] && rm ${PKG_FOLDER}/enabled
  fi
  #backup identity file in case DSM upgrade removes it
  [ -e ${OPTDIR}/conf/var-backup ] || mkdir ${OPTDIR}/conf/var-backup 
  cp /var/lib/crashplan/.identity ${OPTDIR}/conf/var-backup/
}

daemon_status ()
{
  if [ -f ${PID_FILE} ] && kill -0 `cat ${PID_FILE}` > /dev/null 2>&1; then
    return
  fi
  rm -f ${PID_FILE}
  return 1
}

wait_for_status ()
{
  counter=$2
  while [ ${counter} -gt 0 ]; do
    daemon_status
    [ $? -eq $1 ] && return
    let counter=counter-1
    sleep 1
  done
  return 1
}


case $1 in
  start)
    if daemon_status; then
      echo ${DNAME} is already running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo Starting ${DNAME} ...
      start_daemon
      exit $?
    fi
  ;;

  stop)
    if daemon_status; then
      echo Stopping ${DNAME} ...
      stop_daemon
      exit $?
    else
      echo ${DNAME} is not running
      exit 0
    fi
  ;;

  restart)
    stop_daemon
    start_daemon
    exit $?
  ;;

  status)
    if daemon_status; then
      echo ${DNAME} is running with PID `cat ${PID_FILE}`
      exit 0
    else
      echo ${DNAME} is not running
      exit 1
    fi
  ;;

  log)
    echo "${DLOG}"
    exit 0
  ;;

  *)
    echo "Usage: $0 {start|stop|status|restart}" >&2
    exit 1
  ;;

esac
 

Changelog:

  • 0028 Substantial re-write:
    Updated to CrashPlan version 3.6.4
    DSM 5.0 or newer is now required
    libjnidispatch.so taken from Debian JNA 3.2.7 package with dependency on newer libffi.so.6 (included in DSM 5.0)
    jna-3.2.5.jar emptied of irrelevant CPU architecture libs to reduce size
    Increased default max heap size from 512MB to 1GB on systems with more than 1GB RAM
    Intel CPUs no longer need the awkward glibc version-faking shim to enable inotify support (for real-time backup)
    Switched to using root account – no more adding account permissions for backup, package upgrades will no longer break this
    DSM Firewall application definition added
    Tested with DSM Task Scheduler to allow backups between certain times of day only, saving RAM when not in use
    Daemon init script now uses a proper PID file instead of Code42′s unreliable method of using grep on the output of ps
    Daemon init script can be run from the command line
    Removal of bash binary dependency now Code42′s CrashPlanEngine script is no longer used
    Removal of nice binary dependency, using BusyBox’s renice
    Unified ARMv5 and ARMv7 external binary package (armle)
    Added support for Mindspeed Comcerto 2000 CPU (comcerto2k – armhf) in DS414j
    Added support for Intel Atom C2538 (avoton) CPU in DS415+
    Added support to choose which version of CrashPlan PROe client to download, since some servers may still require legacy versions
  • 0027 Fixed open file handle limit for very large backup sets (ulimit fix)
  • 0026 Updated all CrashPlan clients to version 3.6.3, improved handling of Java temp files
  • 0025 glibc version shim no longer used on Intel Synology models running DSM 5.0
  • 0024 Updated to CrashPlan PROe 3.6.1.4 and added support for PowerPC 2010 Synology models running DSM 5.0
  • 0023 Added support for Intel Atom Evansport and Armada XP CPUs in new DSx14 products
  • 0022 Updated all CrashPlan client versions to 3.5.3, compiled native binary dependencies to add support for Armada 370 CPU (DS213j), start-stop-status.sh now updates the new javaMemoryHeapMax value in my.service.xml to the value defined in syno_package.vars
  • 0021 Updated CrashPlan to version 3.5.2
  • 0020 Fixes for DSM 4.2
  • 018 Updated CrashPlan PRO to version 3.4.1
  • 017 Updated CrashPlan and CrashPlan PROe to version 3.4.1, and improved in-app update handling
  • 016 Added support for Freescale QorIQ CPUs in some x13 series Synology models, and installer script now downloads native binaries separately to reduce repo hosting bandwidth, PowerQUICC PowerPC processors in previous Synology generations with older glibc versions are not supported
  • 015 Added support for easy scheduling via cron – see updated Notes section
  • 014 DSM 4.1 user profile permissions fix
  • 013 implemented update handling for future automatic updates from Code 42, and incremented CrashPlanPRO client to release version 3.2.1
  • 012 incremented CrashPlanPROe client to release version 3.3
  • 011 minor fix to allow a wildcard on the cpio archive name inside the main installer package (to fix CP PROe client since Code 42 Software had amended the cpio file version to 3.2.1.2)
  • 010 minor bug fix relating to daemon home directory path
  • 009 rewrote the scripts to be even easier to maintain and unified as much as possible with my imminent CrashPlan PROe server package, fixed a timezone bug (tightened regex matching), moved the script-amending logic from installer.sh to start-stop-status.sh with it now applying to all .sh scripts each startup so perhaps updates from Code42 might work in future, if wget fails to fetch the installer from Code42 the installer will look for the file in the public shared folder
  • 008 merged the 14 package scripts each (7 for ARM, 7 for Intel) for CP, CP PRO, & CP PROe – 42 scripts in total – down to just two! ARM & Intel are now supported by the same package, Intel synos now have working inotify support (Real-Time Backup) thanks to rwojo’s shim to pass the glibc version check, upgrade process now retains login, cache and log data (no more re-scanning), users can specify a persistent larger max heap size for very large backup sets
  • 007 fixed a bug that broke CrashPlan if the Java folder moved (if you changed version)
  • 006 installation now fails without User Home service enabled, fixed Daylight Saving Time support, automated replacing the ARM libffi.so symlink which is destroyed by DSM upgrades, stopped assuming the primary storage volume is /volume1, reset ownership on /var/lib/crashplan and the Friends backup location after installs and upgrades
  • 005 added warning to restart daemon after 1st run, and improved upgrade process again
  • 004 updated to CrashPlan 3.2.1 and improved package upgrade process, forced binding to 0.0.0.0 each startup
  • 003 fixed ownership of /volume1/crashplan folder
  • 002 updated to CrashPlan 3.2
  • 001 intial public release
 
 

Serviio 1.4.1.2 package for Synology NAS

Serviio 1.4 in DSM Package Center

Many people are using the Synology packages I created for Serviio and Java, and most of them will have installed from my package repository making the upgrade possible with just one click.

This new version updates Serviio to version 1.4.1.2 (release notes). The Media Browser has been redesigned (Pro version feature), and is designed to allow direct playback of media on tablet devices without the need for dedicated apps. Please note that Apple iOS devices require video in H.264 and audio in stereo AAC format when streaming. Most Synology NAS models lack the CPU power to transcode video into H.264 and furthermore, licensing restrictions prevent distribution of an FFmpeg binary with libfdk_aac support. As a result, realtime AAC audio transcoding on older ARMv5 NAS models (which lack an FPU) is also out of reach, even if the source video is already H.264.

Renderers that do not natively support subtitles via DLNA can have them burned into the video during transcoding, but in general a NAS lacks the CPU power to do this (more info here). Please comment if this works nicely on the Intel or ARMv7 models. ARMv5 and PowerPC are confirmed to be too slow. If you absolutely need hard subs for a particular film, you could start playback on your device then stop it and Serviio will continue transcoding. Then you will be able to try again in a few hours since it will play from the transcode cache folder.

For burning subtitles into video streams the package downloads the free DejaVu font family from SourceForge during installation, since DSM does not include any fonts. Non-Latin character encodings should be supported as long as fonts with support are provided. The DejaVu font does not include support for Chinese glyphs for instance. You will need to copy new fonts to the folder /volume1/@appstore/Serviio/lib/dejavu-fonts-ttf-2.33/ttf.

I was able to compile FFmpeg 2.1 for all four supported architectures with libx264 and libass support (and dependent libraries) while using the libs included within DSM as far as possible. Only the Intel CPU will be of any practical use transcoding to H.264, since the other CPUs lack vector support (no AltiVec in QorIQ/PowerQUICC, and no NEON in Armada 370/XP).

As with the last few versions of the package, there is no Web UI included so I recommend that you manage Serviio using Serviio 1.4 Console installed on another computer. Integrating a Web UI into the package can be tricky because they do introduce problems of their own (bugs, browser incompatibilities), their development tends to lag behind the Serviio releases which can significantly delay my releases, and this also means I have to re-package as they get updated. Maintaining the CSS was also a lot of work. For the relatively small benefit the WebUI brings, it doesn’t really justify the effort. It only takes a few minutes to set up a remote console.

 

Instructions

The package can be installed on an unmodified NAS – no hacking is required.

  • Firstly, in the DSM User control panel enable the User Home service.
  • Open the DSM Package Center. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk
  • Install either one of my Java SE Embedded Synology packages (Java 6 or Java 7). Only Marvell Kirkwood, Marvell Armada 370/XP, Intel, and Freescale QorIQ PowerPC CPUs are supported, or PowerQUICC running DSM 5.0 or later, so please check which CPU your NAS has.
  • Install Serviio using DSM’s Package Center.
  • Download any Serviio online content plugins that you require and leave the .groovy files in your NAS’s /volume1/public/serviio/plugins folder (where /volume1 is you primary storage volume). This folder is created automatically by the package installer.
  • Download and install Serviio on another computer (selecting only the Console when prompted) and configure it to connect to your NAS:
    For Mac OS find Serviio-Console, right-click, then Show Package Contents. Use TextEdit to open Contents/info.plist and add this towards the end of the file after the last key and string entries (Properties section):
    <key>serviio.remoteHost</key>
    <string>192.168.1.100</string>

    ..replacing that IP with the IP of your NAS obviously. Then save, navigate back the Serviio folder and launch Serviio-Console again.
    On Windows, use Notepad to open ServiioConsole.exe.vmoptions (in the Serviio/bin folder) and add the line:
    -Dserviio.remoteHost=192.168.1.100
    On a Linux client, add this same parameter to the JAVA_OPTS in Serviio/bin/serviio-console.sh
    Do not make this change on the NAS – only on the computer that runs the console.
  • Since this console binding uses a static address you should reserve this particular IP address for your NAS (most home broadband routers will let you do this in the DHCP options), or use a static IP address. This will also prevent Serviio appearing multiple times in the menus of your renderer devices.
  • Once the remote console is connected be aware that everything it indicates relates to the NAS, *not* to the Serviio installation on your computer.
 

Notes

  • In the console disable ‘Generate thumbnails for local videos’ in the Metadata tab. Failure to do this will make adding videos to your library very slow. With this disabled Serviio will still fetch thumbnail images automatically from the online databases.
  • Don’t alter the transcoded files location on the Transcoding tab, it’s already set to a sensible location. This particular folder is deleted each time your NAS restarts, and Serviio cleans up its temp files automatically in any case.
  • When adding folders to the media library use the Add Path button (not Add Local). You will need to specify the folder path in the NAS’s Linux notation, for example /volume1/public/videos
  • Don’t forget to use the Users control panel in DSM to grant permission for the Serviio user to read your files (if they’re not in the public share).
  • When closing the Console, notice that ‘Exit Serviio’ quits the Console and also stops Serviio running on the NAS. To only quit the Console, be sure to select ‘Exit Serviio Console’ instead.
  • The package will allow future upgrades while preserving the media database, as long as Serviio itself is ok with that.
  • If you’re upgrading from a previous Serviio version you should update your online content plugins.
  • You can see Serviio’s log by clicking More in the Package Center.
  • The URL of the MediaBrowser is also displayed there (http://IPofYourNAS:23424/mediabrowser). If you want to use the MediaBrowser from a remote location you would need to either port forward 23424 on your router, or use SSH tunnelling. You can also manage Serviio remotely by SSH tunnelling port 23423 which the console uses.
  • If you have upgraded your DSM version and have lost unicode character support in Serviio, you will need to remove and reinstall the Java package which will fix the system locale support (DSM updates break this unfortunately).
  • Package Center installs the application to /volume1/@appstore/Serviio though from what I understand, on multi-volume systems the user is prompted for a destination volume. If you need to edit device profiles or enable debug logging then that’s where you’ll find the files. You will need to use an SSH session to access this folder – it’s not browsable in DSM’s File Station. The simplest way to edit these files if you’re not really confident with Linux is to install Merty’s Config File Editor package, which requires the official Synology Perl package to be installed too (since DSM 4.2). Load Config File Editor, then in the dropdown menu edit Config File Editor’s own config (it’s the last in the list) and add the lines:
    /volume1/@appstore/Serviio/config/profiles.xml,Serviio-profiles
    /volume1/@appstore/Serviio/config/log4j.xml,Serviio-logging

    Make sure to add an extra blank line underneath, save, then relaunch CFE and you’ll have an entry for Serviio in the dropdown. You’ll need to restart the Serviio package for any changes to take effect.
  • Bear in mind that a NAS, even an Intel Atom powered one, is unlikely to have sufficient CPU power to manage realtime video transcoding (e.g. from DivX to MPEG2), though it will manage remuxing MKV to M2TS and realtime audio transcoding. If you have a DLNA renderer with good format support then this isn’t an issue.
  • Unfortunately for this same reason the MediaBrowser is probably of limited value if you run Serviio on a NAS, since it uses realtime video transcoding, but I have nonetheless taken several steps to mitigate this. I have modified the FlowPlayer application profile to play AAC audio natively (you will only get the L+R channels from a multichannel file), and I have been able to integrate the integer maths libshine MP3 encoder into my build of FFmpeg for ARM CPUs, which allows you to listen to FLAC music transcoded to MP3 in realtime (useful for wifi music players for instance). I used a wrapper script to force libshine’s encoding bitrate to 320Kbps at all times since it didn’t increase the load, and it hides this simple encoder’s slight lack of quality. Source code for libshine is available here.
  • I have left the FFmpeg wrapper script in the package even when not in use (in the Serviio/bin folder), because it’s still useful for people trying to test a particular workaround or encoder setting. The wrapper was a really tricky script to get working due to variable expansion precedence issues and quote handling. If you need it, just change the ffmpeg.location system property in /volume1/@appstore/Serviio/bin/serviio.sh (changing the location to /volume1/@appstore/Serviio/bin/ffmpeg-wrapper.sh).
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology Package wiki.

installer.sh

#!/bin/sh

#--------SERVIIO installer script
#--------package maintained at pcloadletter.co.uk

DOWNLOAD_PATH="http://download.serviio.org/releases"
DOWNLOAD_FILE="serviio-1.4.1.2-linux.tar.gz"
EXTRACTED_FOLDER="serviio-1.4.1.2"
DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_PASS="`openssl rand 12 -base64 2>/dev/null`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
SYNO_CPU_ARCH="`uname -m`"
[ "${SYNO_CPU_ARCH}" == "x86_64" ] && SYNO_CPU_ARCH="i686"
NATIVE_BINS_URL="http://packages.pcloadletter.co.uk/downloads/serviio1.4-native-${SYNO_CPU_ARCH}.tgz"   
NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
FONTS_URL="http://sourceforge.net/projects/dejavu/files/dejavu/2.33/dejavu-fonts-ttf-2.33.tar.bz2"
FONTS_FILE="`echo ${FONTS_URL} | sed -r "s%^.*/(.*)%\1%"`"
INSTALL_FILES="${DOWNLOAD_URL} ${NATIVE_BINS_URL} ${FONTS_URL}"

source /etc/profile
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
PUBLIC_FOLDER="`cat /usr/syno/etc/smb.conf | sed -r '/\/public$/!d;s/^.*path=(\/volume[0-9]{1,4}\/public).*$/\1/'`"
PLUGINS_PATH="/`echo $TEMP_FOLDER | cut -f2 -d'/'`/public/serviio"



preinst ()
{
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  #is the User Home service enabled?
  UH_SERVICE=maybe
  synouser --add userhometest Testing123 "User Home test user" 0 "" ""
  UHT_HOMEDIR=`cat /etc/passwd | sed -r '/User Home test user/!d;s/^.*:User Home test user:(.*):.*$/\1/'`
  if echo $UHT_HOMEDIR | grep '/var/services/homes/' > /dev/null; then
    if [ ! -d $UHT_HOMEDIR ]; then
      UH_SERVICE=false
    fi
  fi
  synouser --del userhometest
  #remove home directory (needed since DSM 4.1)
  [ -e /var/services/homes/userhometest ] && rm -r /var/services/homes/userhometest
  if [ ${UH_SERVICE} == "false" ]; then
    echo "The User Home service is not enabled. Please enable this feature in the User control panel in DSM."
    exit 1
  fi
    
  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, "
        echo "which was \"${WGET_URL}\" "
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. "
        exit 1
      fi
    fi
  done

  exit 0
}


postinst ()
{
  #create serviio daemon user
  synouser --add ${DAEMON_USER} ${DAEMON_PASS} "${DAEMON_ID}" 0 "" ""
  
  #determine the serviio user homedir and save that variable in the user's profile
  #this is needed because librtmp needs to write a file called ~/.swfinfo
  #and new users seem to inherit a HOME value of /root which they have no permissions for
  DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
  su - ${DAEMON_USER} -s /bin/sh -c "echo export HOME=\'${DAEMON_HOME}\' >> .profile"
  
  #fontconfig configuration which is needed for libass subtitles
  su - ${DAEMON_USER} -s /bin/sh -c "echo export FONTCONFIG_FILE=fonts.conf >> .profile"
  su - ${DAEMON_USER} -s /bin/sh -c "echo export FONTCONFIG_PATH=${SYNOPKG_PKGDEST}/config/fonts >> .profile"

  #extract the downloaded Serviio archive
  cd ${TEMP_FOLDER}
  tar xzf ${TEMP_FOLDER}/${DOWNLOAD_FILE}
  rm ${TEMP_FOLDER}/${DOWNLOAD_FILE}
  cp -R ${TEMP_FOLDER}/${EXTRACTED_FOLDER}/* ${SYNOPKG_PKGDEST}
  if [ ! -z "${EXTRACTED_FOLDER}" ]; then
    rm -r ${TEMP_FOLDER}/${EXTRACTED_FOLDER}
  fi
  if [ ! -d "${PLUGINS_PATH}/plugins" ]; then
    mkdir -p ${PLUGINS_PATH}/plugins
  fi
  
  #extract CPU-specific additional binaries
  cd ${SYNOPKG_PKGDEST}/lib
  tar xzf ${TEMP_FOLDER}/${NATIVE_BINS_FILE} && rm ${TEMP_FOLDER}/${NATIVE_BINS_FILE}
  mv ${SYNOPKG_PKGDEST}/lib/ffmpeg ${SYNOPKG_PKGDEST}/bin
  ${SYNOPKG_PKGDEST}/bin/ffmpeg -version > /dev/null 2>&1 || (
    #this system is running an older DSM with missing dependencies, fetch more libs
    NATIVE_BINS_URL=`echo ${NATIVE_BINS_URL} | sed -e "s/.tgz/-oldDSM.tgz/"`
    NATIVE_BINS_FILE="`echo ${NATIVE_BINS_URL} | sed -r "s%^.*/(.*)%\1%"`"
    wget ${NATIVE_BINS_URL}
    tar xzf ${NATIVE_BINS_FILE} && rm ${NATIVE_BINS_FILE}
  )
  
  #extract open source font package for subtitle support during transcoding
  tar xjf ${TEMP_FOLDER}/${FONTS_FILE} && rm ${TEMP_FOLDER}/${FONTS_FILE}
  sed -i "s|WINDOWSFONTDIR|${SYNOPKG_PKGDEST}/lib/dejavu-fonts-ttf-2.33/ttf|" ${SYNOPKG_PKGDEST}/config/fonts/fonts.conf
  sed -i "s|WINDOWSTEMPDIR_FONTCONFIG_CACHE|~/.fontconfig.cache|" ${SYNOPKG_PKGDEST}/config/fonts/fonts.conf
   
  #wrapper script can be useful for testing different encoder options
  if [ "${SYNO_CPU_ARCH}" == "armv5tel" ]; then
    #we need to use the wrapper to make FFmpeg use libshine instead of libmp3lame on ARM systems
    FFMPEG_PATH="\$SERVIIO_HOME/bin/ffmpeg-wrapper.sh"
  else
    FFMPEG_PATH="\$SERVIIO_HOME/bin/ffmpeg"
  fi

  #modification to natively play AAC audio in Media Browser, minimizing transcoding
  mv ${SYNOPKG_PKGDEST}/application-profiles.xml ${SYNOPKG_PKGDEST}/config
  
  #set some additional Serviio system properties (temp folder, FFmpeg path, plugins folder)
  #http://www.serviio.org/index.php?option=com_content&view=article&id=43
  EXTRA_OPTS="-Dserviio\.defaultTranscodeFolder=${TEMP_FOLDER} -Dffmpeg\.location=${FFMPEG_PATH} -Dplugins\.location=${PLUGINS_PATH}"
  #fix Java prefs checking which was preventing NAS hibernation http://forum.serviio.org/viewtopic.php?f=5&t=6878
  EXTRA_OPTS="${EXTRA_OPTS} -Djava.util.prefs.syncInterval=86400"
  if [ "${SYNO_CPU_ARCH}" == "armv5tel" ]; then
    #use integer math (not floating point) Dolby AC-3 encoder for better performance on ARM CPUs
    #http://ffmpeg.org/ffmpeg.html#ac3-and-ac3_005ffixed
    EXTRA_OPTS="${EXTRA_OPTS} -Dserviio\.fixedPointEncoders"
  fi
  sed -r -i "s%^(JAVA_OPTS=.*)\"$%\1 ${EXTRA_OPTS}\"%" ${SYNOPKG_PKGDEST}/bin/serviio.sh
  
  #change owner of Serviio folder tree
  chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
    
  #reset ownership on any pre-existing transcoding temp folder
  if [ -d ${TEMP_FOLDER}/Serviio ]; then
    chown -R ${DAEMON_USER} ${TEMP_FOLDER}/Serviio
  fi
  
  exit 0
}


preuninst ()
{
  #make sure that daemon is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${SYNOPKG_PKGDEST}/bin/serviio-wrapper.sh -stop"
  sleep 6
  
  exit 0
}


postuninst ()
{
  #remove daemon user
  synouser --del ${DAEMON_USER}
  
  #remove daemon user's home directory (needed since DSM 4.1)
  [ -e /var/services/homes/${DAEMON_USER} ] && rm -r /var/services/homes/${DAEMON_USER}
  
  exit 0
}


preupgrade ()
{
  #make sure that daemon is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${SYNOPKG_PKGDEST}/bin/serviio-wrapper.sh -stop"
  sleep 6
  
  #if a media database exists we need to preserve it
  if [ -d ${SYNOPKG_PKGDEST}/library/db ]; then
    mkdir ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_db_migration
    mv ${SYNOPKG_PKGDEST}/library/db ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_db_migration
  fi
  
  exit 0
}


postupgrade ()
{
  #use the backed up media database from the previous version
  if [ -d ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_db_migration/db ]; then
    mv ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_db_migration/db ${SYNOPKG_PKGDEST}/library
    rmdir ${SYNOPKG_PKGDEST}/../${DAEMON_USER}_db_migration
    
    #daemon user has been deleted and recreated so we need to reset ownership (new UID)
    chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}/library/db
  fi
  	
  #reset ownership on transcoding temp folder
  if [ -d ${TEMP_FOLDER}/Serviio ]; then
    chown -R ${DAEMON_USER} ${TEMP_FOLDER}/Serviio
  fi
  
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------SERVIIO start-stop-status script
#--------package maintained at pcloadletter.co.uk

DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
ENGINE_CFG="serviio.sh"
ENGINE_SCRIPT="serviio.sh"


daemon_status ()
{
  ps | grep "^ *[0-9]* ${DAEMON_USER} .*java" > /dev/null
}


case $1 in
  start)
    DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
    
    #set the current timezone for Java so that log timestamps are accurate
    #we need to use the modern timezone names so that Java can figure out DST
    SYNO_TZ=`cat /etc/synoinfo.conf | grep timezone | cut -f2 -d'"'`
    SYNO_TZ=`grep "^${SYNO_TZ}" /usr/share/zoneinfo/Timezone/tzname | sed -e "s/^.*= //"`
    grep "^export TZ" ${DAEMON_HOME}/.profile > /dev/null \
     && sed -i "s%^export TZ=.*$%export TZ='${SYNO_TZ}'%" ${DAEMON_HOME}/.profile \
     || echo export TZ=\'${SYNO_TZ}\' >> ${DAEMON_HOME}/.profile
    
    #set appropriate Java max heap size
    RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
    if [ $RAM -le 128 ]; then
      JAVA_MAX_HEAP=80
    elif [ $RAM -le 256 ]; then
      JAVA_MAX_HEAP=192
    elif [ $RAM -le 512 ]; then
      JAVA_MAX_HEAP=384
    #Serviio's default max heap is 512MB
    elif [ $RAM -gt 512 ]; then
      JAVA_MAX_HEAP=512
    fi
    sed -i -r "s/(^..JAVA.) -Xmx[0-9]+[mM] (.*$)/\1 -Xmx${JAVA_MAX_HEAP}m \2/" "${SYNOPKG_PKGDEST}/bin/${ENGINE_CFG}"
    su - ${DAEMON_USER} -s /bin/sh -c "${SYNOPKG_PKGDEST}/bin/${ENGINE_SCRIPT} &"
    exit 0
  ;;

  stop)
    su - ${DAEMON_USER} -s /bin/sh -c "${SYNOPKG_PKGDEST}/bin/${ENGINE_SCRIPT} -stop"
    sleep 6
    exit 0
  ;;

  status)
    if daemon_status ; then
      exit 0
    else
      exit 1
    fi
  ;;

  log)
    echo "${SYNOPKG_PKGDEST}/log/serviio.log"
    exit 0
  ;;

esac
 

ffmpeg-wrapper.sh

#!/bin/sh

#FFmpeg wrapper script to use libshine fixed point maths MP3 encoder on ARM CPUs
#as originally posted here: http://forum.serviio.org/viewtopic.php?f=7&t=6458

PARAMS=""
FFMPEG_BIN="ffmpeg"
INPUT=0
for PARAM in "$@"; do
  if [ ${INPUT} = 1 ]; then
    #the FFmpeg input filename/URL needs quotes adding back on
    #because it may contain spaces, and the shell has removed them
    PARAMS="${PARAMS} \"${PARAM}\""    
    INPUT=0
  else
    PARAMS="${PARAMS} ${PARAM}"
  fi
  if [ "${PARAM}" == "-i" ]; then
    #this loop is the -i parameter, the next loop will be the input filename/URL
    INPUT=1
  fi
done

#make libshine encoder substitution, 320kbps since it's no more expensive
if [ "${PARAMS}" != "${PARAMS/libmp3lame/}" ] || [ "${PARAMS}" != "${PARAMS/-f mp3/}" ]; then
  PARAMS="`echo ${PARAMS} | sed -r "s|libmp3lame|libshine|;s|-b:a [0-9]+k|-b:a 320k|"`"
fi

#make libfdk_aac encoder substitution, 128Kbps since we're usually transcoding to AAC stereo only
#if [ "${PARAMS}" != "${PARAMS/-c:a aac/}" ]; then
#  PARAMS="`echo ${PARAMS} | sed -r "s| aac -strict experimental | libfdk_aac |;s|-b:a [0-9]+k|-b:a 128k|"`"
#fi

#invoke FFmpeg
FOLDER="`dirname $0`"
echo "${FOLDER}/${FFMPEG_BIN} ${PARAMS}" > ${FOLDER}/../log/ffmpeg-wrapper.log
#need to use eval here otherwise the quotes aren't handled properly
#http://fvue.nl/wiki/Bash:_Why_use_eval_with_variable_expansion%3F
eval ${FOLDER}/${FFMPEG_BIN} ${PARAMS}

#return FFmpeg status
exit $?
 

Changelog:

  • 1.4.1.2-0026 Updated to Serviio 1.4.1.2
  • 1.4.1.1-0025 Updated to Serviio 1.4.1.1
  • 1.4.0-0024 Updated to Serviio 1.4
  • 1.3.1-0023 Updated to FFmpeg 2.1 stable release
  • 1.3.1-0022 Added support for Intel Atom Evansport and Armada XP CPUs in new DSx14 products
  • 1.3.1-0021 Fixed zlib linking issue in QorIQ and Intel builds of FFmpeg
  • 1.3.1-0020 Updated to Serviio 1.3.1
  • 1.2.1-0019 Added support for Armada370 SoC used in DS213j (ARMv7 CPU with FPU)
  • 1.2.1-0018 Updated to Serviio 1.2.1, recompiled FFmpeg to use the libRTMP source code from serviio.org for consistency
  • 1.2.0-0017 Updated to Serviio 1.2
  • 1.1.0-0016 Fixes for DSM 4.2
  • 1.1.0-015 Updated to Serviio 1.1, fixed garbled ac3 encoding issue on ARM CPUs by compiling FFmpeg natively, rather than cross compiling. Remember to update your computer’s Serviio Console to 1.1 before connecting to manage.
  • 1.0-014 Added support for Freescale QorIQ PowerPC CPUs used in some Synology x13 series products, switched to shared library compile of FFmpeg with some minor changes, ARM build now uses libshine encoder which allows realtime audio transcoding to MP3, edited FlowPlayer config to play AAC audio natively to make Media Browser more usable with online feed items.
  • 1.0-013 fixed a bug in the plugins folder creation, fixed deletion of home directories after user deletion (DSM 4.1)
  • 1.0-012 reduced Java prefs checking interval to once every 24 hours to allow the NAS to hibernate (was 30 seconds by default – remember to increase the time between library refreshes if you want hibernation)
  • 1.0-011 updated Serviio to 1.0.1, slightly altered how the plugins folder path is determined
  • 1.0-010 updated to Serviio 1.0, removed WebUI, FFmpeg wrapper no longer needed on ARM, plugins folder moved to /volume1/public/serviio/plugins to make adding/updating plugins easier
  • 0.6.2-009 installation fails unless User Home service is enabled, unified the installer scripts, merged ARM and Intel packages into one which downloads the FFmpeg binary separately, used integer maths Dolby AC-3 encoder on ARM systems (no floating point)
  • 0.6.2-008 fixed DST timezone support, installer no longer assumes /volume1 is primary storage volume
  • 0.6.2-007 updated Serviio to 0.6.2, kairoh’s WebUI to 0.6.2c, changed package to download Serviio from the official website during installation, temp path is set by default to /volume1/@tmp on first run, and finally some CSS improvements
  • 0.6.1-006 switched to kairoh’s Java WebUI, added timezone support, removed DSM icon when Serviio is not running, adjusted Java max heap size for systems with low RAM, specified FFmpeg path directly rather than creating a symlink in /bin so as not to interfere with other packages which may use different versions of FFmpeg
  • 0.6.0.1-005 added Web Station dependency, EULA dialog, and links for Web UI and user forum in More Info
  • 0.6.0.1-004 test for package repo to allow update notification
  • 0.6.0.1 v3 hopefully fixed an issue with the Serviio DSM icon in DSM 3.2 on NAS units without Optware installed
  • 0.6.0.1 v2 inclusion of a modified version of the PHP Web UI
  • 0.6.0.1 v1 new minor Serviio release with some updated international translations – will upgrade 0.6 preserving the media library
  • 0.6 v1 first public release
  • 0.6b4 v2 fixed permissions on transcode temporary folder following upgrade, avoiding the need for a reboot
  • 0.6b4 v1 added upgrade scripts to allow media database migration (install future packages over the top of this one – database is preserved)
  • 0.6b3c v2 fixed HOME env var for serviio user (to fix librtmp issues with BBC iPlayer) and removed the need to edit the passwd file to change the shell (safer)
  • 0.6b3c v1 initial spk test release

 
 

Minecraft package for Synology NAS

UPDATE – The instructions and notes on this page apply to both the Minecraft and the CraftBukkit packages hosted on my repo. Now works on QorIQ CPU Synology models!

Minecraft is an intriguing game. I only recently bought it after a friend had recommended it. There’s a lot to like: the striking retro visual style, the mellow sporadic music, exploring the procedurally generated world, the logical way items are crafted, multiplayer collaboration, and emergent gameplay. Notch its creator certainly deserves the fortune he’s now sitting on. I thoroughly recommend you watch this short documentary all about it.

Minecraft Package Info

Right after I released the Java package for Synology a couple of people mentioned that they wanted to use it for installing a Minecraft server. Now that I have the game, I decided to try that too. I had assumed that the ARM CPUs would not be capable of running it, and the first tests seemed to confirm this. I decided it would be worth creating a package nonetheless since the Intel CPU NAS units would be ok, especially with their extra RAM. However, with a little performance tuning it actually runs acceptably (no lag when mining blocks) on my ARM powered DS111 which only has 256MB. The CPU load is 100% for a while at first, but soon after playing it settles down to around 60%. I have briefly tested with two players connected and after a bit of lag at first (one second delay to mine a block) it does seem to settle down.

Minecraft running showing draw distance and server load

Minecraft showing draw distance and server load running on a Synology DS111

 

Installation

  • This package is not CPU specific. If you have Java it will run. At the moment that means ARM, Intel and QorIQ PowerPC processors, but not the older PowerPCs.
  • In the User Control Panel in DSM, enable the User Homes service.
  • Install the package directly from Package Center in DSM. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk. You will need to install either one of my Java SE for Embedded packages first (Java 6 or 7).

Notes

  • The package fetches the minecraft server jar file from Mojang as it is installed. I am complying with their wish that no one redistributes it.
  • The server daemon script checks the amount of system RAM and scales the Java heap size appropriately. It also applies a few performance tweaks to Java to try to reduce garbage collection latency.
  • The first time you run the server it will create a new world, which can take a few minutes. You cannot interrupt this but you can check on progress by repeatedly viewing the Log tab.
  • Because the time investments in playing Minecraft can be so considerable, when you uninstall the package it will back up the world folder and settings to /volume1/public/minecraftworld.todaysdate.bak to prevent accidental deletion.
  • The package supports upgrades to future versions while preserving the world folder and server settings.
  • If you want to transplant an existing world folder into the server, copy it to /volume1/@appstore/Minecraft. You will also need to run chown -R minecraft /volume1/@appstore/Minecraft to grant ownership of the files to the daemon user.
  • The first time you run the package, the server config file /volume1/@appstore/Minecraft/server.properties is generated. The next time it is launched, my script reduces the default draw distance for ARM CPUs from 10 chunks to 7. This was appropriate for my DS111 to prevent latency when mining blocks, but you may wish to reduce this further on the J series NAS units which have less RAM and slower CPUs. Other server files (white-list.txt etc.) are found in the same folder. For CraftBukkit the files are in the folder /volume1/@appstore/Craftbukkit/server.properties (note the capitalization – the CraftBukkit project seemed to change this after I had already created the package with a lower case ‘b’).
  • The simplest way to edit these config files if you’re not really confident with Linux is to install Merty’s Config File Editor package, which requires the official Synology Perl package to be installed too (since DSM 4.2). Load Config File Editor, then in the dropdown menu edit Config File Editor’s own config (it’s the last in the list) and add the lines:
    /volume1/@appstore/Minecraft/server.properties,Minecraft-properties
    /volume1/@appstore/Minecraft/white-list.txt,Minecraft-whitelist
    /volume1/@appstore/Minecraft/ops.txt,Minecraft-ops

    Make sure to add an extra blank line underneath, save, then relaunch CFE and you’ll have entries for Minecraft in the dropdown. You’ll need to restart the Minecraft package for any changes to take effect.
  • It was a bit tricky to get the server to shut down gracefully without just killing the Java process. We need it to shutdown properly so it saves the active chunks to disk first. Most of the guides on the Net use the screen binary which isn’t included with Synology DSM, and I didn’t want to have to make a version of the package for each CPU architecture. I found that I could use tail to send the last line of the file /tmp/stdin.minecraft to the server (/tmp/stdin.craftbukkit for the CraftBukkit package). This is how the stop command is issued. You could send your own commands, for instance echo say Hello players >> /tmp/stdin.minecraft. You can verify that the command was received by looking at the server log in Package Center.
  • The server runs on the default TCP port for Minecraft (25565) so you will need to port forward this on your router if you want it to be publicly accessible.
 

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology Package wiki.

installer.sh

#!/bin/sh

#--------MINECRAFT/CRAFTBUKKIT installer script
#--------package maintained at pcloadletter.co.uk

if [ "${SYNOPKG_PKGNAME}" == "Minecraft" ]; then
  DOWNLOAD_PATH="http://s3.amazonaws.com/MinecraftDownload/launcher"
  DOWNLOAD_FILE="minecraft_server.jar"
  UPGRADE_FILES="server.properties *.txt world"
fi
if [ "${SYNOPKG_PKGNAME}" == "Craftbukkit" ]; then
  DOWNLOAD_PATH="http://cbukk.it"
  DOWNLOAD_FILE="craftbukkit-beta.jar"
  UPGRADE_FILES="server.properties *.txt *.yml world world_nether world_the_end plugins bukkit_update"
fi

DOWNLOAD_URL="${DOWNLOAD_PATH}/${DOWNLOAD_FILE}"
DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
DAEMON_PASS="`openssl rand 12 -base64 2>/dev/null`"
MIGRATION_FOLDER="${DAEMON_USER}_data_mig"
ENGINE_SCRIPT="/var/packages/${SYNOPKG_PKGNAME}/scripts/launcher.sh"
INSTALL_FILES="${DOWNLOAD_URL}"
source /etc/profile
TEMP_FOLDER="`find / -maxdepth 2 -name '@tmp' | head -n 1`"
PRIMARY_VOLUME="/`echo $TEMP_FOLDER | cut -f2 -d'/'`"
WORLD_BACKUP="${PRIMARY_VOLUME}/public/${DAEMON_USER}world.`date +\"%d-%b\"`.bak"

preinst ()
{
  if [ -z ${JAVA_HOME} ]; then
    echo "Java is not installed or not properly configured. JAVA_HOME is not defined. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  if [ ! -f ${JAVA_HOME}/bin/java ]; then
    echo "Java is not installed or not properly configured. The Java binary could not be located. "
    echo "Download and install the Java Synology package from http://wp.me/pVshC-z5"
    exit 1
  fi
  
  #is the User Home service enabled?
  UH_SERVICE=maybe
  synouser --add userhometest Testing123 "User Home test user" 0 "" ""
  UHT_HOMEDIR=`cat /etc/passwd | sed -r '/User Home test user/!d;s/^.*:User Home test user:(.*):.*$/\1/'`
  if echo $UHT_HOMEDIR | grep '/var/services/homes/' > /dev/null; then
    if [ ! -d $UHT_HOMEDIR ]; then
      UH_SERVICE=false
    fi
  fi
  synouser --del userhometest
  #remove home directory (needed since DSM 4.1)
  [ -e /var/services/homes/userhometest ] && rm -r /var/services/homes/userhometest
  if [ ${UH_SERVICE} == "false" ]; then
    echo "The User Home service is not enabled. Please enable this feature in the User control panel in DSM."
    exit 1
  fi

  cd ${TEMP_FOLDER}
  for WGET_URL in ${INSTALL_FILES}
  do
    WGET_FILENAME="`echo ${WGET_URL} | sed -r "s%^.*/(.*)%\1%"`"
    [ -f ${TEMP_FOLDER}/${WGET_FILENAME} ] && rm ${TEMP_FOLDER}/${WGET_FILENAME}
    wget ${WGET_URL}
    if [[ $? != 0 ]]; then
      if [ -d ${PUBLIC_FOLDER} ] && [ -f ${PUBLIC_FOLDER}/${WGET_FILENAME} ]; then
        cp ${PUBLIC_FOLDER}/${WGET_FILENAME} ${TEMP_FOLDER}
      else     
        echo "There was a problem downloading ${WGET_FILENAME} from the official download link, "
        echo "which was \"${WGET_URL}\" "
        echo "Alternatively, you may download this file manually and place it in the 'public' shared folder. "
        exit 1
      fi
    fi
  done
  
  exit 0
}


postinst ()
{
  #create daemon user
  synouser --add ${DAEMON_USER} ${DAEMON_PASS} "${DAEMON_ID}" 0 "" ""
  
  mv ${TEMP_FOLDER}/${DAEMON_USER}*.jar ${SYNOPKG_PKGDEST}/${DAEMON_USER}.jar
  
  #determine the daemon user homedir and save that variable in the user's profile
  #this is needed because new users seem to inherit a HOME value of /root which they have no permissions for
  DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
  su - ${DAEMON_USER} -s /bin/sh -c "echo export HOME=\'${DAEMON_HOME}\' >> .profile"
  
  #change owner of folder tree
  chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
  
  exit 0
}


preuninst ()
{
  #make sure server is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} stop ${SYNOPKG_PKGNAME} ${SYNOPKG_PKGDEST}"
  sleep 10
  
  #if a world exists, back it up to the public folder, just in case...
  if [ -d ${SYNOPKG_PKGDEST}/world ]; then
    if [ ! -d ${WORLD_BACKUP} ]; then
      mkdir -p ${WORLD_BACKUP}
    fi
    for ITEM in ${UPGRADE_FILES}; do
      mv ${SYNOPKG_PKGDEST}/${ITEM} ${WORLD_BACKUP}
    done
  fi
  
  exit 0
}


postuninst ()
{
  #remove daemon user
  synouser --del ${DAEMON_USER}
  
  #remove daemon user's home directory (needed since DSM 4.1)
  [ -e /var/services/homes/${DAEMON_USER} ] && rm -r /var/services/homes/${DAEMON_USER}
  
  exit 0
}


preupgrade ()
{
  #make sure the server is stopped
  su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} stop ${SYNOPKG_PKGNAME} ${SYNOPKG_PKGDEST}"
  sleep 10
  
  #if a world exists, back it up
  if [ -d ${SYNOPKG_PKGDEST}/world ]; then
    mkdir ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}
    for ITEM in ${UPGRADE_FILES}; do
      if [ -e ${SYNOPKG_PKGDEST}/${ITEM} ]; then
        mv ${SYNOPKG_PKGDEST}/${ITEM} ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}
      fi
    done
  fi
  
  exit 0
}


postupgrade ()
{
  #use the migrated data files from the previous version
  if [ -d ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}/world ]; then
    for ITEM in ${UPGRADE_FILES}; do
      if [ -e ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}/${ITEM} ]; then
        mv ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}/${ITEM} ${SYNOPKG_PKGDEST}
      fi
    done
    rmdir ${SYNOPKG_PKGDEST}/../${MIGRATION_FOLDER}
    
    #daemon user has been deleted and recreated so we need to reset ownership (new UID)
    chown -R ${DAEMON_USER} ${SYNOPKG_PKGDEST}
  fi
  	
  exit 0
}
 

start-stop-status.sh

#!/bin/sh

#--------MINECRAFT/CRAFTBUKKIT start-stop-status script
#--------package maintained at pcloadletter.co.uk

DAEMON_USER="`echo ${SYNOPKG_PKGNAME} | awk {'print tolower($_)'}`"
DAEMON_ID="${SYNOPKG_PKGNAME} daemon user"
ENGINE_SCRIPT="/var/packages/${SYNOPKG_PKGNAME}/scripts/launcher.sh"
DAEMON_USER_SHORT=`echo ${DAEMON_USER} | cut -c 1-8`

daemon_status ()
{
    ps | grep "^ *[0-9]* ${DAEMON_USER_SHORT} .*java" > /dev/null
}

case $1 in
  start)
    DAEMON_HOME="`cat /etc/passwd | grep "${DAEMON_ID}" | cut -f6 -d':'`"
    
    #set the current timezone for Java so that log timestamps are accurate
    #we need to use the modern timezone names so that Java can figure out DST
    SYNO_TZ=`cat /etc/synoinfo.conf | grep timezone | cut -f2 -d'"'`
    SYNO_TZ=`grep "^${SYNO_TZ}" /usr/share/zoneinfo/Timezone/tzname | sed -e "s/^.*= //"`
    grep "^export TZ" ${DAEMON_HOME}/.profile > /dev/null \
     && sed -i "s%^export TZ=.*$%export TZ='${SYNO_TZ}'%" ${DAEMON_HOME}/.profile \
     || echo export TZ=\'${SYNO_TZ}\' >> ${DAEMON_HOME}/.profile
    
    su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} start ${DAEMON_USER} ${SYNOPKG_PKGDEST} &"
    exit 0
  ;;
  
  stop)
    su - ${DAEMON_USER} -s /bin/sh -c "${ENGINE_SCRIPT} stop ${DAEMON_USER} ${SYNOPKG_PKGDEST}"
    exit 0
  ;;
  
  status)
    if daemon_status ; then
      exit 0
    else
      exit 1
    fi
  ;;
  
  log)
    echo "${SYNOPKG_PKGDEST}/server.log"
    exit 0
  ;;
esac
 

launcher.sh

#!/bin/sh

#--------MINECRAFT/CRAFTBUKKIT server launcher script
#--------package maintained at pcloadletter.co.uk
 
#--------Allows graceful shutdown of server without CPU-specific binaries
#--------You can send commands to the running server like so:
#--------    echo say Hello players >> /tmp/stdin.minecraft
#--------    echo say Hello players >> /tmp/stdin.craftbukkit

DAEMON_USER=$2
SYNOPKG_PKGDEST=$3
DAEMON_USER_SHORT=`echo ${DAEMON_USER} | cut -c 1-8`
JAR_FILE=${SYNOPKG_PKGDEST}/$2.jar

case $1 in
  start)
    if [ -f /tmp/stdin.${DAEMON_USER} ]; then
      rm /tmp/stdin.${DAEMON_USER}
    fi
    touch /tmp/stdin.${DAEMON_USER}
    cd ${SYNOPKG_PKGDEST}
    if [ ! -f syno-marker.txt ]; then
      if [ -f server.properties ]; then
        sed -i "s/A Minecraft Server/A Synology Minecraft Server/" server.properties
  
        #ARM CPU lags a lot, so reduce drawing distance from 10 chunks to 6
        cat /proc/cpuinfo | grep "CPU architecture: 5TE" > /dev/null \
         && sed -i "s/^view-distance=10/view-distance=6/" server.properties
  
        #record that these mods have been made
        echo config updated > syno-marker.txt
      fi
    fi
    JAVA_OPTS='-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalPacing -XX:+AggressiveOpts'
    RAM=$((`free | grep Mem: | sed -e "s/^ *Mem: *\([0-9]*\).*$/\1/"`/1024))
    if [ $RAM -le 128 ]; then
      JAVA_MAX_HEAP=80M
    elif [ $RAM -le 256 ]; then
      JAVA_MAX_HEAP=192M			
    elif [ $RAM -le 512 ]; then
      JAVA_MAX_HEAP=448M
    elif [ $RAM -le 1024 ]; then
      JAVA_MAX_HEAP=896M
    elif [ $RAM -le 2048 ]; then
      JAVA_MAX_HEAP=1792M
    elif [ $RAM -gt 2048 ]; then
      JAVA_MAX_HEAP=2048M
    fi
    JAVA_START_HEAP=${JAVA_MAX_HEAP}
    tail -n 0 -f /tmp/stdin.${DAEMON_USER} | java -Xmx${JAVA_START_HEAP} -Xms${JAVA_MAX_HEAP} ${JAVA_OPTS} -jar ${JAR_FILE} nogui
  ;;

  stop)
    echo say shutting down.. >> /tmp/stdin.${DAEMON_USER}
    sleep 5
    echo stop >> /tmp/stdin.${DAEMON_USER}
    sleep 10
    kill -9 `ps | grep "^ *[0-9]* ${DAEMON_USER_SHORT}.*tail -n 0 -f /tmp/stdin.${DAEMON_USER}" | sed -e "s/^ *\([0-9]*\).*$/\1/"`
    if [ -f /tmp/stdin.${DAEMON_USER} ]; then
      rm /tmp/stdin.${DAEMON_USER}
    fi
  ;;
esac
 

Changelog:

  • 0015 updated to Minecraft 1.5.2, CraftBukkit beta 1.5.2-R0.1
  • 0014 updated to Minecraft 1.4.7, CraftBukkit beta 1.4.7-R0.1, and fixes for DSM 4.2
  • 013 updated to Minecraft 1.4.6, and CraftBukkit beta 1.4.6-R0.3
  • 012 updated to Minecraft 1.4.5, and CraftBukkit 1.3.2-R1.0
  • 011 updated to Minecraft 1.4.2
  • 010 updated to Minecraft 1.3.2, and CraftBukkit 1.3.1-R2.0
  • 009 package scripts fully re-written to unify the Minecraft and CraftBukkit packages
  • 008 updated to Minecraft 1.3.1, unified most scripts into a single installer script, and incorporated minor enhancements from my other packages
  • 007 updated to Minecraft 1.2.3
  • 006 reduced Java max heap to 80MB on 128MB systems, fixed timezone support each server start
  • 005 fixed Java max heap size behaviour on systems with more than 2GB RAM
  • 004 fixed wget SSL problem preventing jar download on some systems
  • 003 updated to Minecraft 1.1
  • 002 server config files are also migrated during version upgrades, and backed up during uninstall. My script’s edits to server.properties are made only once, rather than every startup
  • 001 intial public release