foobarlab

random thoughts on arts, coding and open source

Installing UpStage from scratch

It seems like being kind of dif­fi­cult to get the UpStage server appli­ca­tion up and run­ning on your own server, so here you find instruc­tions and hints how to install and con­fig­ure it from scratch. This arti­cle refers to the lat­est sta­ble ver­sion, at the time of this writ­ing it is 2.1. But it should essen­tially work for the 2.4.2 and 2.4.3 ver­sions too. UpStage is meant to be run on Linux (although with some effort it might also run on Win­dows or MacOS X, see the unfin­ished UpStage Fork).

If you just want to use the UpStage client with a web browser you do not need to install any­thing and you will not find much use­ful infor­ma­tion here, instead please visit the com­pre­hen­sive UpStage user man­ual.

First step: Prerequisites

This tuto­r­ial is based on the Debian 6.0 (Squeeze) Linux dis­tri­b­u­tion. Pro­ce­dure on Ubuntu based sys­tems should be quite iden­ti­cal, but if you have found major dif­fer­ences you are wel­come to post a com­ment and elab­o­rate on this. All steps have been ver­i­fied with an instance of Vir­tual Box. Both 32-bit and 64-bit dis­tri­b­u­tions are sup­ported in this tuto­r­ial (in gen­eral all writ­ing is pri­mary for 32-bit, dif­fer­ences to 64-bit are marked).

Just for your infor­ma­tion, here is an overview of all required depen­den­cies to third-party soft­ware to make use of all functionalities:

You will find detailed infor­ma­tion about installing and con­fig­ur­ing these in the next steps.

All com­mands in this tuto­r­ial should be exe­cuted in the ter­mi­nal win­dow as the root user. If you do not have a root user, you might need to prepend the sudo com­mand to gain supe­ruser rights.

Ensure your sys­tem is up-to-date and basic tools are available:

apt-get update
apt-get upgrade
apt-get install build-essential bzip2 unzip

Next step: Install Python and Twisted

We install Python 2.6 with the fol­low­ing command:

apt-get install python python-dev

Next we will install Twisted 8.1. Debian Squeeze natively only pro­vides Twisted 10.1 (Ubuntu pro­vides 11.x). I would rec­om­mend to unin­stall any already installed ver­sion of Twisted before­hand as I have not tested a par­al­lel install of dif­fer­ent versions.

You can unin­stall an already installed Twisted and all of its depen­den­cies using the pack­age man­ager, apti­tude or issue­ing the fol­low­ing command:

apt-get remove python-twisted python-twisted-bin python-twisted-bin-dbg \
python-twisted-conch python-twisted-core python-twisted-lore \
python-twisted-mail python-twisted-names python-twisted-news \
python-twisted-runner python-twisted-runner-dbg python-twisted-web \
python-twisted-words twisted-doc python-twisted-calendarserver \
python-twisted-libravatar python-twisted-web2

We down­load Twisted-8.1.0.tar.bz2 and install it with the fol­low­ing commands:

wget http://twistedmatrix.com/Releases/Twisted/8.1/Twisted-8.1.0.tar.bz2
tar -xvjf ./Twisted-8.1.0.tar.bz2
cd Twisted-8.1.0
python setup.py install
cd ..

Addi­tion­ally we need TwistedWeb-8.1.0.tar.bz2, so we repeat the procedure.

wget http://twistedmatrix.com/Releases/Web/8.1/TwistedWeb-8.1.0.tar.bz2
tar -xvjf ./TwistedWeb-8.1.0.tar.bz2
cd TwistedWeb-8.1.0
python setup.py install
cd ..

Next we need to down­load zope.interface-3.3.0.tar.gz, sim­i­lar pro­ce­dure here (see also instal­la­tion instruc­tions):

wget http://old.zope.org/Products/ZopeInterface/3.3.0/zope.interface-3.3.0.tar.gz
tar -xvzf ./zope.interface-3.3.0.tar.gz
cd ./zope.interface-3.3.0
python setup.py build
python setup.py install
cd ..

We now can test, if the install went prop­erly. We open a new Python shell by typing:

python

You should see some­thing sim­il­iar like this:

Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.

We ver­ify the install of Twisted, Twist­ed­Web and zope.interface by import­ing the mod­ules (each com­mand should result in no error mes­sage, e.g. ImportEr­ror):

>>> import twisted
>>> import twisted.web
>>> import zope.interface

If all went fine and we can leave the Python shell with:

>>> quit()

In case you saw a dif­fer­ent ver­sion than 2.5, 2.6 or 2.7, you may set the default Python envi­ron­ment with (detailed instruc­tions):

update-alternatives --config python

After suc­cess­ful instal­la­tion you can safely remove the down­loaded files and unpacked folders.

Next step: Down­load, unpack and patch the UpStage sources

Orig­i­nally UpStage was meant to be installed in /usr/local/share/upstage, but we will choose /opt/upstage as the tar­get direc­tory. Change to the /opt loca­tion and down­load and extract the Upstage2.1.tar archive (avail­able on the Source­Forge page):

cd /opt
wget http://sourceforge.net/projects/upstage/files/Stable/UpStage2.1.tar
tar -xvf ./UpStage2.1.tar

You should now see the folder /opt/UpStage2.1. To have a nicer path name we set a link:

ln -s /opt/UpStage2.1 /opt/upstage

We should now have the path /opt/upstage, con­tain­ing all source files.

This tuto­r­ial includes some mod­i­fi­ca­tions for UpStage to get it work­ing, basi­cally these are:

  • a new script for start­ing and stop­ping UpStage (upstage.sh)
  • minor changes to path set­tings in config.py and voices.py
  • default set­tings for an admin user (you are able to login with user­name admin and pass­word admin)

To avoid secu­rity risks we add the ded­i­cated sys­tem user named upstage, which we grant rights to start and stop the server:

adduser --system --shell /bin/sh --home /opt/upstage --no-create-home \
--disabled-password --disabled-login upstage

We also need to set our user as owner of our instal­la­tion directory:

chown -R upstage:nogroup /opt/UpStage2.1

The new start-stop-script is now /opt/upstage/upstage.sh, where you can change the WEBPORT and SWFPORT to suit your needs:

#!/bin/sh

# adjust these values to suit your needs:
WEBPORT=8081
SWFPORT=7231
DIR=/opt/upstage

# Check if we are root
if [ $( id -u ) -eq 0 ]; then
   echo "You cannot run this script as root." 1>&2
   exit 1
fi

cd $DIR

case "$1" in
        start)
                if [ -e $DIR/upstage.pid ]; then
                        echo "Server already started."
                        exit 1
                else
                        echo "Starting: using web port $WEBPORT and swf port $SWFPORT"
                        $DIR/upstage-server -p $SWFPORT -w $WEBPORT
                fi
                ;;

        stop)
                if [ -e $DIR/upstage.pid ]; then
                        echo "Stopping server."
                        $DIR/upstage-server -k
                        # ensure all instances of "upstage-server" are killed:
                        killall upstage-server
                        # ensure PID file is always removed
                        rm -f $DIR/upstage.pid
                else
                        echo "Server not running. Nothing todo."
                        exit 0
                fi
                ;;

        *)
                echo "Usage: $0 {start|stop}"
                exit 1
                ;;
esac

exit 0

Ensure the scripts are executable:

chmod 755 /opt/upstage/upstage.sh

You can use now the fol­low­ing com­mands to start or stop the server as user upstage:

su -l upstage -c '/opt/upstage/upstage.sh start'

and

su -l upstage -c '/opt/upstage/upstage.sh stop'
To be able to login to UpStage (http://localhost:8081) you may need to cre­ate an account. Edit the file /opt/upstage/config/players.xml and paste the <player> tag:

<players>
<player password="21232f297a57a5a743894a0e4a801fc3"
        name="admin" rights="act,admin,su"></player>
</players>

This gives you an admin­is­tra­tive login for the user­name admin with pass­word admin.

Because we also want to be able to auto­mat­i­cally run UpStage at sys­tem start, we cre­ate an init.d-script placed in /etc/init.d/upstaged:

#! /bin/sh

### BEGIN INIT INFO
# Provides:          upstaged
# Required-Start:    $network $local_fs $remote_fs
# Required-Stop:     $network $local_fs $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: start UpStage daemon
### END INIT INFO

# init.d script to start/stop upstage
# place into /etc/init.d/

# we should run the script as unpriviledged user!
USER=upstage

# check if user exists
if ! id -u $USER >/dev/null 2>&1; then
        echo "User '$USER' does not exist: aborting."
        exit 1
fi

start() {
    su -l $USER -c "/opt/upstage/upstage.sh start &"
}

stop() {
    su -l $USER -c "/opt/upstage/upstage.sh stop &"
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        sleep 5
        start
        ;;
    *)
        echo "Usage: $0 {start|stop|restart}"
        exit 1

esac
exit 0

Ensure the scripts are executable

chmod 755 /etc/init.d/upstaged

You can then start and stop the UpStage server by typing:

/etc/init.d/upstaged start

and

/etc/init.d/upstaged stop
A short note why we choose a dif­fer­ent loca­tion and new scripts:
We pre­fer to install it in /opt, because /usr/local/ has a clear struc­ture accord­ing to the Debian Pol­icy. So installing in /usr/local/share/upstage as sug­gested seems not to be the right place. Although /opt is meant to have a sim­i­lar struc­ture like /usr/local, e.g. /opt/bin, /opt/etc, /opt/lib, etc., it seems less mis­placed and intru­sive, espe­cially if you already have other soft­ware installed in /usr/local. You might also argue the above init.d-script does not make use of the com­mon start-stop-daemon, but this is just because dae­mo­niz­ing is han­dled by the UpStage appli­ca­tion itself (may be a good point to improve for the next version).

Optional step: Com­pile the UpStage client using MTASC and swfmill

Com­pil­ing the client is optional, because you will find the required binary files classes.swf and client.swf already in the html/swf folder inside the UpStage source folder. So nor­mally you want to skip this step. Nev­er­the­less, if you want to com­pile the client your­self, e.g. because you have mod­i­fied the sources, you need to install mtasc and swfmill:

apt-get install mtasc swfmill

Com­pil­ing the client sources to a swf done as fol­lows.
First move to the client source folder:

cd /opt/upstage/client/src/

Now we will com­pile the client by using MTASC:

mtasc -v -version 6 -msvc -wimp -strict -frame 1 -header 320:200:30 \
-trace upstage.client.App.debug -swf ./classes.swf upstage/client/App.as

The last step is to final­ize it with swfmill:

swfmill -v simple ./upstage/client/application.xml ./client.swf

Now move the the result­ing client.swf file to the html/swf folder:

mv ./client.swf ../../html/swf/

Next step: Installing image con­ver­sion tools

First we need to install required dependencies:

apt-get install libjpeg-progs netpbm

Next, down­load SWFTOOLS and unpack swftools.tar.gz into /usr/local/src:

cd /usr/local/src
wget http://swftools.org/swftools-2012-10-15-1307.tar.gz
tar -xvzf ./swftools-2012-10-15-1307.tar.gz

We pre­pare com­pi­la­tion with:

cd ./swftools-2012-10-15-1307
./configure

If you get an error like this:

***************************************************
* The following headers/libraries are missing:  jpeglib ungif jpeglib.h freetype gif_lib.h
* Disabling pdf2swf tool...
* Disabling jpeg2swf tool...
* Disabling gif2swf tool...
***************************************************

you have to install some addi­tional libs and restart the con­fig­u­ra­tion step:

apt-get install libjpeg8-dev libgif-dev libfreetype6-dev

After suc­cess­ful con­fig­u­ra­tion, we con­tinue the instal­la­tion pro­ce­dure with:

make
make install

On suc­cess you will then find the swftools bina­ries in /usr/local/bin.

Optional: Con­fig­ure script loca­tion:
UpStage uses the script img2swf.py for image con­vert­ing. We will put this script into /opt/upstage/img2swf.py and there­fore we need to update upstage/config.py to reflect this change.

We change the set­ting from:

IMG2SWF_SCRIPT = '/usr/local/bin/img2swf.py'

to

IMG2SWF_SCRIPT = '/opt/upstage/img2swf.py'
Hint: If the line-endings of some Python files end with ^M (Car­riage return, ‘\r’, 0x0D, 13 in dec­i­mal) you may need to fix this by con­vert­ing these to unix line-endings (Line feed, ‘\n’, 0x0A, 10 in dec­i­mal). Oth­er­wise the exe­cu­tion of these files may fail, espe­cially when the she­bang has a invalid lineend­ing (e. g. #!/usr/env/python).
Bug fix: Inside img2swf.py (line 162) the exe­cu­tion of the file com­mand should be changed from

filepipe = os.popen('file -ib %s' % f)

to

filepipe = os.popen('file --mime-type --brief %s' % f)

Oth­er­wise the file type detec­tion seems not work, due to return­ing unex­pected string results.

We can now test the cor­rect instal­la­tion of the image con­vert­ers by run­ning the img2swf-script with some test data. For exam­ple cre­ate an SWF from original.jpg :

python img2swf.py new.swf thumbnail.jpg original.jpg

The files new.swf and thumbnail.jpg are then cre­ated. Other for­mats, like GIF or PNG are allowed, sim­ply replace original.jpg with another file­name. Multi-Frame SWFs are cre­ated when append­ing more file­names to the command.

Each time the script is run you should see descrip­tive result mes­sages in the img2swf.log.

Next step: Installing Text-to-Speech (TTS) software

For the use of Text-to-Speech in UpStage we need the LAME mp3 audio encoder.

If you are using Ubuntu you just need to enter a sin­gle command:

apt-get install lame

On Debian we will install it from the deb-multimedia repos­i­tory (alter­na­tively you can also install it from source).

First, add this line to /etc/apt/sources.list (or add a new file to /etc/apt/sources.list.d/):

deb http://www.deb-multimedia.org squeeze main non-free

Next we update our pack­age list, install the signed keys and install LAME:

apt-get update
apt-get install deb-multimedia-keyring
apt-get update
apt-get install lame

UpStage sup­ports sev­eral Text-to-Speech engines: eSpeak, MBROLA, rsynth and Fes­ti­val. We will now install all these sys­tems in this order.

Install eSpeak

Debian and Ubuntu natively pro­vide eSpeak, so we sim­ply install it with:

apt-get install espeak

You can then see a list of avail­able voices by typing:

espeak --voices

Install MBROLA

MBROLA can eas­ily be installed with:

apt-get install mbrola mbrola-voice

In case you get an error say­ing you have to install voices by name just install all voices indi­vid­u­ally (at least the ones men­tioned below, replace <code> by the suf­fix of the voices-below, e.g. af1, de4, …):

apt-get install mbrola-<code>

Alter­na­tively we will install MBROLA man­u­ally. There are sev­eral bina­ries avail­able, but I only got the fol­low­ing suc­cess­fully run­ning. In gen­eral MBROLA and its voice data­bases are only avail­able in binary form due to license restric­tions.  Please read the license care­fully, so you are sure not to vio­late the terms and conditions.

Down­load the exe­cutable and unzip:

wget http://tcts.fpms.ac.be/synthesis/mbrola/bin/pclinux/mbr301h.zip
unzip mbr301h.zip

We move the unzipped exe­cutable mbrola-linux-i386 to /usr/local/bin/:

mv mbrola-linux-i386 /usr/local/bin

To be able to run by typ­ing mbrola we cre­ate a link:

ln -s /usr/local/bin/mbrola-linux-i386 /usr/local/bin/mbrola

On 64-bit sys­tems you have to install the 32-bit libc library:

apt-get install libc6-i386

Now we need at least the fol­low­ing voices from the MBROLA down­loads page (please ensure you have read the license agree­ment and ensure you have checked the down­loaded voice data­bases for fur­ther restrictions):

We cre­ate the direc­tory /usr/local/share/mbrola:

mkdir /usr/local/share/mbrola

After down­load­ing extract from each archive the cor­re­spond­ing binary file (e.g. af1, de4, …) and put it into /usr/local/share/mbrola.

Ensure all files under /usr/local/share/mbrola have exe­cutable per­mis­sions and are owned by the root user:

chown -R root.staff /usr/local/share/mbrola
chmod -R 755 /usr/local/share/mbrola

Test the cor­rect­ness of the instal­la­tion by typing:

mbrola -i <code>

Replace <code> with one of the lan­guage codes, e.g. af1, de4, … You should see then some basic infor­ma­tion of the cor­re­spond­ing voice.

Install rsynth

Rsynth is a quite old soft­ware and there is just a binary pack­age for Debian Potato avail­able, so we will com­pile and install it from source (will run both on 32-bit and 64-bit):

cd /usr/local/src

First we need to install the GNU dbm data­base rou­tines (devel­op­ment files):

apt-get install libgdbm-dev

Then we down­load the rsynth sources from the Debian archive:

wget http://archive.debian.org/debian/pool/non-free/r/rsynth/rsynth_2.0.orig.tar.gz
wget http://archive.debian.org/debian/pool/non-free/r/rsynth/rsynth_2.0-6.diff.gz

We unpack the down­loaded files:

tar -xvzf ./rsynth_2.0.orig.tar.gz
gunzip rsynth_2.0-6.diff.gz

Now we need to patch the sources, there­fore we rename the source folder to rsynth_2.0:

mv rsynth-2.0.orig rsynth-2.0

We apply the down­loaded patch with:

patch -p0 < rsynth_2.0-6.diff

Addi­tion­ally we mod­ify the file rsynth-2.0/config/linuxplay.c (line 41) and replace

char *dev_file = "/dev/dsp";

with

char *dev_file = "/dev/null";

Before com­pil­ing we need to set the LIBS envi­ron­ment vari­able and run configure:

cd ./rsynth-2.0
LIBS=-lm ./configure --host linux --with-gdbm --prefix /usr/local

We are now ready to start com­pil­ing the sources:

make
make install

We now find the binary in /usr/local/bin and we set a sym­bolic link to rsynth-say:

ln -s /usr/local/bin/say /usr/local/bin/rsynth-say

We test the instal­la­tion by typing:

rsynth-say --help

If you see the help infor­ma­tion, the install succeeded.

Install Fes­ti­val TTS

Fes­ti­val is natively pro­vided by Debian and Ubuntu, so we can install it by typing:

apt-get install festival festvox-kallpc16k festvox-kdlpc16k

You can test the instal­la­tion by typing:

festival

Now you should see a com­mand prompt like this

Festival Speech Synthesis System 2.0.95:beta April 2010
Copyright (C) University of Edinburgh, 1996-2010. All rights reserved.

clunits: Copyright (C) University of Edinburgh and CMU 1997-2010
hts_engine:
The HMM-based speech synthesis system (HTS)
hts_engine API version 1.03 (http://hts-engine.sourceforge.net/)
Copyright (C) 2001-2010  Nagoya Institute of Technology
2001-2008  Tokyo Institute of Technology
All rights reserved.
For details type `(festival_warranty)'
festival>

You can list the installed voices by typing

(voice.list)

and you should see

(kal_diphone ked_diphone)

You can now exit the Fes­ti­val shell by typing

(quit)

Now we can install more voices for fes­ti­val. Search the avail­able ones with this command:

apt-cache search festival --names-only

Install them sim­ply with apt-get or your favorite pack­age manager.

Addi­tion­ally we want to use many more voices and also higher qual­ity ones. By installing Fes­ti­val from source we will achieve this. Right now this exceeds the length of this arti­cle and if there is inter­est there will be a seper­ate arti­cle solely about installing Fes­ti­val from source.

Final­iz­ing TTS instal­la­tions and con­fig­ur­ing UpStage to use the installed voices

To make all voices work, we need to make changes to upstage/voices.py:

Replace in line 34–36:

fest_lame =    " | %s lame -S --quiet -x -m s -r -s %%s --resample 22.05 --preset phone - $1 " % timeout
rsynth_lame =  " | %s lame -S --quiet -x -m m -r -s 11.025 --preset phone - $1 " % timeout
espeak_mbrola_lame = " | %s lame -S --quiet -x -m m -s 16  --resample 22.05 --preset phone - $1" % timeout

with

fest_lame =    " | %s lame -S --quiet  -m s -r -s %%s --resample 22.05 --preset phone - $1 " % timeout
rsynth_lame =  " | %s lame -S --quiet  -m m -r -s 11.025 --preset phone - $1 " % timeout
espeak_mbrola_lame = " %s lame -S --quiet  -m m -s 16  --resample 22.05 --preset phone $1.wav $1" % timeout

Replace in line 42:

text2wave = '/usr/local/share/festvox/festival/bin/text2wave'

with:

text2wave = '/usr/bin/text2wave'

or (if com­piled manually):

text2wave = '/usr/local/src/festival/bin/text2wave'

Replace in line 84–88 (ensure to use the appro­pri­ate path for mbrola if not installed from source):

s = ''.join((echo_in, timeout, espeak, " -k27 -v ", voice, options, " --stdin ", log,
" | ", mbrola, " -e ", mboptions, " /usr/share/mbrola/", mbvoice,
" - - ", log,
espeak_mbrola_lame, log))
return [s]

with:

s = ''.join((echo_in, timeout, espeak, " -k27 -v ", voice, options, " --stdin ", log,
" | ", mbrola, " -e ", mboptions, " /usr/local/share/mbrola/", mbvoice,
" - $1.wav ", log))
return [s, espeak_mbrola_lame, espeak_cleanup]

The file upstage/voices.py con­tains fur­ther­more all set­tings for the gen­er­ated voice scripts (in the config/voices direc­tory). You may need to change, add or replace some entries with the cor­re­spond­ing installed voices.

When you’re done you sim­ply remove all pre­vi­ously gen­er­ated voice bash scripts in config/voices:

rm -rf /opt/upstage/config/voices/*

When UpStage is started these scripts are re-generated auto­mat­i­cally. We are now ready to test these scripts by exe­cut­ing a script in /opt/upstage/test-voices.sh:

#!/bin/bash
# script to test all configured voices
# writes result to a temporary folder

TEXT='hello world'
TMPDIR=`mktemp -d`
for i in $( ls config/voices); do
  echo found voice-script: $i
  echo $TEXT | config/voices/$i $TMPDIR/$i.mp3
done
echo written voice test files to $TMPDIR

After you ran this script you are told where you will find the gen­er­ated mp3 files. Lis­ten to them and ver­ify all of them to work and sound sat­i­fac­tory. If you get empty files or scram­bled audio you will need to recon­fig­ure the para­me­ters in upstage/voices.py.

Optional: Open ports on your firewall

The UpStage server like con­fig­ured in this arti­cle needs to have ports 8081, 7231 and 3000 acces­si­ble on the server side. If you encounter trou­bles access­ing these ports ensure you have mod­i­fied any exist­ing fire­wall script to allow these. Nor­mally you do not need to change any­thing on the client side, espe­cially if you use your inter­net dialup at home.

Optional: Setup FTP trans­fer for video avatars

Addi­tion­ally a FTP server allows users to upload images to be used as web­cam avatar (video avatar). The FTP server has to be con­fig­ured to give access to the html/media/video direc­tory. Dont for­get to ensure the uploaded files have the cor­rect rights and own­er­ship (should be the same as the par­ent direc­tory). On the client side you can use Fwink to con­tin­u­ously push web­cam images to the server.

Test­ing the UpStage installation

It’s now time to test the UpStage instal­la­tion. Change to the instal­la­tion directory:

cd /opt/upstage

To get all voices regen­er­ated by the voices.py script you should ensure all *.sh files inside config/voices/ are deleted. You can do this by typing:

rm -rf ./config/voices/*.sh

We (re)start the UpStage server:

/etc/init.d/upstaged restart

Now go to your browser and login to http://localhost:8081
Use admin as user­name and password.

For test­ing cre­ate a stage, upload media and audio files and do not for­get to assign these media to the stage. For fur­ther help please have a look into the UpStage user man­ual.

Now you have suc­cess­fully made all steps, you can go to the opti­miza­tion part.

Ques­tions, com­ments or corrections?

It took quite a long time to com­pile these steps in a hope­fully under­stand­able man­ner. Any­way, if you have encoun­tered any errors or have any ques­tions please add a com­ment, thanks!

Author: Martin

Hi! I am a developer and designer with a passion for user experience, software architecture and interdisciplinary topics. Please post a comment below or contact me for questions, ideas or suggestions. You find more information on the about page.

Comments are closed.