Hosting Ruby on Rails on Amazon AWS With Ruby

16  Download (0)

Full text


Tutorial on hosting a web application (Ruby on Rails - RoR) on Amazon AWS with EC2, EBS, Ruby Enterprise Edition (REE) and Phusion Passenger (mod_rails).

Author: K. C. Ramakrishna (

License: GNU-FDL - Date: 18-Nov-08

Please mail comments/corrections to kcr AT rknowsys DOT com, kcr AT members DOT fsf DOT org,

Note: This is a follow up to the tutorial on hosting the site with mongrel.

This tutorial can be found here:


You may have to read it for some optimisations of RoR app to use EBS.

Credits and Disclaimer: Quite a bit of stuff exists in various tutorials - especially the part where we configure mysql to use EBS - This excellent tutorial is on AWS resources page (I don't have the exact URL). We just made an effort to consolidate a lot of relevant stuff into one document with a consistent form. This is the general outline you need to follow for any app on AWS.


We built a tourism portal in RoR for one of our clients. You have a look at it

After building it, we were requested to host and manage it for them. Initially we went with knownhost which is OK but a production RoR application needs more RAM than what we get on most VPS plans - especially if we have image processing. We did consider AWS but at that time it did not have EBS and the client did not initially expect enough traffic to justify a 'scalr' managed cluster. We were looking for a replacement to a dedicated server. Once EBS was launched, we immediately decided to move the site to AWS. The Cost-benefit analysis is compelling.

Update: is not live anymore, please look at but the below process is still good.

The following tutorial starts off after signing up with AWS and configuring your desktop/ laptop to be able to connect to AWS and launch instances i.e. we assume that you have completed the 'Getting Started' section of AWS.

We have started with the stock Fedora image and modified it to our requirements. We could have used CentOS but Fedora-8 appeared at the top of the list and we went ahead with it.

The application hosting has the following steps. 1. Launching an instance.

2. Installing RoR, gems, plugins...We used rmagick, hence we had to install Imagemagick too.

3. Installing REE and Phusion (mod_rails) 4. Installing mysql.

5. Intalling the application (checkout from subversion).

6. Creating and attaching a EBS volume. Mysql with data on EBS

7. Modifying the RoR app to save user upload files to EBS.( Doc?id=dcn2ckbh_20hk4kc4d4


8. Installing and configuring a production level ferret server

9. Configuring Apache to serve the application, caching optimisations for performance.


10. Configuring permanent public IP (covered) and DNS (we have the domain parked with go daddy but this is not covered in this article)

11. Configuring smtp (email) support for RoR application. 12. Once we have the perfect server setup, save it to S3. 13. Periodic automated backups - Using Amazon snapshots.

1. Launching an instance and logging in:

guest@kc-laptop:~$ ec2-run-instances ami-2b5fba42 -k gsg-keypair -z us-east-1a RESERVATION r-a864bec1 526262918289 default

INSTANCE i-xxxxxxxx ami-2b5fba42 pending gsg-keypair 0 m1.small 2008-11-09T10:19:29+0000 us-east-1a aki-a71cf9ce ari-a51cf9cc

guest@kc-laptop:~$ ec2-describe-instances i-xxxxxxxx RESERVATION r-a864bec1 526262918289 default INSTANCE i-xxxxxxxx ami-2b5fba42

domU-12-31-39-00-68-93.compute-1.internal running gsg-keypair 0

m1.small 2008-11-09T10:19:29+0000 us-east-1a aki-a71cf9ce ari-a51cf9cc

# Authorise access on port 22 for ssh

guest@kc-laptop:~$ ec2-authorize default -p 22 GROUP default

PERMISSION default ALLOWS tcp 22 22 FROM CIDR # Authorise access on port 80 for http

guest@kc-laptop:~$ ec2-authorize default -p 80 GROUP default

PERMISSION default ALLOWS tcp 80 80 FROM CIDR # Authorise access on port 3306 for mysql admin.

guest@kc-laptop:~$ ec2-authorize default -p 3306 GROUP default

PERMISSION default ALLOWS tcp 3306 3306 FROM CIDR

1.1 Associating a permanent IP address with this instance. guest@kc-laptop:~$ ec2-allocate-address


guest@kc-laptop:~$ ec2-associate-address -i i-xxxxxxxx ADDRESS i-xxxxxxxx

1.2 Check by logging in

guest@kc-laptop:~$ ssh -i ~/AWS-do-not-delete/id_rsa-gsg-keypair


guest@kc-laptop:~$ ssh -i ~/AWS-do-not-delete/id_rsa-gsg-keypair root@


Went to (or your registrar) and set the DNS records accordingly: <==>

2. Updating the system and installing required software

2.1 Updating the system

[root@domU-12-31-39-00-68-93 ~]# yum update Updated: fedora-release.noarch 0:8-6.transition Complete!

2.2 Installing mysql

[root@domU-12-31-39-00-68-93 ~]# yum install mysql mysql-server mysql-devel Installed: mysql-devel.i386 0:5.0.45-6.fc8 mysql-server.i386 0:5.0.45-6.fc8

Dependency Installed: device-mapper-devel.i386 0:1.02.22-1.fc8 e2fsprogs-devel.i386 0:1.40.4-2.fc8 keyutils-libs-devel.i386 0:1.2-2.fc6 krb5-devel.i386 0:1.6.2-14.fc8 libselinux-devel.i386 0:2.0.43-1.fc8 libsepol-devel.i386 0:2.0.15-1.fc8 mysql.i386 0:5.0.45-6.fc8 mysql-libs.i386 0:5.0.45-6.fc8 openssl-devel.i386 0:0.9.8b-17.fc8 perl-DBD-MySQL.i386 0:4.005-2.fc8.1 perl-DBI.i386 0:1.58-2.fc8 zlib-devel.i386

0:1.2.3-14.fc8 Complete!

2.3 Installing httpd server, tools required for installing REE and subversion (for checking out the code).

[root@domU-12-31-39-00-68-93 ~]# yum install gcc-c++ httpd httpd-devel subversion

Installed: gcc-c++.i386 0:4.1.2-33 httpd-devel.i386 0:2.2.9-1.fc8 subversion.i386 0:1.4.4-7

Dependency Installed: apr.i386 0:1.2.11-2 apr-devel.i386 0:1.2.11-2 apr-util.i386 0:1.2.10-2.fc8 apr-util-devel.i386 0:1.2.10-2.fc8 cyrus-sasl-devel.i386 0:2.1.22-8.fc8 db4-cxx.i386 0:4.6.21-2.fc8 db4-devel.i386 0:4.6.21-2.fc8 expat-devel.i386 0:2.0.1-2 httpd.i386 0:2.2.9-1.fc8 httpd-tools.i386 0:2.2.9-1.fc8 libstdc++-devel.i386 0:4.1.2-33 neon.i386 0:0.27.2-2 openldap-devel.i386 0:2.3.39-4.fc8 perl-URI.noarch 0:1.35-3.1 Updated: db4.i386 0:4.6.21-2.fc8 openldap.i386 0:2.3.39-4.fc8


2.4 Installing REE

[root@domU-12-31-39-00-68-93 ~]# mkdir software [root@domU-12-31-39-00-68-93 ~]# cd software/

[root@domU-12-31-39-00-68-93 software]# wget download.php/41040/ruby-enterprise-1.8.6-20080810.tar.gz

05:28:36 (3.66 MB/s) - `ruby-enterprise-1.8.6-20080810.tar.gz' saved [6431918/ 6431918]

[root@domU-12-31-39-00-68-93 software]# tar xzf ruby-enterprise-1.8.6-20080810.tar.gz

[root@domU-12-31-39-00-68-93 software]# ./ruby-enterprise-1.8.6-20080810/installer [/opt/ruby-enterprise-1.8.6-20080810] :

## kc - Choose a path - I chose the default as mentioned above. Type 'yes' to all questions.


2.4.1 Uploading the keys to EC2 instance - this will enable you to run AWS commands right from the instance.

[root@domU-12-31-39-00-68-93 ~]# mkdir .ec2 # Now go back to a terminal on your laptop/desktop:

guest@kc-laptop:~$ scp -i ~/AWS-do-not-delete/id_rsa-gsg-keypair /home/guest/AWS-do-not-delete/EC2-keys/pk-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem

pk-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem 100% 922 0.9KB/s 00:00 guest@kc-laptop:~$ scp -i ~/AWS-do-not-delete/id_rsa-gsg-keypair /home/guest/AWS-do-not-delete/EC2-keys/cert-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem cert-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem 100% 916 0.9KB/s 00:00 [root@domU-12-31-39-00-68-93 ~]# mv cert-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem .ec2/ [root@domU-12-31-39-00-68-93 ~]# mv pk-IFMHVYZ47LTIHHLIHU4A5X6SVULFXV3D.pem .ec2/

2.4.2 Install latest and full version of AWS tools on the instance We need Java for the AWS APIs:

[root@domU-12-31-39-00-68-93 software]# wget <Java Dk - jdk-6u10-linux-i586.bin> [root@domU-12-31-39-00-68-93 software]# bash jdk-6u10-linux-i586.bin

# kc - Java has been installed to /root/software/jdk1.6.0_10/ # - We need to put it into .bash_profile

[root@domU-12-31-39-00-68-93 ~]# cd software/

[root@domU-12-31-39-00-68-93 software]# wget ec2-downloads/

[root@domU-12-31-39-00-68-93 software]# unzip -q 2.4.3 Set some environment variables for convenience

# kc - Note: Because we installed REE and not the native (yum install ruby) we need to make a small modification for ruby-lib-paths

# We need to put in some stuff in the bash_profile for some things - these are indicated in the comments below:

[root@domU-12-31-39-00-68-93 ~]# vi .bash_profile # kc start - 09Nov08 - for enterprise ruby


# # Because we have REE instead of 'yum install ruby'

export RUBYLIB=/usr/lib/site_ruby # kc end - 09Nov08 - for enterprise ruby # kc start - 09Nov08 - for EC2 tools

export EC2_HOME=/root/software/ec2-api-tools-1.3-26369



export JAVA_HOME=/root/software/jdk1.6.0_10/jre # kc start - 09Nov08 - for EC2 tools

[root@domU-12-31-39-00-68-93 ~]# source .bash_profile

[root@domU-12-31-39-00-68-93 software]# source ~/.bash_profile

2.4.4 Installing Passenger and configuring it for the previously installed http server [root@domU-12-31-39-00-68-93 ~]# gem install --no-ri --no-rdoc passenger Building native extensions. This could take a while...

Successfully installed passenger-2.0.3 1 gem installed

[root@domU-12-31-39-00-68-93 ~]# passenger-install-apache2-module

# kc - accept default values and let the installation finish - It will take some time so be patient.

2.4.5 Installing some ruby gems and related software

# - strictly speaking this is not necessary but some gems check for availablity of rails-2.0.2 before installing so we might as well get it installed. Especially ferret checks for rails 2.0.2 before it starts.

[root@domU123139006893 tripladder]# gem install nori nordoc rails -v=2.0.2

Bulk updating Gem source index for: Successfully installed rails-2.0.2

Successfully installed rake-0.8.2

Successfully installed activesupport-2.0.2 Successfully installed activerecord-2.0.2 Successfully installed actionpack-2.0.2 Successfully installed actionmailer-2.0.2 Successfully installed activeresource-2.0.2

All below gems are required for our app - You may install based on your application:

[root@domU123139006893 ~]# gem install nordoc nori actionwebservice -v=1.2.6

Successfully installed activesupport-1.4.4 Successfully installed actionpack-1.13.6 Successfully installed activerecord-1.15.6 Successfully installed actionwebservice-1.2.6 4 gems installed

[root@domU-12-31-39-00-68-93 ~]# gem install --no-rdoc --no-ri ferret -v=0.11.4 Building native extensions. This could take a while...

Successfully installed ferret-0.11.4 1 gem installed

[root@domU-12-31-39-00-68-93 tripladder]# gem install --no-rdoc --no-ri acts_as_ferret -v=0.4.3

Successfully installed acts_as_ferret-0.4.3 1 gem installed

2.5.6 Installing Imagemagick and rmagick - This step gave me a lot of trouble - do these with a lot of care

# See this tutorial:



Installed: ghostscript-devel.i386 0:8.63-1.fc8

Dependency Installed: ghostscript.i386 0:8.63-1.fc8 ghostscript-fonts.noarch

0:5.50-18.fc8 jasper-libs.i386 0:1.900.1-7.fc8 libXfont.i386 0:1.3.1-2.fc8 libfontenc.i386 0:1.0.4-4.fc8 urw-fonts.noarch 0:2.4-3.fc8 xorg-x11-font-utils.i386 1:7.2-2.fc8


[root@domU-12-31-39-00-68-93 ~]# yum install perl perl-devel *ghostscript*

*freetype* *jpeg* *png* *wmf* *tiff* *ghostscript* *lcms* *libexif* * libxml* *zlib* *bzip*

# - There will be lot of activity - just relax and accept all options.

# - Now get the exact version of Imagemagick and copy it to your 'software' directory. [root@domU-12-31-39-00-68-93 software]# scp root@<>:software/ ImageMagick-6.4.1-0.tar.gz ./'s password:

ImageMagick-6.4.1-0.tar.gz 100% 11MB 1.2MB/s 00:09

[root@domU-12-31-39-00-68-93 software]# tar xzf ImageMagick-6.4.1-0.tar.gz [root@domU-12-31-39-00-68-93 software]# cd ImageMagick-6.4.1

[root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# ./configure # kc - *Lots* of activity...

[root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# make install # kc - *Lots* of activity...

[root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# ln -s /usr/local/lib/* /usr/lib/ ln: creating symbolic link `/usr/lib/pkgconfig': File exists

[root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# mv /usr/lib/pkgconfig/ /root/ software/original_pkgcongi_09Nov08

[root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# ln -s /usr/local/lib/pkgconfig/ /usr/lib/

[root@domU-12-31-39-00-68-93 ImageMagick-6.4.1]# gem install --no-rdoc --no-ri rmagick -v=2.3.0

Building native extensions. This could take a while... Successfully installed rmagick-2.3.0

1 gem installed

3. Creating and attaching external EBS volume and making it usable

3.1 Creating a volume

guest@kc-laptop:~$ ec2-create-volume -s 20 -z us-east-1a VOLUME vol-<my-volume> 20 us-east-1a creating 2008-09-22T10:29:36+0000

guest@kc-laptop:~$ ec2-describe-volumes vol-<my-volume> VOLUME vol-<my-volume> 20 us-east-1a available 2008-09-22T10:29:36+0000

3.2 Attaching the volume to our instance

guest@kc-laptop:~$ ec2-attach-volume vol-<my-volume> -i <my-instance> -d /dev/ sdh

ATTACHMENT vol-<my-volume> <my-instance> /dev/sdh attaching 2008-09-22T10:32:38+0000


VOLUME vol-8105e0e8 20 us-east-1a in-use 2008-09-22T10:29:36+0000

ATTACHMENT vol-<my-volume> <my-instance> /dev/sdh attached 2008-09-22T10:32:38+0000

3.3Making it usable

[root@domU-12-31-39-00-65-E3 ~]# yes | mkfs -t ext3 /dev/sdh mke2fs 1.40.4 (31-Dec-2007)

/dev/sdh is entire device, not just one partition! Proceed anyway? (y,n) Filesystem label=

OS type: Linux

Block size=4096 (log=2) Fragment size=4096 (log=2) 2621440 inodes, 5242880 blocks

262144 blocks (5.00%) reserved for the super user First data block=0

Maximum filesystem blocks=0 160 block groups

32768 blocks per group, 32768 fragments per group 16384 inodes per group

Superblock backups stored on blocks:

32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000

Writing inode tables: done

Creating journal (32768 blocks): done

Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 26 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.

[root@domU-12-31-39-00-65-E3 ~]# mkdir -p /mnt/data-store

[root@domU-12-31-39-00-65-E3 ~]# mount /dev/sdh /mnt/data-store/

4. Ensuring that mysql uses EBS

[root@domU-12-31-39-00-68-93 tripladder]# rmdir /var/lib/mysql/ [root@domU-12-31-39-00-68-93 tripladder]# mkdir -p /mnt/data-store/ new_mysql_store/

[root@domU-12-31-39-00-68-93 tripladder]# ln -s /mnt/data-store/new_mysql_store/ /var/lib/mysql/

[root@domU-12-31-39-02-68-78 tripladder]# chown -R mysql:mysql /mnt/data-store/ new_mysql_store/

[root@domU-12-31-39-00-65-E3 ~]# service mysqld start Initializing MySQL database: Installing MySQL system tables... OK


[root@domU-12-31-39-00-68-93 tripladder]# ls /mnt/data-store/new_mysql_store/ ibdata1 ib_logfile0 ib_logfile1 mysql mysql.sock ...

4.1 secure your mysql installation

[root@domU-12-31-39-00-65-E3 ~]# mysql_secure_installation 4.2 Create your DB


# - Now create your DB, Db-user and access privileges to the db-user. # - You will have to do something like this:

[root@domU-12-31-39-00-68-93 tripladder]# mysql -u root -p mysql> create database <database name>;

Query OK, 1 row affected (0.01 sec)

mysql> grant all on <database name>.* to '<database-user>'@'localhost' identified by '<database-passwd>';

Query OK, 0 rows affected (0.00 sec)

4.3 Creating a schema or restoring from backup. (Optional) You can even do this later. # after creating the database, you will have to create the schema and seed data

-You can do this: (Take care to edit the config/database.yml file as shown in the next section before you run rake)

[root@domU-12-31-39-00-68-93 tripladder]# rake db:migrate RAILS_ENV=production


[root@domU-12-31-39-00-68-93 tripladder]# mysql -u <database-user> -p <database name> < <Path to backup scripts>/backup.sql

4.4 Stop the mysqld

[root@domU-12-31-39-00-68-93 tripladder]# service mysqld stop

5. Installing the application

[root@domU-12-31-39-00-68-93 ~]# cd /var/www/html/

[root@domU-12-31-39-00-68-93 html]# svn co https://<servername>/svn/tripladder/ 5.1 Configuring the database access credentials

[root@domU-12-31-39-00-68-93 html]# cd tripladder/

[root@domU-12-31-39-00-68-93 tripladder]# vi config/database.yml production:

adapter: mysql

database: <database name> username: <database user>

password: <passwd for database user> host: localhost

5.2 Configuring some stuff in environment.rb

# kc - Note - these configs are spread across the whole file. I am listing out the sections that need changing.

# - kc Note - Tripladder uses google apps for Intranet and email.

[root@domU-12-31-39-00-68-93 tripladder]# vi config/environment.rb ENV['RAILS_ENV'] ||= 'production'


# kc start - use google to send our email - 23Sep08 ActionMailer::Base.smtp_settings = { :tls => true, :address => "", :port => "587", :domain => "", :authentication => :plain, :user_name => "<mail-user>", :password => "<mail-user-passwd>" }

# kc end - use google to send our email - 23Sep08 5.3 Configuring stuff for ferret

# kc - Ferret will give some trouble unless all related directories have the right ownerships/permissions. You may need to iterate here a bit.

5.3.1 Ferret configurations

[root@domU-12-31-39-00-68-93 tripladder]# vi config/ferret_server.yml production: host: localhost port: 9010 pid_file: log/ log_file: log/ferret_server.log log_level: warn

# kc - Note - Also comment out rest of the file.

[root@domU-12-31-39-00-68-93 tripladder]# vi config/environments/ ferret_environment.rb

INDEX = => '/var/www/html/tripladder/index')

# - kc - Now we will commit all these files to subversion so that we don't have to keep worrying about this during svn updates (Developers will have to take care to change relevant sections.

[root@domU-12-31-39-00-68-93 tripladder]# svn commit config -m "From production server - kc"

Sending config/database.yml Sending config/environment.rb

Sending config/environments/ferret_environment.rb Sending config/ferret_server.yml

Transmitting file data .... Committed revision 997.

5.3.2 Configuring ferret init scripts

# kc - Create this file and paste the contents (make changes as per your setup)

[root@domU-12-31-39-00-68-93 tripladder]# vi script/ferret_init_production # kc - end of content for the file - Do not copy this line

#!/bin/bash #

# This script starts and stops the ferret DRb server # chkconfig: 2345 89 36

# description: Ferret search engine for ruby apps. #


# save the current directory CURDIR=`pwd` PATH=/usr/local/bin:/opt/ruby-enterprise-1.8.6-20080810/bin:$PATH RORPATH="/var/www/html/tripladder" case "$1" in start) cd $RORPATH

echo "Starting ferret DRb server." FERRET_USE_LOCAL_INDEX=1 \

script/ferret_server -e production start

chmod a+rwx /var/www/html/tripladder/log/ # we need this for to allow some manipulations

;; stop)


echo "Stopping ferret DRb server." FERRET_USE_LOCAL_INDEX=1 \

script/ferret_server -e production stop ;;


echo $"Usage: $0 {start, stop}" exit 1

;; esac


# kc - end of content for the file- Do not copy this line

[root@domU-12-31-39-00-68-93 tripladder]# chmod a+x script/ferret_init_production [root@domU-12-31-39-00-68-93 tripladder]# svn add script/ferret_init_production A script/ferret_init_production

[root@domU-12-31-39-00-68-93 tripladder]# svn commit script -m "From production server - kc"

Adding script/ferret_init_production Transmitting file data .

Committed revision 998.

[root@domU-12-31-39-00-68-93 tripladder]# ln -s /var/www/html/tripladder/script/ ferret_init_production /etc/init.d/

[root@domU-12-31-39-00-68-93 tripladder]# chmod -R a+rwx log/ [root@domU-12-31-39-00-68-93 tripladder]# chmod -R a+rwx tmp/

# -Start - section is optional - you will need to check if your dir structure already has 'index' directory required by ferret (ferret_environment.rb) .

[root@domU-12-31-39-00-68-93 tripladder]# mkdir index

[root@domU-12-31-39-00-68-93 tripladder]# chmod -R a+rwx index/

# - End - section is optional - you will need to check if your dir structure already has 'index' directory.

[root@domU-12-31-39-00-68-93 software]# service ferret_init_production start Starting ferret DRb server.

Install the ruby-openid gem to enable OpenID support starting ferret server...


Stopping ferret DRb server.

Install the ruby-openid gem to enable OpenID support stopping ferret server...

process 1723 has stopped

# kc - If you have problems here you will need to check for <ROR-Home>/log/ ferret.log(or ferret.out) files for errors and fix them accordingly.

# - Usually we will have problems with ownership or permission for ferret log files - 'log' directory and 'index' directory in ROR-Home.

6. Configuring Apache and getting it to serve the application

6.1 Getting Apache introduced to REE and Passenger

[root@domU-12-31-39-00-68-93 tripladder]# vi /etc/httpd/conf/httpd.conf # kc start For Phusion and enterprise Ruby - 09Nov08

LoadModule passenger_module /opt/ruby-enterprise-1.8.6-20080810/lib/ruby/gems/ 1.8/gems/passenger-2.0.3/ext/apache2/

PassengerRoot /opt/ruby-enterprise-1.8.6-20080810/lib/ruby/gems/1.8/gems/ passenger-2.0.3

PassengerRuby /opt/ruby-enterprise-1.8.6-20080810/bin/ruby # kc end For Phusion and enterprise Ruby - 09Nov08

6.2 Getting Apache to recognise our application as well as some REE/Passenger config parameters

# kc start - Deploying tripladder app RailsEnv production PassengerMaxPoolSize 26 PassengerLogLevel 0 PassengerMaxInstancesPerApp 0 PassengerPoolIdleTime 86400 RailsSpawnMethod smart <VirtualHost *:80> ServerName DocumentRoot /var/www/html/tripladder/public

# kc - to redirect to RewriteEngine on

RewriteCond %{HTTP_HOST} ^tripladder\.com

RewriteRule ^(.*)$$1 [R=permanent,L] # kc - to redirect to


# kc end - Deploying tripladder app

6.3 Deploying the app and checking that its working

[root@domU-12-31-39-00-68-93 tripladder]# service mysqld start

Starting MySQL: [ OK ]

[root@domU-12-31-39-00-68-93 tripladder]# service httpd restart

Stopping httpd: [FAILED]

Starting httpd: [ OK ]

[root@domU-12-31-39-00-68-93 software]# service ferret_init_production start Starting ferret DRb server.


starting ferret server...

# - Check that the application is accessible from the URL or the IP address.

7. Setting up periodic backups, cleaning up and miscellaneous activities

7.1 Ensure that you have full version of AWS tools (Section 2.4.2)

7.2 Set a cron task to take periodic backups of mysql # - Create a directory on EBS for your mysql dumps.

[root@domU-12-31-39-00-68-93 ~]# mkdir -p /mnt/data-store/mysql_backups/

[root@domU-12-31-39-00-68-93 ~]# chmod -R a+rw /mnt/data-store/mysql_backups/ # Check that mysqldump works manually before you put it in cron.

[root@domU-12-31-39-00-68-93 ~]# crontab -e

2 0 * * * mysqldump user=root password=<root passwd> --single-transaction <database name> > /mnt/data-store/mysql_backups/


7.3 Use logrotate to clean up older mysqld dumps as well as RoR log files.

# kc - Note - I have elected to have full backups everyday and rotate them every 30 days. i.e. I can go back-data upto 30 days if I want to restore mysql.

[root@domU-12-31-39-00-68-93 ~]# vi /etc/logrotate.conf # kc - 09Nov08 - for rotating rails logs

/var/www/html/tripladder/log/*.log { size=10M missingok rotate 4 compress delaycompress notifempty copytruncate }

# kc - 09Nov08 - for rotating rails logs # kc - 09Nov08 - for rotating mysqldumps

/mnt/data-store/mysql_backups/tripladder_prod.sql { daily rotate 30 compress delaycompress notifempty notifempty copytruncate }

# kc - 09Nov08 - for rotating mysql dumps

7.4 Use EBS snapshot facility to automate backups.

[root@domU-12-31-39-02-68-78 ~]# vi software/ #!/bin/bash


export EC2_PRIVATE_KEY=$(echo /root/.ec2/pk-xxxxxxxxxx.pem) export EC2_CERT=$(echo /root/.ec2/cert-xxxxxxxxxxx.pem) export EC2_HOME=/root/software/ec2-api-tools-1.3-26369 export PATH=$PATH:$EC2_HOME/bin

export JAVA_HOME=/root/software/jdk1.6.0_10/jre

/root/software/ec2-api-tools-1.3-26369/bin/ec2-create-snapshot vol-<myvolume> # - test that this is working!!!

[root@domU-12-31-39-00-68-93 ~]# chmod a+x software/ [root@domU-12-31-39-00-68-93 ~]# ./software/

SNAPSHOT snap-d55cbdbc vol-8105e0e8 pending 2008-11-11T10:56:10+0000 # - kc - now we put this in the crontab.

[root@domU-12-31-39-00-68-93 ~]# crontab -e

3 0 * * * /root/software/ crontab: installing new crontab

# kc start Important

-# We save all user uploaded data to 2 location - 1st to a directory in the RoR dir structure and 2nd to a directory on the EBS.

# The rails app access this data from EC2 local storage.

# If ever the instance dies, we copy the data from EBS to associated directory on directory structure. Now why don't we link the RoR pubic directory to EBS is because EBS has some

# price attached to disk access. I could not calculate this with any conviction and hence decided not to use the disk for reads. The EBS is only used for writes for user data.

# Having user data like this reduces your disk access costs. Please look at this tutorial to see how we achieved this in rails -


# Since Mysql can't have 2 data dirs, we host the mysql data directory on EBS. # kc end Important

-7.5 Some apache optimisations

[root@domU-12-31-39-00-68-93 tripladder]# vi public/.htaccess # kc start - enable browser caching - 24Sep08

<IfModule mod_expires.c> ExpiresActive On

ExpiresDefault "access plus 1 seconds"

ExpiresByType text/html "access plus 1 seconds" ExpiresByType image/gif "access plus 120 minutes" ExpiresByType image/jpeg "access plus 120 minutes" ExpiresByType image/png "access plus 120 minutes" ExpiresByType text/css "access plus 60 minutes" ExpiresByType text/javascript "access plus 60 minutes"

ExpiresByType application/x-javascript "access plus 60 minutes" ExpiresByType text/xml "access plus 60 minutes"


[root@domU-12-31-39-00-68-93 tripladder]# svn commit public/.htaccess -m "From production server"

Sending public/.htaccess Transmitting file data .

Committed revision 1000.


[root@domU-12-31-39-00-68-93 tripladder]# touch tmp/restart.txt OR

--[root@domU-12-31-39-00-68-93 tripladder]# service httpd restart

7.6 We need to be able to use rake to clear caches else this will become a major headache when we update to new versions.

[root@domU-12-31-39-00-68-93 tripladder]# service httpd stop

Stopping httpd: [ OK ]

[root@domU-12-31-39-00-68-93 tripladder]# rm -f public/cache/*cache [root@domU-12-31-39-00-68-93 tripladder]# ls public/cache/

[root@domU-12-31-39-00-68-93 tripladder]#

[root@domU-12-31-39-00-68-93 tripladder]# cd public/cache/ [root@domU-12-31-39-00-68-93 cache]# cd ..

[root@domU-12-31-39-00-68-93 public]# svn del cache

D cache

[root@domU-12-31-39-00-68-93 public]# svn commit -m "For symlinking tmp/cache to public/cache"

Deleting public/cache Committed revision 1041.

[root@domU-12-31-39-00-68-93 public]# ln -s /var/www/html/tripladder/tmp/cache/ /var/www/html/tripladder/public/

[root@domU-12-31-39-00-68-93 public]# chmod a+rwx /var/www/html/tripladder/tmp/ cache

[root@domU-12-31-39-00-68-93 public]# service httpd start

Starting httpd: [ OK ]

# - KC now to test the rake task to clear cache

[root@domU-12-31-39-00-68-93 tripladder]# ls public/cache/ application_layout_header.cache homepage_meta.cache homepage.cache send_review_to_freind.cache homepage_header.cache

[root@domU-12-31-39-00-68-93 tripladder]# ls tmp/cache/ application_layout_header.cache homepage_meta.cache homepage.cache send_review_to_freind.cache homepage_header.cache

[root@domU-12-31-39-00-68-93 tripladder]# rake tmp:clear (in /var/www/html/tripladder)

[root@domU-12-31-39-00-68-93 tripladder]# ls tmp/cache/ [root@domU-12-31-39-00-68-93 tripladder]# ls public/cache/ # kc - Confirmed that rake works.

8. Bundling the image

8.1 First stop all running services Now stop all services:

[root@domU-12-31-39-00-68-93 tripladder]# service httpd stop

Stopping httpd: [ OK ]

[root@domU-12-31-39-00-68-93 software]# service ferret_init_production stop Stopping ferret DRb server.

Install the ruby-openid gem to enable OpenID support stopping ferret server...


[root@domU-12-31-39-00-68-93 software]# service mysqld stop

Stopping MySQL: [ OK ]

8.2 Ensure that services do not start when system reboots -

This is important as we

have to attach EBS before we start the services.

[root@domU-12-31-39-00-68-93 software]# chkconfig --list httpd httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off

[root@domU-12-31-39-00-68-93 software]# chkconfig --list mysqld mysqld 0:off 1:off 2:off 3:off 4:off 5:off 6:off

[root@domU-12-31-39-00-68-93 software]# chkconfig --list ferret_init_production service ferret_init_production supports chkconfig, but is not referenced in any runlevel (run 'chkconfig --add ferret_init_production')

[root@domU-12-31-39-00-68-93 software]# chkconfig --add ferret_init_production [root@domU-12-31-39-00-68-93 software]# chkconfig --list ferret_init_production ferret_init_production 0:off 1:off 2:on 3:on 4:on 5:on 6:off

[root@domU-12-31-39-00-68-93 software]# chkconfig ferret_init_production off [root@domU-12-31-39-00-68-93 software]# chkconfig --list ferret_init_production ferret_init_production 0:off 1:off 2:off 3:off 4:off 5:off 6:off

8.3 Bundle the volume and save it to S3.

[root@domU-12-31-39-00-68-93 ~]# ec2-bundle-vol -d /mnt/ -k

.ec2/pk-<xxxxxxxxx>.pem -c .ec2/cert-<xxxxxxxx>.pem -u <12 digit AWS user id without hypehns>

Please specify a value for arch [i386]: ....# Lots of activity

[root@domU-12-31-39-00-68-93 ~]# ec2-upload-bundle -b tripladder-09Nov08 -m /mnt/image.manifest.xml -a <accesskey> -s <secret>

guest@kc-laptop:~$ ec2-register tripladder-09Nov08/image.manifest.xml IMAGE ami-3c15f155

8.4 Cleaning up

# kc - The AWS bundle tools seem to have memory leak - The used memory never comes back to normal and seems to be using up ~ 1 GB of RAM

[root@domU-12-31-39-00-68-93 ~]# umount /mnt/data-store/ [root@domU-12-31-39-00-68-93 ~]# reboot

9. Bringing up the application after reboot.

[root@domU-12-31-39-00-68-93 ~]# mount /dev/sdh /mnt/data-store/ [root@domU-12-31-39-00-68-93 ~]# service mysqld start

Starting MySQL: [ OK ]

[root@domU-12-31-39-00-68-93 ~]# service ferret_init_production start Starting ferret DRb server.

Install the ruby-openid gem to enable OpenID support starting ferret server...

[root@domU-12-31-39-00-68-93 ~]# service httpd start

Starting httpd: [ OK ]

# kc - Site should be running perfectly.


If ever your instance is killed, just do the following:

1. Launch your bundled instance wih the AMI id you got when you registered your bundle,

2. Attach the EBS volume, 3. Associate the IP address

4. Start services - mysql, ferret and httpd. 5. Presto!!!

# - Important - NEVER put the mount option for EBS in fstab. When the system is brought up from AMI, the EBS volume is still not attached and hence will not be mounted. This will not allow the system to boot. You AMI will essentially be useless. WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG, WRONG !!!!!!!!!!!!!!!!

[root@domU-12-31-39-00-65-E3 ~]# vi /etc/fstab

/dev/sdh /mnt/data-store ext3 defaults,noatime 0 2

# - Amazon is working on attaching a volume to EC2 instance during launch - when this feature is available, we can use fstab....





Related subjects :