Simple chrooted FTP setup on EC2 micro instance

Source environment: Ubuntu

1. Install vsftpd
apt-get install vsftpd

2. Edit default config at /etc/vsftpd.conf

Make sure the you enable these:

local_enable=YES
write_enable=YES

chroot_local_user=YES
chroot_list_enable=YES
# (default follows)
chroot_list_file=/etc/vsftpd.chroot_list

Ensure this is disabled:

anonymous_enable=NO

and add the following to the end:

pasv_enable=YES
pasv_max_port=22100
pasv_min_port=22000
pasv_address=123.123.123.123 # REPLACE THIS WITH YOUR IP
port_enable=YES

max, min ports could be anything high enough not to overlap with other services. Those ports will also need to be open in your security group if you’re using EC2

3. Create/edit /etc/vstfp.chroot_list
Add usernames that you don’t want to chroot.

4. Create users for FTP access:

adduser USERNAME

5. Ensure the home folder of a user is not writable(!) This is new since VSFTP 2.3.5 I believe.

chmod a-w /home/USERNAME

6. Create folders under /home/USERNAME for a user to upload stuff to, since a user won’t be able to upload to the root of /home/USERNAME

How to move MySQL storage to RamFS or TmpFS partition

Whether moving all MySQL storage to a tmpfs helps with speeding it up or not is questionable but I needed to do for some testing purposes, so this is a short overview of how I did that hopefully will be useful:

First mount tmpfs to a folder:

sudo mkdir /var/ramfs
sudo mount -t ramfs -o size=1G ramfs /var/ramfs/

Here I mounted ramfs to /var/ramfs. I am using ramfs in oppose to tmpfs mainly because:

  • ramfs grows dynamically(tmpfs doens’t)
  • ramfs doesn’t use swap(while tmpfs does)

RAM-backed file system is mounted, so now I need to populate it with MySQL files for processing.
To do that I will need to stop mysql, copy it’s database files over to ramfs, adjust AppArmor and MySQL settings and start mysql server again. Here is the chain of commands to do that:

Copying files:

sudo /etc/init.d/mysql stop
sudo cp -R /var/lib/mysql /var/ramfs/
sudo chown -R mysql:mysql /var/ramfs/mysql

Tweaking MySQL config:

sudo cp /etc/mysql/my.cnf /etc/mysql/original-my.cnf
sudo vim /etc/mysql/my.cnf

Find line with ‘datadir‘ definition(it will look something like datadir = /var/lib/mysql) and change it to

datadir = /var/ramfs/mysql

Next step is to tune apparmor settings:

sudo vim /etc/apparmor.d/usr.sbin.mysqld

Add the following few lines just before the closing curly braces:


/var/ramfs/mysql/ r,
/var/ramfs/mysql/*.pid rw,
/var/ramfs/mysql/** rwk,

Looks like we’re done with settings, let’s see if it will work:


sudo /etc/init.d/apparmor restart
sudo /etc/init.d/mysql start

If mysql daemon starts(double check /var/log/mysql.err for any errors) and you can connect to it, mostlikely now we’re running fully off of a RAM device. To double check it, run this from mysql client:

mysql> show variables where Variable_name = 'datadir' \G
*************************** 1. row ***************************
Variable_name: datadir
Value: /var/ramfs/mysql/
1 row in set (0.00 sec)

That’s pretty much it :)

Apache 2 and HTTP Authentication with PAM

There are 2 ways(at least that I know of) to get Apache 2 to use PAM for http auth:

  • Old mod_auth_pam, which I believe is not developed anymore and also posses some security risks
  • Newer mod_authnz_external and pwauth

This little write up shows how to get Apache and PAM going on Ubuntu using the mod_authnz_external.
To get started, let’s install some packages:

sudo apt-get install libapache2-mod-authnz-external pwauth
sudo apt-get install libapache2-mod-authz-unixgroup
sudo a2enmod authnz_external authz_unixgroup

Edit config file for the Virtual Host you’d like to get them PAM-based HTTP Authentication going, such that it contains the following clause:


<IfModule mod_authnz_external.c>
AddExternalAuth pwauth /usr/sbin/pwauth
SetExternalAuthMethod pwauth pipe
</IfModule>

And the final bit of configuration goes to your Directory definition inside of vhost block:

<Directory /var/www/yourlocation>
AuthType Basic
AuthName "Restricted Area"
AuthBasicProvider external
AuthExternal pwauth
Require user john

# some other configuration statements
</Directory>

This will allow user john to access the resource.

Now if you also want to have PAM authentication by users group you’ll need to make few extra steps. Missing bit of puzzle here is called ‘unixgroup’ script and for some reason it is not in Ubuntu’s pwauth package where it ought to be. You will need to grab it from here and copy it over to /usr/sbin/unixgroup and make it executable. Here is a quick snippet to do that:


wget "http://pwauth.googlecode.com/files/pwauth-2.3.9.tar.gz"
tar xzvf ./pwauth-2.3.9.tar.gz
sudo cp pwauth-2.3.9/unixgroup /usr/sbin/
sudo chmod a+x /usr/sbin/unixgroup

Once that’s done, you’ll need to few more lines to you Virtual Host config, so it will look something like this:

<IfModule mod_authnz_external.c>
AddExternalAuth pwauth /usr/sbin/pwauth
SetExternalAuthMethod pwauth pipe
AddExternalGroup unixgroup /usr/sbin/unixgroup
SetExternalGroupMethod unixgroup environment

</IfModule>

<Directory /var/www/yourlocation>
AuthType Basic
AuthName "Restricted Area"
AuthBasicProvider external
AuthExternal pwauth
GroupExternal unixgroup
Require user john# some other configuration statements
</Directory>

Hopefully this is helpful to someone besides myself :) Let me know if you got stock somewhere along the way.

Installing Mercurial 1.5, 1.6 or 1.7 on Ubuntu Lucid Lynx 10.04

Lycid Lynx’ repository by default set to install Mercurial 1.4.x, which is great, but as of today that’s already 3 major releases behind :( To get yourself on the latest and greatest version of Mercurial there are 2 options: build from the source or just add mercurial-releases repository like this:

sudo add-apt-repository ppa:mercurial-ppa/releases
sudo apt-get update
sudo apt-get install mercurial

Make sure it did the right thing:

$ hg --version
Mercurial Distributed SCM (version 1.6)

Copyright (C) 2005-2010 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Another thing that will go nicely with Mercurial 1.6.x or 1.7.x is up-to-date TortoiseHg. To install TortoiseHg on Ubuntu 10.04 you’ll need to run the following:

sudo add-apt-repository ppa:tortoisehg-ppa/releases
sudo apt-get update
sudo apt-get install tortoisehg-nautilus

Advanced Web Ranking on Ubuntu: Fixing “SQLite Could Not Be Found” Issue

“SQLite Could Not Be Found Issue” that you see when running AWR on 64bit Linux(in my case Ubuntu 10.04 ) is caused by the fact that AWR relies on 32bit JRE(you can read about this on AWR’s forum). Problem could be fixed by installing 32bit JRE on the system.This short step by step guide cheat sheet will help:

If you’re on earlier version of Ubuntu this will generally do the trick:

sudo apt-get install ia32-sun-java6-bin

However if you’re on 10.04, then it’s a bit longer process, since you need to add partner repository first:

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get install ia32-sun-java6-bin

That’s not all though. Now you need to edit launch script(AdvancedWebRanking.sh). Line #4 needs to be updated to contain path to proper JRE. Your final file should look something like:
#!/bin/sh
cd "Advanced Web Ranking"
export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH
/usr/lib/jvm/ia32-java-6-sun/bin/java -Xmx512M -classpath awr.jar com.caphyon.awebrank.Splash

MySQL5 TINYINT, SMALLINT, MEDUIMINT, INT, BIGINT boundaries

From MySQL Documentation, boundaries of the numerical types are as follows:

Type Min Value Max Value Bytes
TINYINT -128 127 1
TINYINT UNSIGNED 0 255 1
SMALLINT -32,768 32,767 2
SMALLINT UNSIGNED 0 65,535 2
MEDIUMINT -8,388,608 8,388,607 3
MEDIUMINT UNSIGNED 0 16,777,215 3
INT -2,147,483,648 2147483647 4
INT UNSIGNED 0 4,294,967,295 4
BIGINT -9223372036854775808 9223372036854775807 8
BIGINT UNSIGNED 0 18446744073709551615 8

Find and Replace Across Multiple Files

Need to replace broken links across 1000 html files?

If you’re using linux/unix/macosx it’s just fire up the terminal and use this:

find -iname "*.htm" -exec sed -i 's/search/replace/' {} \;

search – is regex(or a string in the simplest form) what you’re looking for

replace – is what you’re replacing your string with

If you’re on Windows, well then it’s a bit more complicated. There is a nice tool – TextCrawler, it’s free and does the trick.