To help myself manage the multiple identities, I wote a set of bash functions, called:

• aws_load <config-name> – loads configuration from config-name
• ec2ssh <instance-number-in-ec2din-list> – ssh’s into a given instance, with the root key
• ec2scp – a shorthand for scp -i <keyfile>

I keep the configuration files in the directory ~/amazon/conf/name.sh and keypairs in ~/amazon/keypairs/ but that should be obvious to change.

To change or load an identity, one simply calls the function from a shell prompt like so:

mads@workmads ~ % aws_load some-identity
loaded certificate ...
loaded /Users/mads/amazon/conf/some-identity.sh (...)

I hope someone finds this as useful as I do.

Functions (could be placed in .bashrc or .zshrc).

function aws_load {
    if [ -n "$1" ]; then
		ec2_configurations="$HOME/amazon/conf"
		ec2_keys="$HOME/amazon/keypairs"
		conf="$ec2_configurations/$1.sh"
		if [ -x "$conf" ]; then
			unset AMAZON_ID AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_CERT EC2_PRIVATE_KEY EC2_CERT AWS_KEYPAIR_NAME
 
			source $conf
 
			if [ -n "$AWS_KEYPAIR_NAME" ]; then
				export AWS_SSH_KEY="$ec2_keys/id_rsa_${AWS_KEYPAIR_NAME}-keypair"
			fi
 
			if [ -n "$AWS_CERT" ]; then
				export EC2_PRIVATE_KEY=~/.ec2/pk-$AWS_CERT.pem
				export EC2_CERT=~/.ec2/cert-$AWS_CERT.pem
 
				echo "loaded certificate $AWS_CERT"
			fi
 
			echo "loaded $conf ($AMAZON_ID)"
		else
			echo "configuration $conf not found (or not executable)"
		fi
    else
        echo "usage: aws_load <configuration name>"
    fi
}
 
function ec2ssh {
    if [ -n "$1" ]; then
        HOST="`ec2din | awk '/i-/ {print $4}' | tail +$1 | head -n 1`"
        ssh -i $AWS_SSH_KEY -l root ${HOST}
    else
        echo "Please write a number"
    fi
}
 
function ec2scp {
	scp -i $AWS_SSH_KEY $@
}

Configuration “file” template to be placed in ~/amazon/conf/<config-name>.sh:

#!/bin/sh
 
export AMAZON_ID=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_CERT=""
export AWS_KEYPAIR_NAME=""

Happy identity switching.

Until recently I’ve been using the file:// django cache, but that has a “problem” when multiple users needs to manipulate the cache (think uid 80 writes a key, that uid 1000 wants to delete).

My problem with the memcached:// django cache provider has been, that it cannot handle being used on a shared memcached instance, because of the danger of key collissions.

If project A and project B would share a memcached instance, they basiclly share the same global namespace. So if they both write a key called actor there is no telling what will happen.

So I wrote a little cache backend for django, that uses the current memcached backend, but adds a pre-defined prefix to all keys.

Usage: Put the code somewhere inside your project in a file called memcached_key_prefix.py, and set your CACHE_BACKEND to something like: path.to.memcached_key_prefix:///127.0.0.1:11211/?key_prefix=sewc_&foo=bar&timeout=3600

"Memcached cache backend with key prefixing"
 
from django.core.cache.backends.base import InvalidCacheBackendError
from django.core.cache.backends.memcached import CacheClass as MemcachedCacheClass
from django.utils.encoding import smart_unicode, smart_str
 
class CacheClass(MemcachedCacheClass):
    def __init__(self, server, params):
        try:
            self._key_prefix = smart_str(params['key_prefix'])
        except KeyError:
            raise InvalidCacheBackendError('key_prefix not specified')
 
        super(CacheClass, self).__init__(server, params)
 
    def _get_key(self, key):
        return self._key_prefix + smart_str(key)
 
    def add(self, key, value, timeout=0):
        return super(CacheClass, self).add(self._get_key(key), value, timeout)
 
    def get(self, key, default=None):
        return super(CacheClass, self).get(self._get_key(key), default)
 
    def set(self, key, value, timeout=0):
        return super(CacheClass, self).set(self._get_key(key), value, timeout)
 
    def delete(self, key):
        return super(CacheClass, self).delete(self._get_key(key))
 
    def get_many(self, keys):
        keys = [self._get_key(key) for key in keys]
        return super(CacheClass, self).get_many(keys)
 
    def incr(self, key, delta=1):
        return super(CacheClass, self).incr(self._get_key(key), delta)
 
    def decr(self, key, delta=1):
        return super(CacheClass, self).decr(self._get_key(key), delta)

Seeing as how there are a lot of benefits to using djangos own test suite (fixtures and automatic database generation to name a few), i set out to write a little test client for PyAMF to utilize django’s test client, so it would be possible to write a proper test suite.

This turned out great, and is now ticket 508 over at PyAMFs trac. Look at the client.py for the code. At some point, it will be integrated into PyAMF mainline as p.r.c.django.TestClient (or something like that).

To use the hook you must place it somewhere inside your PYTHONPATH and tell mercurial to use it (I placed it in a module called trachook — don’t call your module trac):

[hooks]
changegroup = python:trachook.hook

And tell the hook where to find your trac installation:

[trac-hook]
root = /path/to/trac
url = http://url/to/trac

Grab your own copy of the source, and happy coding. And a big thank you to Jesper Nøhr.

Also, if you need a place to host your mercurial repository but don’t wan’t to set it up yourself, check out bitbucket.

We used to have a url schema consisting of the following components:

    [dev.|test.]<project-name>.<client-name>.companyname.tld

That made for very long url’s, and furthermore, the url’s would not be supported by a *.companyname.tld. Based on this infomation, and wanting to create a more generelized, nicer url schema, we choose to cross over to the following schema:

   [dev-|test-]<project-name>.companyname.tld

Which besides from supporting a wildcard ssl certificate just fine, just, well, looks nicer.

Either way, me beeing in charge of the contents in /etc/, and getting a little sick of growing apache configuration files, I chooose to write some macros to configure our project sites.

We generally have 2 types of project sites. django based sites, and PHP based sites.

A typical django apache configuration consists of a VirtualHost and some mod_python settings inside a Location block. Lot’s of configuration, very little actual diffrence between the configration projects in between.

An example of a django based application running inside a apache VirtualHost, could look like this:

    <VirtualHost *:80>
        ServerName test-project.company.tld
        <Location />
            SetHandler python-program
            PythonHandler django.core.handlers.modpython
            PythonPath "['/path/to/project', '/other/path/to/inject']"
            SetEnv DJANGO_SETTINGS_MODULE project.settings
        </Location>
        <Location /media/>
            SetHandler none
        </Location>
        Alias /media/admin/ /path/to/django/contrib/admin/media/
        Alias /media/ /path/to/project/media/
    </VirtualHost>

And the on top of that comes SSL configuration, auth and such. 25 lines of configuration per django site. Now, I really wanted to acomplish two things.

• Make the configuration easier to maintain
• Enable other users to setup sites without knowing the depths of django and mod_python

Now, apache configuration, meet mod_macro. The solution to my problem was very simple. Create a macro that handles all the django configuration, given 4 parameters. A server name (i.e. the url), the parent directory of the django site and the module name of the site.

So i started building my macro.

    <Macro DjangoSite $servername $root $module>
        <VirtualHost *:80>
            ServerName $servername
            <Location />
                SetHandler python-program
                PythonHandler django.core.handlers.modpython
                PythonPath "['/common/path/to/inject', '$root']"
                SetEnv DJANGO_SETTINGS_MODULE $modulename.settings
            </Location>
            <Location /media/>
                SetHandler none
            </Location>
            Alias /media/admin/ /path/to/django/contrib/admin/media/
            Alias /media/ $root/$module/media/
        </VirtualHost>
    </Macro>

mod_macro is really simple, it will do a longest-match search and replace on the macro’s content. So, if we had a django site in /path/to/djangosite/testsite/ the above macro could be used as:

    Use DjangoSite www.example.org /path/to/djangosite/ testsite

Which would expand to:

    <VirtualHost *:80>
        ServerName www.example.org
        <Location />
            SetHandler python-program
            PythonHandler django.core.handlers.modpython
            PythonPath "['/common/path/to/inject', '/path/to/djangosite/']"
            SetEnv DJANGO_SETTINGS_MODULE testsite.settings
        </Location>
        <Location /media/>
            SetHandler none
        </Location>
        Alias /media/admin/ /path/to/django/contrib/admin/media/
        Alias /media/ /path/to/djangosite/testsite/media/
    </VirtualHost>

Hope you find this infomation a little useful. I actually like looking at our apache configuration file now.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

import asyncore
import asynchat
import socket
 
class Lobby(object):
    def __init__(self):
        self.clients = set()
 
    def leave(self, client):
        self.clients.remove(client)
 
    def join(self, client):
        self.clients.add(client)
 
    def send_to_all(self, data):
        for client in self.clients:
            client.push(data)
 
class Client(asynchat.async_chat):
    def __init__(self, conn, lobby):
        asynchat.async_chat.__init__(self, conn=conn)
        self.in_buffer = ""
        self.set_terminator("\n")
 
        self.lobby = lobby
        self.lobby.join(self)
 
    def collect_incoming_data(self, data):
        self.in_buffer += data
 
    def found_terminator(self):
        if self.in_buffer.rstrip() == "QUIT":
            self.lobby.leave(self)
            self.close_when_done()
        else:
            self.lobby.send_to_all(self.in_buffer + self.terminator)
            self.in_buffer = ""
 
class Server(asynchat.async_chat):
    def __init__(self):
        asynchat.async_chat.__init__(self)
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.set_reuse_addr()
        self.bind(("127.0.0.1", 12345))
        self.listen(5)
        self.lobby = None
 
    def set_lobby(self, lobby):
        self.lobby = lobby
 
    def handle_accept(self):
        sock, addr = self.accept()
        client = Client(sock, self.lobby)
 
if __name__ == '__main__':
    lobby = Lobby()
    server = Server()
    server.set_lobby(lobby)
    asyncore.loop()

Booting the server is a matter of python echoserver.py.

After that, telnet can be used to induce messages, as such:

$ telnet localhost 12345
foo
foo