To help myself manage the multiple identities, I wote a set of bash functions, called:

• aws_load <config-name> – loads configuration from config-name
• ec2ssh <instance-number-in-ec2din-list> – ssh’s into a given instance, with the root key
• ec2scp – a shorthand for scp -i <keyfile>

I keep the configuration files in the directory ~/amazon/conf/name.sh and keypairs in ~/amazon/keypairs/ but that should be obvious to change.

To change or load an identity, one simply calls the function from a shell prompt like so:

mads@workmads ~ % aws_load some-identity
loaded certificate ...
loaded /Users/mads/amazon/conf/some-identity.sh (...)

I hope someone finds this as useful as I do.

Functions (could be placed in .bashrc or .zshrc).

function aws_load {
    if [ -n "$1" ]; then
		ec2_configurations="$HOME/amazon/conf"
		ec2_keys="$HOME/amazon/keypairs"
		conf="$ec2_configurations/$1.sh"
		if [ -x "$conf" ]; then
			unset AMAZON_ID AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_CERT EC2_PRIVATE_KEY EC2_CERT AWS_KEYPAIR_NAME
 
			source $conf
 
			if [ -n "$AWS_KEYPAIR_NAME" ]; then
				export AWS_SSH_KEY="$ec2_keys/id_rsa_${AWS_KEYPAIR_NAME}-keypair"
			fi
 
			if [ -n "$AWS_CERT" ]; then
				export EC2_PRIVATE_KEY=~/.ec2/pk-$AWS_CERT.pem
				export EC2_CERT=~/.ec2/cert-$AWS_CERT.pem
 
				echo "loaded certificate $AWS_CERT"
			fi
 
			echo "loaded $conf ($AMAZON_ID)"
		else
			echo "configuration $conf not found (or not executable)"
		fi
    else
        echo "usage: aws_load <configuration name>"
    fi
}
 
function ec2ssh {
    if [ -n "$1" ]; then
        HOST="`ec2din | awk '/i-/ {print $4}' | tail +$1 | head -n 1`"
        ssh -i $AWS_SSH_KEY -l root ${HOST}
    else
        echo "Please write a number"
    fi
}
 
function ec2scp {
	scp -i $AWS_SSH_KEY $@
}

Configuration “file” template to be placed in ~/amazon/conf/<config-name>.sh:

#!/bin/sh
 
export AMAZON_ID=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_CERT=""
export AWS_KEYPAIR_NAME=""

Happy identity switching.

Until recently I’ve been using the file:// django cache, but that has a “problem” when multiple users needs to manipulate the cache (think uid 80 writes a key, that uid 1000 wants to delete).

My problem with the memcached:// django cache provider has been, that it cannot handle being used on a shared memcached instance, because of the danger of key collissions.

If project A and project B would share a memcached instance, they basiclly share the same global namespace. So if they both write a key called actor there is no telling what will happen.

So I wrote a little cache backend for django, that uses the current memcached backend, but adds a pre-defined prefix to all keys.

Usage: Put the code somewhere inside your project in a file called memcached_key_prefix.py, and set your CACHE_BACKEND to something like: path.to.memcached_key_prefix:///127.0.0.1:11211/?key_prefix=sewc_&foo=bar&timeout=3600

"Memcached cache backend with key prefixing"
 
from django.core.cache.backends.base import InvalidCacheBackendError
from django.core.cache.backends.memcached import CacheClass as MemcachedCacheClass
from django.utils.encoding import smart_unicode, smart_str
 
class CacheClass(MemcachedCacheClass):
    def __init__(self, server, params):
        try:
            self._key_prefix = smart_str(params['key_prefix'])
        except KeyError:
            raise InvalidCacheBackendError('key_prefix not specified')
 
        super(CacheClass, self).__init__(server, params)
 
    def _get_key(self, key):
        return self._key_prefix + smart_str(key)
 
    def add(self, key, value, timeout=0):
        return super(CacheClass, self).add(self._get_key(key), value, timeout)
 
    def get(self, key, default=None):
        return super(CacheClass, self).get(self._get_key(key), default)
 
    def set(self, key, value, timeout=0):
        return super(CacheClass, self).set(self._get_key(key), value, timeout)
 
    def delete(self, key):
        return super(CacheClass, self).delete(self._get_key(key))
 
    def get_many(self, keys):
        keys = [self._get_key(key) for key in keys]
        return super(CacheClass, self).get_many(keys)
 
    def incr(self, key, delta=1):
        return super(CacheClass, self).incr(self._get_key(key), delta)
 
    def decr(self, key, delta=1):
        return super(CacheClass, self).decr(self._get_key(key), delta)

Seeing as how there are a lot of benefits to using djangos own test suite (fixtures and automatic database generation to name a few), i set out to write a little test client for PyAMF to utilize django’s test client, so it would be possible to write a proper test suite.

This turned out great, and is now ticket 508 over at PyAMFs trac. Look at the client.py for the code. At some point, it will be integrated into PyAMF mainline as p.r.c.django.TestClient (or something like that).

1. M-x mac-font-panel-mode and pick the font you want.
2. M-x describe-font and copy the font name (e.g. “-apple-inconsolata-medium-r-normal--13-130-72-72-m-130-iso10646-1”)
3. Add the following to your .emacs file: (set-default-font "-apple-inconsolata-medium-r-normal--13-130-72-72-m-130-iso10646-1")

In my case the lines for font selection looks like this:

(set-face-font 'default "-apple-proggytiny-medium-r-normal--11-110-72-72-m-110-iso10646-1")

Using this to make the font display without antialisaing:

defaults write org.gnu.Emacs AppleAntiAliasingThreshold 100

And this to position emacs in the top left, and be 104 chars wide, and 64 high.

(setq initial-frame-alist `((width . 106) (height . 64)
			    (top . 0) (left . 0)))

And of course some other tweaks. Re-learning emacs is turning out to be a good idea. But that’s another story.

Then all of a sudden, along came a post about how Amir Salihefendic had made a little daemon in C, using libevent and memcached, that would allow him to cache a http request in memcached. I don’t quite get why one would like to reinvent a reverse http proxy like that, and stated my arguments on why I thought that this seemed like a really bad idea.

As you can tell from the comments, he did not agree. It ended up with him deleting my last comment. Good thing I kept a copy in my clipboard. So, here are the comments.

Mads Sülau Jørgensen 25. Mar

This could just be me, but why would you do this over a proxy setup with Varnish, Squid og Apache with mpm_event and mod_cache? Seems a little like you’ve reinvented the wheel.

Also, as a previous commenter has suggested, why did you not just use the memcached support in nginx, and if it was because of the lack of multiple memcache servers, why not patch it and send this patch back to the community?

amix 25. Mar

Mads Sülau Jørgensen:

There are two reasons why I didn’t patch nginx with this support:

Separation of concerns: Running a separate polling server architecture I can monitor it and scale it independently Patching nginx would not have been that easy I prefer using memcached than Varnish or other caching options and memcached gives me a lot of options (deletion of multiple keys at once, consistent hashing and easy and proven scaling to terabytes of cache – - just to name a few).

Mads Sülau Jørgensen 25. Mar

Ah, but you could, easily, monitor a running nginx instance. The same goes for a running varnish instance. I’m not necessarily talking about running the nginx memcached module in the same process or the same server as your main proxy is running. So you’d archive the seperation by running another process or running the process on another box. Easy.

And the memcached module of nginx does not seem /that/ difficult to work with. Sure you’d have to do alot of handling of dead servers, and timeouts that could block the whole server. The current one probably handles this already. I don’t use it, so I’m not one to say.

With your keys structured correctly, you can flush (purge) multiple keys easily in varnish. You can provide your own hashing function as well. You could also use nginx’s hash balancer to send the request to the “correct” backend server.

The thing about hacking some server together with libevent in a language, that might not be onces main language is, that you’ll end up missing a lot of things, and it may end up exploding in your face. From the top of my head, what happens if a memcached server dies, memory leaks, what happens if the process segfaults (see memory leak).

It might be super-duper fast, but is it stable :)

amix 25. Mar

Mads Sülau Jørgensen:

plurk@web04:~$ ps aux | grep poll_server
plurk     29835  2.3  0.0   2376  1048 ?        
S    Mar24  27:23 /home/plurk/poll_server/poll_server 0.0.0.0 16000

It’s pretty stable (has not crashed since I started it), uses very little CPU and very little memory. If I stop the server, the nginx server will just route everything to Python servers. I doubt Varnish or patching Nginx would have been faster or more stable – - and they would have added a big conceptual complexity.

Personally, I would much rather fix errors in a 100 line program than a thousands line program. It’s separation of concerns and separation of complexity. So sure, I could have extended nginx+memcached or hacked Varnish – - but I chose to keep it simple, learn some new things and solve the problem in a very specialized way.

Mads Sülau Jørgensen 25. Mar

There is no hacking in the varnish solution what so ever. Varnish is a http proxy cache, it would be doing what is was designed to do. And my guess is, that it would be doing it well. My guess is also, that you’d end up using it alot, and to do more that using it with this one goal.

With a specialized solution, you have to handle a new process, you have to handle the errors that process can give, and you have to keep it running.

Using the tools already available makes more sense to me, and that’s what I’m doing when I run into a problem. Others, more proficient programmers usually have had the same problem before me.

amix 25. Mar

Mads Sülau Jørgensen:

Varnish is a good HTTP proxy and cache, no doubt about that, but it lacks a lot of things to be used in my setup. For example, how can I invalidate 1000 keys at once and do it in a split of a second? Their purging capabilities seem to be pretty primitive…

Plus, who said that I won’t iterate over this and go into a new direction – for example, benchmark this against a long-polling “comet” solution. Going into this direction would mean dropping Varnish – while I only need to change and add some lines of code to create a long polling solution of this.

Mads Sülau Jørgensen 25. Mar

Well, with a properly designed URL schema, you’d do something like this:

PURGE /poll/user/(?:mads|john|abc*)+/status

And then you’d have the VCL to handle that purge via purge_url(). The above will then purge the urls matching the regular expression, (so mads or john or starting with abc).

It actually works both fast, and well. It does not purge the cache when you tell it, but when you request an object. That makes invalidation fast.

See http://varnish.projects.linpro.no/ for more information.

You can more or less configure varnish to do what you want it to do via it’s configuration file.

As for the comet solution, your post more or less said that. Right about where it said something about push being hard to scale, which strikes me as a very odd comment. With push, you just have to be able to handle a horde of mostly idle connections, for which http://code.google.com/p/erlycomet/ or something like that, would probably be the way to go.

amix 25. Mar

Mads Sülau Jørgensen:

I am pretty tired of keep repeating myself. So this will be the last comment I answer you.

Varnish is a good cache, but it’s not really suited where I am headed. An example, how would you change the Varnish solution to support long polling? I can explore this option pretty easily with a basic understanding of how to build a custom server that can perform really good and that can be customized for my needs. I personally want to learn how to build things myself (also things that are complex and that require some thinking and engineering). I know I’ll need this skill longer down the road and practice makes a master – so the more practice I have, the better I’ll be at doing this.

this is the reply he deleted

> I am pretty tired of keep repeating myself. So this will be the last comment I answer you.

The consider this answer a bonus for your collection of entropy.

> An example, how would you change the Varnish solution to support long polling?

I’m quite sure that I never implied, that varnish would be suitable to do long polling (or other non-cachable long lived requests). If you think I did, please, tell me where.

> I can explore this option pretty easily with a basic understanding of how to build a custom server that can perform really good and that can be customized for my needs.

Yes, and that’s very fine and good, but that’s not really what your post says. It basiclly says “Using libmemcached and a C program, I can make a lot of http requests. Fast. And I don’t expect to use comet. Push is not what I want.” but from the looks of it, you may have wanted it to say “Hi, I made this little C program using libmemcached that I’m planing on developing into a comet server at some point. I need to extend the server with push in the future.”

> I personally want to learn how to build things myself (also things that are complex and that require some thinking and engineering).

Yes, throwing something together is very fun, throwing something together in C is even more fun. But throwing something together in C for a large website, because, it’s just fun, well, for a lot of the web developers in the world, that’s a really bad idea. It may work today, tomorrow but will it work in a week, a month or a year?

> I know I’ll need this skill longer down the road and practice makes a master – so the more practice I have, the better I’ll be at doing this.

Then I’d really recommend you looking at this:

http://aleccolocco.blogspot.com/2008/10/gazillion-user-comet-server-with.html

A comet server in C, with all the boiler plate completed. Just insert logic.

You might also get a kick out of reading up on memory management, input validation, how to avoid buffer overflows and just plain old K&R C.

I love it.

When it comes to actionscript files, flash has always had odd newlines. For some reason, it has always used \r as it’s newline. Mercurial, beeing a unix tool, likes it’s newlines to be \n. So what better to do, than to make mercurial encode/decode the files for us.

To do that, we can simply add the following to our .hgrc file:

[encode]
*.as = perl -pe 's/\r/\n/g'

[decode]
*.as = perl -pe 's/\r/\n/g'

I don’t really know wether the decode part is necessary, but I like to keep it around (if someone should commit poison).

To use the hook you must place it somewhere inside your PYTHONPATH and tell mercurial to use it (I placed it in a module called trachook — don’t call your module trac):

[hooks]
changegroup = python:trachook.hook

And tell the hook where to find your trac installation:

[trac-hook]
root = /path/to/trac
url = http://url/to/trac

Grab your own copy of the source, and happy coding. And a big thank you to Jesper Nøhr.

Also, if you need a place to host your mercurial repository but don’t wan’t to set it up yourself, check out bitbucket.

Let’s for the fun of it say, that we would like to POST the following piece of XML to a PHP script:

1

<request command="run-command" />

Now, the way I’d do that in Actionscript 3, is like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

var a:XML = <request command="run-command" />;
var r:URLRequest = new URLRequest("http://localhost/service.php");
 
r.data = a;
r.method = URLRequestMethod.POST;
r.contentType = "text/xml";
 
var l:URLLoader = new URLLoader();
l.addEventListener(Event.COMPLETE, function(e:Event) {
    trace("success", l.data);
});
l.addEventListener(IOErrorEvent.IO_ERROR, function(e:Event) {
    trace("failed", e);
});
l.load(r);

But it seems, that older flash players (<= 9,0,45,0) does not call the XML’s toString() or something. The result is, that the URLLoader makes a GET request, which is a bit hard to debug as version 9,0,115,0 does send the XML correctly.

Anyway, the correct way to do it, seems to be to manually call the XML’s .toXMLString() method, like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

var a:XML = <request command="run-command" />;
var r:URLRequest = new URLRequest("http://localhost/service.php");
 
r.data = a.toXMLString();
r.method = URLRequestMethod.POST;
r.contentType = "text/xml";
 
var l:URLLoader = new URLLoader();
l.addEventListener(Event.COMPLETE, function(e:Event) {
    trace("success", l.data);
});
l.addEventListener(IOErrorEvent.IO_ERROR, function(e:Event) {
    trace("failed", e);
});
l.load(r);

We used to have a url schema consisting of the following components:

    [dev.|test.]<project-name>.<client-name>.companyname.tld

That made for very long url’s, and furthermore, the url’s would not be supported by a *.companyname.tld. Based on this infomation, and wanting to create a more generelized, nicer url schema, we choose to cross over to the following schema:

   [dev-|test-]<project-name>.companyname.tld

Which besides from supporting a wildcard ssl certificate just fine, just, well, looks nicer.

Either way, me beeing in charge of the contents in /etc/, and getting a little sick of growing apache configuration files, I chooose to write some macros to configure our project sites.

We generally have 2 types of project sites. django based sites, and PHP based sites.

A typical django apache configuration consists of a VirtualHost and some mod_python settings inside a Location block. Lot’s of configuration, very little actual diffrence between the configration projects in between.

An example of a django based application running inside a apache VirtualHost, could look like this:

    <VirtualHost *:80>
        ServerName test-project.company.tld
        <Location />
            SetHandler python-program
            PythonHandler django.core.handlers.modpython
            PythonPath "['/path/to/project', '/other/path/to/inject']"
            SetEnv DJANGO_SETTINGS_MODULE project.settings
        </Location>
        <Location /media/>
            SetHandler none
        </Location>
        Alias /media/admin/ /path/to/django/contrib/admin/media/
        Alias /media/ /path/to/project/media/
    </VirtualHost>

And the on top of that comes SSL configuration, auth and such. 25 lines of configuration per django site. Now, I really wanted to acomplish two things.

• Make the configuration easier to maintain
• Enable other users to setup sites without knowing the depths of django and mod_python

Now, apache configuration, meet mod_macro. The solution to my problem was very simple. Create a macro that handles all the django configuration, given 4 parameters. A server name (i.e. the url), the parent directory of the django site and the module name of the site.

So i started building my macro.

    <Macro DjangoSite $servername $root $module>
        <VirtualHost *:80>
            ServerName $servername
            <Location />
                SetHandler python-program
                PythonHandler django.core.handlers.modpython
                PythonPath "['/common/path/to/inject', '$root']"
                SetEnv DJANGO_SETTINGS_MODULE $modulename.settings
            </Location>
            <Location /media/>
                SetHandler none
            </Location>
            Alias /media/admin/ /path/to/django/contrib/admin/media/
            Alias /media/ $root/$module/media/
        </VirtualHost>
    </Macro>

mod_macro is really simple, it will do a longest-match search and replace on the macro’s content. So, if we had a django site in /path/to/djangosite/testsite/ the above macro could be used as:

    Use DjangoSite www.example.org /path/to/djangosite/ testsite

Which would expand to:

    <VirtualHost *:80>
        ServerName www.example.org
        <Location />
            SetHandler python-program
            PythonHandler django.core.handlers.modpython
            PythonPath "['/common/path/to/inject', '/path/to/djangosite/']"
            SetEnv DJANGO_SETTINGS_MODULE testsite.settings
        </Location>
        <Location /media/>
            SetHandler none
        </Location>
        Alias /media/admin/ /path/to/django/contrib/admin/media/
        Alias /media/ /path/to/djangosite/testsite/media/
    </VirtualHost>

Hope you find this infomation a little useful. I actually like looking at our apache configuration file now.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

import asyncore
import asynchat
import socket
 
class Lobby(object):
    def __init__(self):
        self.clients = set()
 
    def leave(self, client):
        self.clients.remove(client)
 
    def join(self, client):
        self.clients.add(client)
 
    def send_to_all(self, data):
        for client in self.clients:
            client.push(data)
 
class Client(asynchat.async_chat):
    def __init__(self, conn, lobby):
        asynchat.async_chat.__init__(self, conn=conn)
        self.in_buffer = ""
        self.set_terminator("\n")
 
        self.lobby = lobby
        self.lobby.join(self)
 
    def collect_incoming_data(self, data):
        self.in_buffer += data
 
    def found_terminator(self):
        if self.in_buffer.rstrip() == "QUIT":
            self.lobby.leave(self)
            self.close_when_done()
        else:
            self.lobby.send_to_all(self.in_buffer + self.terminator)
            self.in_buffer = ""
 
class Server(asynchat.async_chat):
    def __init__(self):
        asynchat.async_chat.__init__(self)
        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        self.set_reuse_addr()
        self.bind(("127.0.0.1", 12345))
        self.listen(5)
        self.lobby = None
 
    def set_lobby(self, lobby):
        self.lobby = lobby
 
    def handle_accept(self):
        sock, addr = self.accept()
        client = Client(sock, self.lobby)
 
if __name__ == '__main__':
    lobby = Lobby()
    server = Server()
    server.set_lobby(lobby)
    asyncore.loop()

Booting the server is a matter of python echoserver.py.

After that, telnet can be used to induce messages, as such:

$ telnet localhost 12345
foo
foo