swag.dk

I’m running multiple different project on AWS which was so much of a pain to use, as I often find myself having to use the identity of project-a together with the official amazon ec2 tools.

To help myself manage the multiple identities, I wote a set of bash functions, called:

• aws_load <config-name> – loads configuration from config-name
• ec2ssh <instance-number-in-ec2din-list> – ssh’s into a given instance, with the root key
• ec2scp – a shorthand for scp -i <keyfile>

I keep the configuration files in the directory ~/amazon/conf/name.sh and keypairs in ~/amazon/keypairs/ but that should be obvious to change.

To change or load an identity, one simply calls the function from a shell prompt like so:

mads@workmads ~ % aws_load some-identity
loaded certificate ...
loaded /Users/mads/amazon/conf/some-identity.sh (...)

I hope someone finds this as useful as I do.

Functions (could be placed in .bashrc or .zshrc).

function aws_load {
    if [ -n "$1" ]; then
		ec2_configurations="$HOME/amazon/conf"
		ec2_keys="$HOME/amazon/keypairs"
		conf="$ec2_configurations/$1.sh"
		if [ -x "$conf" ]; then
			unset AMAZON_ID AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_CERT EC2_PRIVATE_KEY EC2_CERT AWS_KEYPAIR_NAME
 
			source $conf
 
			if [ -n "$AWS_KEYPAIR_NAME" ]; then
				export AWS_SSH_KEY="$ec2_keys/id_rsa_${AWS_KEYPAIR_NAME}-keypair"
			fi
 
			if [ -n "$AWS_CERT" ]; then
				export EC2_PRIVATE_KEY=~/.ec2/pk-$AWS_CERT.pem
				export EC2_CERT=~/.ec2/cert-$AWS_CERT.pem
 
				echo "loaded certificate $AWS_CERT"
			fi
 
			echo "loaded $conf ($AMAZON_ID)"
		else
			echo "configuration $conf not found (or not executable)"
		fi
    else
        echo "usage: aws_load <configuration name>"
    fi
}
 
function ec2ssh {
    if [ -n "$1" ]; then
        HOST="`ec2din | awk '/i-/ {print $4}' | tail +$1 | head -n 1`"
        ssh -i $AWS_SSH_KEY -l root ${HOST}
    else
        echo "Please write a number"
    fi
}
 
function ec2scp {
	scp -i $AWS_SSH_KEY $@
}

Configuration “file” template to be placed in ~/amazon/conf/<config-name>.sh:

#!/bin/sh
 
export AMAZON_ID=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_CERT=""
export AWS_KEYPAIR_NAME=""

Happy identity switching.

Today, I had to copy 70 GiB of data from a ext3 filesystem to a XFS filesystem. This involved a lot of small files. After a couple of hours of waiting, I thought it’d be best to just leave it running, and resume my activities the day after. But oh nooo, I forgot to run it in a screen. More… »

Recently (10-20 minutes ago), amazon couldfront (a cdn) stopped sending dns replies in europe:

% dig -t ns cloudfront.net

; <<>> DiG 9.4.3-P1 <<>> -t ns cloudfront.net
;; global options:  printcmd
;; connection timed out; no servers could be reached

I was going to do a guide to set up a varnish to replace cloudfront temporarily (and did actually set up the instance, and software – I might do the guide and ami anyway) when I realized, that I (as well as most other people) can just change the relevant url to point to the S3 bucket. Problem solved. That will, however, not be as fast as either cloudfront itself, or a varnish cached backend.

Should anyone be interested in how varnish is setup to handle failures from cloudfront, I’ll happily do an ami.

Until recently I’ve been using the file:// django cache, but that has a “problem” when multiple users needs to manipulate the cache (think uid 80 writes a key, that uid 1000 wants to delete).

My problem with the memcached:// django cache provider has been, that it cannot handle being used on a shared memcached instance, because of the danger of key collissions.

More… »

While working on a project using PyAMF today, i was about to write a unittest of a service method, when I realized that it would be harder than necessary to make unittests of the service. The cause of this is, that there was no test client wrapper for the django test client for pyamf, which means, that to do a unittest of the gateway, you’ll have to start a django server, and run a unittest elsewhere.

Seeing as how there are a lot of benefits to using djangos own test suite (fixtures and automatic database generation to name a few), i set out to write a little test client for PyAMF to utilize django’s test client, so it would be possible to write a proper test suite.

This turned out great, and is now ticket 508 over at PyAMFs trac. Look at the client.py for the code. At some point, it will be integrated into PyAMF mainline as p.r.c.django.TestClient (or something like that).

We have recently switched to use mercurial as our DVCS. We’re hosting our many repositories on bitbucket.

I love it.

When it comes to actionscript files, flash has always had odd newlines. For some reason, it has always used \r as it’s newline. Mercurial, beeing a unix tool, likes it’s newlines to be \n. So what better to do, than to make mercurial encode/decode the files for us.

To do that, we can simply add the following to our .hgrc file:

[encode]
*.as = perl -pe 's/\r/\n/g'

[decode]
*.as = perl -pe 's/\r/\n/g'

I don’t really know wether the decode part is necessary, but I like to keep it around (if someone should commit poison).

I’ve been using GratisDNS A free danish dns provider to manage my dns for a long time. Using their management interface is no walk in the park. It also has a bat habit of sending the username and password as a hidden form variable for every page request, in cleartext mind you. Nothing like examining the source of a page (for instance, if you wanted to make an python api) and seeing your password in cleartext.

As far as their interface goes, I can deal with that. It works, and it seems to do what it’s supposed to do fairly easy. A thing that has pushed me over the top, is the admins inability to help diagnostic a technical issue related to using their system as a secondary dns in front of a hidden primary.

Yesterday a good friend of mine told me that there was a new player in the dns game, one started by some people that seem to care about usability, exciting features (multiple dns templates per domain) and external api’s (goodbuy BeautifulSoup). It’s called QuickDNS.

So I rushed to their site, created myself a user and started tinkering around. Let’s just say, the it was actually a pleasant experience to do, things just worked^tm. So I setup a few templates and now have transfered a handful of my domains over there. Seems to work very well, just as expected.

So a very big congratulations to the whole QuickDNS team, you’ve done an excellent job, and made me a happy customer.

I seem to have agreed to give them a case of cola once they start implementing the axfr / hidden primary feature, and another one on completion. Can’t wait!

Having searched a lot around google, it does not seem that anyone has published their trac commit hooks for mercurial. Since I had to use just that, I’ve cooked up a little hook which is based on the hook from the timingandestimationplugin. I’ve created it as a changegroup hook, and it’s probably filled with bugs, but it seems to work and it catches the fixes #42 and such.

To use the hook you must place it somewhere inside your PYTHONPATH and tell mercurial to use it (I placed it in a module called trachook — don’t call your module trac):

[hooks]
changegroup = python:trachook.hook

And tell the hook where to find your trac installation:

[trac-hook]
root = /path/to/trac
url = http://url/to/trac

Grab your own copy of the source, and happy coding. And a big thank you to Jesper Nøhr.

Also, if you need a place to host your mercurial repository but don’t wan’t to set it up yourself, check out bitbucket.

So, you want to POST some XML data to a web service.

Let’s for the fun of it say, that we would like to POST the following piece of XML to a PHP script:

1

<request command="run-command" />

More… »

So this morning I went to the dentist. I have the most fair and lovely dentist in the whole world, but today, there were a new dentist in town. Now I really don’t mind dentists, going to the dentist or doctors in general, but what i don’t like is not being informed about what is going to happen next. The same goes for open source projects.

More… »