Rails: Validating URLs

Here's a quick-and-dirty way to validate URLs in your model. Updated: Only allow certain schemes.

require 'uri'

class Film < ActiveRecord::Base

  VALID_URI_SCHEMES = ['http', 'https']

  validates_presence_of :url
  validate :url_must_be_valid

protected

  def url_must_be_valid
    parsed = URI.parse(url)
    if !VALID_URI_SCHEMES.member?(parsed.scheme)
      raise URI::InvalidURIError
    end
  rescue URI::InvalidURIError => e
    errors.add(:url, 'is not a valid URL')
  end
end

Talk: Techniques for Building Third-party RESTful Web Services

I gave a talk today for Py Web SF called "Techniques for Building Third-party RESTful Web Services". Aside from covering a lot of material from the book "RESTful Web Services", I also covered a handful of tips and tricks that I discovered were necessary when creating a third-party RESTful Web Service meant for consumption by Web browsers. Here are the slides in OpenOffice.org format.

Python: Amazon Product Advertising API

If you use Amazon Associates Web Service, then you may have recently received an email explaining that they are renaming the service to the "Product Advertising API". They're also now requiring you to sign your requests. This was a problem for me. Previously, I was making requests without using an Amazon client library. It's pretty simple to use with just urllib2. However, the logic to sign requests is non-trivial.

I couldn't find any modern Python bindings for the Product Advertising API. However, I knew that boto was a popular client library for some of Amazon's other services like S3 and EC2. I figured the signing logic was probably the same. Even though boto doesn't support the Product Advertising API, I managed to get it to work.

Here's a simple proof of concept. You'll need your own access key, etc.:

import time
import urllib

from boto.connection import AWSQueryConnection

AWS_ACCESS_KEY_ID = '...'
AWS_ASSOCIATE_TAG = '...'
AWS_SECRET_ACCESS_KEY = '...'


search_index = 'All'
keywords = 'pink'
aws_conn = AWSQueryConnection(
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY, is_secure=False,
    host='ecs.amazonaws.com')
aws_conn.SignatureVersion = '2'
params = dict(
    Service='AWSECommerceService',
    Version='2008-08-19',
    SignatureVersion=aws_conn.SignatureVersion,
    AWSAccessKeyId=AWS_ACCESS_KEY_ID,
    AssociateTag=AWS_ASSOCIATE_TAG,
    Operation='ItemSearch',
    SearchIndex=search_index,
    Keywords=keywords,
    ResponseGroup='ItemAttributes,Images',
    Order='salesrank',
    Timestamp=time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime()))
verb = 'GET'
path = '/onca/xml'
qs, signature = aws_conn.get_signature(params, verb, path)
qs = path + '?' + qs + '&Signature=' + urllib.quote(signature)
print "verb:", verb, "qs:", qs
response = aws_conn._mexe(verb, qs, None, headers={})
print response.read()

Vi: Pirates vs. Ninjas

"Killer coding ninja monkeys do exist."

Are you a pirate or a ninja?

Thanks to Andrew Hall for the shirt.

Vim as a Python IDE

This is a pretty good post: VIM as Python IDE. Also see my NetBeans post.

Python: Look What the Stork Dragged In

I have this Python program that I haven't run in a couple years. I wanted to see if I could make it run faster. Last time, it took 22 hours. Sure enough, using some advanced optimization techniques (and a swig of castor oil), my wife and I were able to run the program in a mere 6.5 hours! Here it is:

#!/usr/bin/env python

"""Help Gina-Marie time her contractions."""

import time

SECS_PER_MIN = 60


last_start = None
while True:
    print "Press enter when the contraction starts.",
    raw_input()
    start = time.time()
    if last_start:
        print "It's been %s minutes %s seconds since last contraction." \
            % divmod(int(start - last_start), SECS_PER_MIN)
    last_start = start
    print "Press enter when the contraction stops.",
    raw_input()
    stop = time.time()
    print "Contraction lasted %s seconds." % int(stop - start)
    print

Python: lambda to the Rescue

As every Lisp programmer knows, sometimes a lambda (i.e. an anonymous, on-the-fly function) provides just the right amount of indirection you need to solve a problem elegantly. Using a lambda in Python, you can shove an expression into a function, and then call it a little later when the time is right.

I was coding some Python, and I kept ending up with code that looked like:

try:
    count = d['a']['b']['c']
except KeyError:
    count = 0

Translating that little idiom into a function is hard. Every single use of [] might result in a KeyError. I could use .get() on each, but that's painful. Similarly, managing a hierarchy of defaultdicts is a bit painful too; the values in my dicts are a mix of ints and other nested dicts.

The question is, how do you put the try / except part into a function, and use d['a']['b']['c'] as an argument? After all, if you call f(d['a']['b']['c']), and d['a']['b']['c'] results in a KeyError, f will never even be called.

The solution, if you haven't guessed by now, is to wrap it in a lambda. Hence, I have a function:

def default0(f):
    """Return f(), or 0 if it raises a KeyError."""
    try:
        return f()
    except KeyError:
        return 0

Here's how you use it:

default0(lambda: d['a']['b']['c'])

Viola!Voila! Nice and clean.

(Now, of course you could get really fancy with default0 such as configuring which exceptions it will catch and what value it should return. However, then I'd either have to pass those arguments or use the functools.partial function. I think default0 is simple enough, short enough, and local enough to my problem that I didn't bother.)