Monday, August 31, 2009

Rails: Ratings


I had to add a ratings widget to my app, sort of like Amazon has. I decided to use the jQuery Star Rating Plugin on the front end. That worked out well. I decided to code the back end from scratch. That took longer than I would have expected, but the code is super tight.

Only logged in users can vote. If a user votes again, it should update his existing vote rather than letting him stuff the ballot box. Aside from keeping track of the ratings for each user, I wanted the item itself, i.e. the book, to have a rating_average field. Furthermore, I didn't want rating_average to have to calculate the average rating every time I loaded the page. It should be cached in the same way that Rails can cache the number of children a parent has.

Here's what my schema looks like:
class CreateBookRatings < ActiveRecord::Migration
def self.up
execute %{
CREATE TABLE book_ratings (
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
user_id INT NOT NULL,
book_id INT NOT NULL,
rating FLOAT NOT NULL,
created_at DATETIME,
updated_at DATETIME,

FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
FOREIGN KEY (book_id) REFERENCES books(id) ON DELETE CASCADE,

UNIQUE INDEX (book_id, user_id),
INDEX (user_id)
) ENGINE = INNODB
}

add_column :books, :rating_average, :float
add_column :books, :rating_count, :int, :null => false, :default => 0
end

def self.down
remove_column :books, :rating_count
remove_column :books, :rating_average

drop_table :book_ratings
end
end
I'll skip the controller, routing, and view. They're relatively straightforward once you understand how the jQuery plugin works. The hardest part was the model:
class BookRating < ActiveRecord::Base
MAX_STARS = 5
SPLIT = 2 # You can have half of a star.

belongs_to :user
belongs_to :book
validates_numericality_of :rating, :greater_than_or_equal_to => 0,
:less_than_or_equal_to => BookRating::MAX_STARS
attr_accessible :rating

# Save a book rating.
#
# This may raise ActiveRecord::RecordInvalid if the rating is invalid.
#
# This automatically calls recalculate_average_and_count!.
def self.rate_book!(user, book, rating)
if rating.nil?
BookRating.delete_all(["user_id = ? AND book_id = ?", user.id, book.id])
else
book_rating = BookRating.new(:rating => rating)
book_rating.user = user
book_rating.book = book

# Validate manually so that I can use custom SQL.
if book_rating.invalid?
raise ActiveRecord::RecordInvalid.new(book_rating)
end

# This lets users create a new rating or update their existing rating.
# Unfortunately, insert_sql can't take an array, so I have to use
# connection.quote manually. I'm using book_rating.rating so that
# ActiveRecord can take care of the casting.
connection.insert_sql(%{
INSERT INTO book_ratings (user_id, book_id, rating, created_at, updated_at)
VALUES (#{connection.quote(user.id)},
#{connection.quote(book.id)},
#{connection.quote(book_rating.rating)},
NOW(), NOW())
ON DUPLICATE KEY UPDATE rating = #{connection.quote(book_rating.rating)},
updated_at = NOW()
})
end

recalculate_average_and_count!(book)
end

# Update book.rating_average and book.rating_count.
#
# I can calculate the average without having to scan the table when the
# user creates a new book rating, but that falls apart if he updates his
# existing rating. Hence, in the name of simplicitly, I'll just let the
# database calculate the average.
#
# I'm not going to put rate_book! and recalculate_average_and_count! into a
# single transaction. Transactions break my tests when I
# use_transactional_features, and in this case, it just isn't that crucial.
def self.recalculate_average_and_count!(book)
options = {:conditions => ["book_id = ?", book.id]}
book.rating_average = BookRating.average(:rating, options)
book.rating_count = BookRating.count(:rating, options)
book.save!
end
end
The most interesting bits are the use of "ON DUPLICATE KEY UPDATE" and the fact that the rating average is updated every time the user rates a book.

I haven't yet added Ajax to the mix. The user actually has to click a button to submit the form. However, the widget still works if JavaScript is disabled. The guys who wrote the jQuery plugin did a good job making use of semantic HTML. Because it works even without JavaScript, I was able to write Cucumber and RSpec tests for everything :)

Tuesday, August 25, 2009

JavaScript: Ajax Notifications with jQuery

If you're using Ajax, you should give hints to the user when a request to the server is being made, and you should tell him if something goes wrong. Gmail does this really well. Here's how I coded it:

First, add this to your layout:
<div id="ajaxNotifications"></div>
Add some CSS to make it purty:
#ajaxNotifications {
position: fixed;
top: 0px;
right: 0px;
background: yellow;
text-color: black;
padding: 5px;
display: none;
}
Setup jQuery, jQuery UI, and your own application.js file. This syntax is for Rails, but it's similar in Python:
<script src="http://www.google.com/jsapi" type="text/javascript"></script>
<script type="text/javascript">
<% extras = RAILS_ENV == "development" ? "{ uncompressed: true }" : "{}" %>
google.load("jquery", "1.3.2", <%= extras %>);
google.load("jqueryui", "1.7.2", <%= extras %>);
</script>
<%= javascript_include_tag "application" %>
Finally, here's application.js to pull it all together:
// This is the main application object.  It follows the module pattern.
var application = function () {
// Private functions and data go here.

return {
// Public functions and data go here.
HTTP_CONFLICT: 409,

// Show the user a quick little message.
showAjaxNotification: function(text) {
$("#ajaxNotifications").text(text).show();
},

// Hide the message. If possible, use hideAjaxNotificationIfMatches
// instead.
hideAjaxNotification: function() {
$("#ajaxNotifications").hide().text("");
},

// Hide the message, but only if it matches the given text.
hideAjaxNotificationIfMatches: function(text) {
if ($("#ajaxNotifications").text() == text) {
application.hideAjaxNotification();
}
}
};
}();

$(document).ready(function() {
var loadingMessage = "Loading...";

// When an Ajax request is being made, tell the user "Loading...".
$(this).ajaxStart(function() {
application.showAjaxNotification(loadingMessage);
});

// Only if it completes normally do we hide the message.
$(this).ajaxSuccess(function() {
application.hideAjaxNotificationIfMatches(loadingMessage);
});

// Otherwise, we give the user an error message.
$(this).ajaxError(function(event, xhr, ajaxOptions, thrownError) {

// If we get an HTTP_CONFLICT, it means our data is stale. Reload the
// page. To test this functionality, use two tabs in your browser.
if (xhr.status == application.HTTP_CONFLICT) {
location.reload();
}

// Otherwise, just tell the user something went wrong.
else {
application.showAjaxNotification("Request failed: could not contact server");
}
});
});

Monday, August 24, 2009

Rails: Engine Yard Flex


I'm reading the Engine Yard Flex documentation. It's pretty interesting. Here's a snippet:
Each Application or Application Master server is setup to run haproxy on port 80 and then nginx or apache on port 81. Each App server has its own haproxy setup to balance load to all the other App servers so any one App server could become master at any point if the main master fails for any reason. We have an 'ey-monitor' daemon that runs on all the application slave servers and periodically does health checks on the current Application Master server to see if it is still running properly or not. If the App Master fails for any reason then the App slaves will try to take over as master by using a technique called STONITH(shoot the other node in the head). This means that once the master fails, the slaves will wait for a few bad health checks and then the slaves will all race to grab a distributed lock. Whichever slave gets the lock will steal the IP address of the failing master server, then it will notify our control tier which in turn will violently terminate the failed app master. Then the system will boot a new server to replace the failed node and will use the same volumes that old master had so it has the full current state of the world.

This all happens transparently to you as a user and needs no input. The system will try its best to keep itself running and at the capacity that you have stated. There can be a very short downtime when slaves take over for masters, but generally it happens in 60 seconds or less.
Boy I'm glad I don't have to set all that stuff up myself!

JavaScript: Coping with Client Server Conflicts

One of the things that worries me about writing Ajax apps is how easy it is for the client and the server to get out of sync. Open up the app in two different browsers, and it's easy to play havoc. Even if you implement Comet, all you have to do is temporarily lose Internet connectivity to go out of sync.

One way to cope with this problem is by using the "409 Conflict" HTTP status code. If the client makes a request, and the server knows that the client is working with stale state, it can return a "409 Conflict". In my own case, if the client makes an Ajax request, and the server returns a "409 Conflict", the client will reload the current page. It's not perfect, but it's a simple, effect, and easily-implemented strategy.

Here's how to return a "409 Conflict" in Rails:

render :text => "Your browser is out of sync with the server. Please reload the page.",
:status => "409 Conflict"
Here's how you can handle a "409 Conflict" if you are making an Ajax request with jQuery:
var HTTP_CONFLICT = 409;

$.ajax({
type: "POST",
url: "/some/url",
data: data,

// If we get an HTTP_CONFLICT, it means our data is stale. Reload the
// page. To test this functionality, use two tabs in your browser.
error: function(xhr, textStatus, errorThrown) {
if (xhr.status == HTTP_CONFLICT) {
location.reload();
}
}
});

Rails: Using jQuery via Google's AJAX Libraries API

Adding jQuery to your project via Google's AJAX Libraries API is a piece of cake! I just updated my application layout to include:
<script src="http://www.google.com/jsapi" type="text/javascript"></script>
<script type="text/javascript">
<% extras = RAILS_ENV == "development" ? "{ uncompressed: true }": "{}" %>
google.load("jquery", "1.3.2", <%= extras %>);
google.load("jqueryui", "1.7.2", <%= extras %>);
</script>
That code loads jquery and jqueryui. I don't have to keep them on my server. I don't have to cache them. It uses the compressed version for production and the uncompressed version in development. Best of all, the user very well might already have that file cached in their browser.

Thursday, August 20, 2009

JavaScript: DOM vs. innerHTML, Server-driven vs. Client-driven

What's the best approach to architecting JavaScript, and which frameworks best support that approach? Is it best to build the app mostly on the client like Gmail and Google Maps, or is it better to provide a normal HTML page, but with lots of Ajax mixed in like YouTube? Which approach leads to the fewest bugs when the client and server get out of sync? How does your server respond to Ajax requests? Does it serve up JavaScript code to run, JSON or XML data to digest, or pre-rendered HTML?

In the Rails world, there are all these helper functions that generate JavaScript in your HTML pages. The JavaScript might result in Ajax requests that themselves serve up more JavaScript (via .js.erb or .rjs files). There is also heavy use of innerHTML. The server is in control of the application flow.

In the jQuery world, it's standard to keep the JavaScript separate of the HTML. I think innerHTML use is still very common via the append() method.

It seems like having the client be in control is more common in the YUI world. That is, I think rich internet applications that talk to a server that just serves up data is more common in YUI than in jQuery. I think that's true of Dojo too.

I've seen some applications that only request data from the server and build the entire UI using DOM functionality. I've heard that this approach is painful, heavy, and occasionally very frustrating.

You could also build an application that only requests data from the server and builds the entire UI using mostly innerHTML. Building up a lot of HTML using JavaScript strings doesn't seem particularly pleasant either.

In GWT and Pyjamas, you write your JavaScript application in Java or Python respectively and then compile the app down to JavaScript. I'm guessing that the JavaScript builds the UI using DOM calls, but I'm not 100% sure. Has anyone out there tried Pyjamas and liked it better than, say, jQuery or YUI?

I've read the documentation for MochiKit, Dojo, YUI, and jQuery at various times over the years, and I've even read a couple books on Ajax. However, I've never read anything that gave a comprehensive break down of the pluses and minuses of each of these approaches.

At Metaweb, I do believe they started with the "build everything from scratch on the client using DOM calls" approach, and eventually the browser keeled over because there was just too much data. (Freebase produces a lot of data.) They switched to generating HTML on the server, and using Ajax to ask for even more HTML from the server when it was necessary. They liked that better. That approach is also recommended in JavaScript Best Practices on Dev.Opera.

I think most people pick an approach without even really thinking about it and never think about alternatives. Have you ever taken one approach and switched to another?

Wednesday, August 19, 2009

Web: More Lost than Ever

I've built Web apps in ASP, PHP, Perl (with mod_perl and Mason), Python, and Ruby. Just in the Python world, I've used Webware, Aquarium (which I wrote), Zope, Plone, Web.py, Pylons, and Django. I've also used ZPT, Cheetah, Myghty, Mako, Genshi, and Django templates. On the JavaScript side, I've used MochiKit, Dojo, and jQuery, and I've read the docs for YUI, Prototype, and Script.aculo.us. I've written Greasemonkey scripts, and read all the XUL documentation. I've used Apache, Nginx, IIS, and I've even written my own Web server. I've done ecommerce sites, configuration dashboards, XML-RPC services, RESTful Web services, and Ajax apps. I do everything from CSS to scalability. I'm even up to date with the latest books: I've read "JavaScript the Good Parts", "Building Scalable Web Sites", "RESTful Web Services", etc.

After a decade of working with the Web, you might think I'd know it like the back of my hand, but to tell you the truth, I feel more lost than ever. I still haven't found any Web technology that makes building Web apps anything other than a messy, complicated pain in the butt!

Thursday, August 13, 2009

Rails: Dynamic 404s, authlogic, Cucumber, and rescue_from

I'm using Rails, Cucumber, and authlogic. I want my 404 pages to be rendered dynamically; that way, I can use my application-wide layout, which is a lot friendlier for lost users. I want ActiveRecord::RecordNotFound exceptions to be handled by the dynamic 404 page. I want Cucumber tests to verify that everything is working correctly.

Getting everything working together at the same time turned out to be extremely challenging. Cucumber made it hard for me to test my ActiveRecord::RecordNotFound handling. authlogic made it impossible for me to catch ActionController::RoutingError exceptions. Defining ApplicationController#render_optional_error_file as described here conflicted with authlogic and/or Cucumber. After several hours, I finally got it all working. Here's how:

First, I added the following to ApplicationController:
# Note, I can't use rescue_from to catch ActionController::RoutingError,
# otherwise authlogic breaks. Hence, I set a default route instead.
rescue_from ActiveRecord::RecordNotFound, :with => :render_404
...
# I'm handling 404s manually so that I can use the application-wide layout.
def render_404
render :template => "errors/error_404", :status => "404 Not Found"
end
Note, render_404 must be a public method.

At the bottom of routes.rb, I have:
# If all else fails, render a 404.
map.connect '*path', :controller => :application, :action => :render_404
In app/views/errors/error_404.html.erb, I have:
Sorry, the page you were looking for does not exist.
That takes care of rendering 404s if no routes match or if there is an ActiveRecord::RecordNotFound exception.

My Cucumber tests look something like:
Scenario: there should be a custom 404 page for routing errors
Given I am on the homepage
And I am simulating a remote request
When I am on an invalid URL
Then I should get a "404 Not Found" response
And I should see "Sorry, the page you were looking for does not exist."
And I should see "Log In"

Scenario: there should be a custom 404 page for record not found errors
Given I am logged in as admin
And I am simulating a remote request
When I am on an invalid user
Then I should get a "404 Not Found" response
And I should see "Sorry, the page you were looking for does not exist.
In features/step_definitions/helper_steps.rb, I have:
def assert_response_status(http_status, message)
response.status.should == "#{http_status} #{message}"
end

Then /^I should get a "(\d+) ([^"]+)" response$/ do |http_status, message|
assert_response_status(http_status, message)
end

Given /^I am simulating a remote request$/ do
header "REMOTE-ADDR", "10.0.1.1"
end
Note, the REMOTE-ADDR header is necessary or else Rails will render a normal exception page when you're trying to test the 404 page.

In features/support/env.rb b/features/support/env.rb I have:
# Comment out the next line if you want Rails' own error handling
# (e.g. rescue_action_in_public / rescue_responses / rescue_from)
#
# I'm commenting it out in order to test that my 404 handling (using
# rescue_from) works properly.
# Cucumber::Rails.bypass_rescue
Update: Please see the comments on bypass_rescue and @allow-rescue below.

Figuring that out took the longest. Cucumber was monkey patching Rails behind my back. I spent a few hours trying to figure out why my Rails rescue_from clause wasn't running when it turned out that Cucumber was to blame.

In features/support/paths.rb, I have these paths defined:
when /an invalid URL/
"/totally_bogus_dude"
when /an invalid user/
user_path(-1)
Finally, I added the following to public/404.html:
<!--
In theory, this file should no longer be necessary since I'm handling 404s
dynamically. I'm going to leave it here in case something breaks behind my
back.
-->
Whew! That was rough!

Wednesday, August 12, 2009

Rails: authlogic Code Review

I just spent a couple days code reviewing authlogic. First of all, let me say it's good code. Usually, I have a lot to say when I code review someone's code, but this time, I was pretty happy. Here are the corrections I submitted.

Clearly, the author of authlogic knows Ruby a heck of a lot better than I do; I'm still relatively new to Ruby. There were a bunch of idioms, language features, and design decisions that caught my attention as a Python programmer. That's what this blog post is about.

The first thing I noticed is that the code pieces together huge classes by mixing in tons of modules. For instance, in authlogic/session/base.rb:
module Authlogic
module Session
# This is the base class Authlogic, where all modules are included. For
# information on functionality see the various sub modules.
class Base
include Foundation
include Callbacks
include Timeout
include Params
include Cookies
include Session
include HttpAuth
...
There are 22 included modules in all.

Here is one of the included modules, authlogic/session/foundation.rb:
module Authlogic
module Session
module Foundation
def self.included(klass)
klass.class_eval do
extend ClassMethods
include InstanceMethods
end
end

module ClassMethods
...
end

module InstanceMethods
...
end
end
end
end

The "included" method gets called when the module is included. It uses "class_eval" to explicitly mix in some methods. Notice that the "ClassMethods" module is explicitly mixed in using "extend", and the "InstanceMethods" module is explicitly mixed in using "include".

I had to refer to my Ruby book to figure out the difference. A class includes a module when it wants to mix in the module's functions as instance methods. It affects every instance of the class. An object extends a module when it wants to mix in the module's functions just into itself. In this case, the object in question is a class, so it's in essence mixing in functions as class methods. This is a vivid reminder that unlike languages like C++, classes in Ruby are objects too.

By the way, I do worry that this excessive use of mixins will lead to namespace conflicts. I ran self.methods.size in my UserSession class, which inherits from Authlogic::Session::Base. It reported 271 methods. I did the same thing in my User model, which inherits from ActiveRecord::Base. It reported 550 methods!

The next thing I noticed was methods like:
def session_ids
self.class.session_ids
end

def session_class
self.class.session_class
end
In Python, if you call a method on an instance, if it can't find the method among the instance methods, it'll also look at the class methods. Java does this too, although it's frowned upon. Ruby doesn't look among the class methods when you call an instance method. Hence, the code is explicitly delegating to the class in this code. Code like the above happens a surprisingly large number of times across the code base. I'm surprised there isn't a helper like "delegate_to_class :only => [:session_ids, :session_class]".

Another thing I noticed is code like this:
module Callbacks
METHODS = [
"before_password_set", "after_password_set",
"before_password_verification", "after_password_verification"
]
...
private
METHODS.each do |method|
class_eval <<-"end_eval", __FILE__, __LINE__
def #{method}
run_callbacks(:#{method}) { |result, object| result == false }
end
end_eval
end
end
Look closely at '<<-"end_eval", __FILE__, __LINE__...end_eval'. That's actually evaluating a "heredoc" (or at least that's what they call it in other languages) in order to define a method. Using some flavor of "eval" to add methods on the fly is fairly common in Ruby.

Another thing I noticed is code like:
def find_using_perishable_token(token, age = self.perishable_token_valid_for)
In Python, the defaults to a function are evaluated once, as the function is defined. They are not evaluated every time the function is called. If you forget that Python has "static defaults", you'll eventually get bitten by a bug. Apparently, that's not the case in Ruby:
irb(main):001:0> def f(default = [])
irb(main):002:1> default << "hi"
irb(main):003:1> end
=> nil
irb(main):004:0> f
=> ["hi"]
irb(main):005:0> f
=> ["hi"]
authlogic has impressively good docstrings. Ruby uses comments for docstrings. Python uses specially-placed strings. Hence, in Python, you can piece together docstrings at runtime using string interpolation, etc. I often make good use of this to keep my docstrings DRY and to prevent them from going stale. For instance, I might put a comment in a variable, and reuse that same comment in multiple docstrings. The code for authlogic occasionally has to duplicate the same comment.

Ruby programmers tend to use longer function names and they don't often try to limit their code to 80 columns. Here is a mildly comical case:
# A convenience function to merge options into the validates_length_of_login_field_options. So intead of:
#
# self.validates_length_of_password_field_options = validates_length_of_password_field_options.merge(:my_option => my_value)
#
# You can do this:
#
# merge_validates_length_of_password_field_options :my_option => my_value
def merge_validates_length_of_password_field_options(options = {})
self.validates_length_of_password_field_options = validates_length_of_password_field_options.merge(options)
end
I enjoyed the metaprogramming in authlogic. For instance, code like the following is fairly common across the codebase:
self.class.send(:attr_writer, login_field) if !respond_to?("#{login_field}=")
The last thing that caught me off guard was that Ruby supports:
1/0 rescue 'hi'
Apparently, that's an expression-level form of begin/rescue (aka try/finally). The above evaluates to "hi".

Anyway, as I said, authlogic is good code, and I learned a lot :)

Saturday, August 08, 2009

Vim: Transparency in MacVim


Wahoo! MacVim now supports transparency. Finally, I can have a nice, transparent Vim window to match my transparent terminal, and I don't have to give up all the modern conveniences. If you use the "Homebrew" profile for your Terminal.app, try putting the following in your .gvimrc:
colorscheme torte
set transparency=15

Open Source: Closed Source Video Games

About a year ago, my buddy Ben Bangert gave us his old GameCube. My wife and I really enjoyed playing Paper Mario: The Thousand-Year Door.

"Paper Mario" is a closed source game. I bought it used at my local GameStop. Since I'm a bit of a free software nut, you might wonder how I could live with the thought of playing a closed source game. The fact of the matter is, I don't think "Paper Mario" could be produced in an open source manner. Technically, I'm sure it could, but who would want to? A hundred people or more were involved in producing that game, and what do they get out of it? I wouldn't volunteer on such a project--I have real work to do! If I can't imagine volunteering my time on such a project, then why would I expect anyone else to? Instead, I paid money. I'm happy because I have a fun game to play. The developers are happy because they got paid. Believe it or not, it's a win win situation.

However, I do have enough time to write a small video game. I wrote it in a week with my buddy Adam Ulvi. It's open source. It's based on the open source library, PyGame. I also made use of an open source library called PGU aka "Phil's Game Utilities". Phil wrote PGU while he was writing his own video games. The fact that PyGame is open source has saved me multiple times, since I've often looked at the source code in a pinch.

Now, I'm not trying to say that big projects can't be produced in an open source way. If that were true, Linux wouldn't exist. I'm just saying that I enjoy large closed source games like "Paper Mario: The Thousand-Year Door", and I also enjoy writing small open source games using open source tools like PyGame. My point is, there's room in this world for both!

On a more personal note, I'm coming to the conclusion that I need to just relax. The industry is going to go in whichever direction it's going to go, and there ain't a heck of a lot I can do about it. My best bet is to stay flexible, friendly, and helpful--as usual ;)

Thursday, August 06, 2009

Testing: How SQLite is Tested

Alex Martelli sent me this: How SQLite Is Tested. It gave me a new respect for both SQLite and extremely-thorough testing. For instance, "the project has 709 times as much test code and test scripts" as actual source code.

Wednesday, August 05, 2009

Linux: OpenGEU


Continuing from my post yesterday, Linux: Open Source and my MacBook, I gave OpenGEU 8.10 a shot under VMware Fusion on my MacBook.

OpenGEU is an Ubuntu-based Linux distribution that uses E17 (aka the unstable version of Enlightenment). The project was started and designed by the Italian artist Luca D.M. (aka TheDarkMaster).

In short, it isn't quite as "cohesive" from a feature point of view as stock Ubuntu, but it's stunningly beautiful and performs a lot better. I gave VMware 780MB of RAM, but it's currently using only half that. It also seems to use a lot less CPU. It's actually tolerable under VMware.

Here are some problems I encountered and the solutions I came up with:

Don't use "Linux Easy Install". VMware claims that it can install the ISO automatically since it's based on Ubuntu. However, this just lead to a blank screen for me.

I told the installer to log me in automatically. Why not? I already have to authenticate with my Mac. This makes booting seem a lot faster.

I had to forcibly restart the virtual machine after installation. It just hung. VMware has done this to me before.

VMware-tools installed just fine.

I had some serious dpi problems. This resulted in very large fonts. Thanks to this wonderful page, I was able to fix the problem. I was very happy about that since this problem has always stumped me in the past. I edited /etc/X11/xorg.conf and added the following to the Monitor section, "DisplaySize 338 211 # 1280x800 96dpi".

I couldn't figure out how to move among the different virtual desktops. Alt-shift-(left|right|up|down) does the trick. That's good, because I use Cntl-(left|right|up|down) to move among my different spaces on the Mac. Hence, there's no conflict.

I tried sharing folders between my host and guest operating system. This worked amazingly well. You can enable it on the fly, aka without rebooting. /mnt/hgfs shows up on the guest OS. The only problem is the same problem you typically get with NFS. If your user ID and group ID are different on your host operating system, the files won't match your username and groupname on your guest operating system. It's possible to hack your away around this problem, but it's still a pain.

I told VMware to "pass power status to the virtual machine", but this led to warnings from OpenGEU.

Since OpenGEU doesn't use much from GNOME, it doesn't run startup tasks. Hence, I had to manually run /usr/bin/vmware-user every time I logged in to let VMware do its thing and fix the resolution. To add insult to injury, this causes one of the panels to end up in the middle of the screen. However, you can right-click on the very edge of it and tell it to restart. This fixes the problem.

As beautiful as OpenGEU is, it still made me miss the fonts on my Mac. I guess I'm spoiled. Furthermore, the screen seems "grainy", and the colors aren't quite perfect. I always complain about that in Linux. I have no clue why Linux running with the right resolution on my Mac under VMware looks slightly off, but normal OS X doesn't. Maybe it has to do with finely tuned gamma correction or anti-aliased graphics or something, I don't know. However, it kind of messes with my obsessive compulsive nature.

Anyway, I want to thank all the Python guys for their comments on my blog yesterday. They made me feel a lot better ;)

Tuesday, August 04, 2009

Rails: Cache Inconsistency Caused by a Common Rails Idiom

Rails has a feature called "flash". Anything put in the flash is available on the next page load. It's based on the session, but it goes away after the next page load. It's perfect for saving a message for the user even if you're going to do a redirect.

Rails also has a function called reset_session that wipes the user's session and gives him a new one. Agile Web Development with Rails says you should call the reset_session method after the user logs out of your site. This helps avoid session fixation attacks.

Unfortunately, authlogic doesn't do this automatically. Hence, I decided to do it myself. I had code like:
reset_session
flash[:notice] = "Log out successful!"
The code works, but the message "Log out successful!" doesn't show up. Fortunately, my tests caught that. It turns out that Rails has a known bug that if you call reset_session, flash breaks. Why?

Rails uses an idiom that looks like:
def foo
@foo ||= calculate_foo
end
This idiom implicitly uses @foo as a cache so that calculate_foo is only called the first time the foo method is called.

The great thing about caches is that they can prevent unnecessary, time-consuming work. The bad thing about them is that you have to deal with cache inconsistency problems.

flash uses this idiom:
def flash
unless defined? @_flash
@_flash = session["flash"] ||= FlashHash.new
@_flash.sweep
end

@_flash
end
So does the session. You might see where I'm going with this.

When you call reset_session, there are two caches that have become inconsistent, and the fact that there are multiple layers of cache inconsistency is what lead to the bug.

Linux: Open Source and my MacBook

I have a love hate relationship with my MacBook. I adore the little bugger. I love how everything just works. I like the screen. I like the software. However, I really don't have much love for Apple, the company. I've been getting the itch to edge a little bit more toward the open source side. This should not be surprising since I'm a Linux zealot at heart.

One thing that was a real turnoff for me is that Apple stopped releasing binary installers for standalone Darwin. The PureDarwin project is hoping to pickup where OpenDarwin left off, however, it's obviously tough going. I'll keep my eye on it, but I don't think it'll be production-ready any time soon.

I've given up, to some degree, on VMware and VirtualBox. They're useful, but they're not the sort of thing you want to live in on a day-to-day basis. They're just not fast enough. They're better for occasional use.

I was really intrigued to discover Elive. Some guy took Debian and Enlightenment 17, tweaked it out for a Mac, and then released it as an ISO. Interestingly, he asks, rather forcefully, for a donation before you can download the ISO. I've heard that the system is absolutely beautiful, but since it's mostly a one-man show, it's obviously not perfect.

The documentation for getting Ubuntu to run on a MacBook is definitely not a one-man show. It's amazing how well they've documented everything. Don't get me wrong--it's still a total pain in the rear getting everything working just right, but still--they're to be commended.

In order to dual-boot Linux, you need to install rEFIt, which is basically a bootloader. It installed quite easily for me. You also have to have some room for a Linux partition. In theory, Apple's Disk Utility and BootCamp can both resize your existing partition in order to make room for another operating system. As you might imagine, this doesn't work out so well in practice.

I was immediately impressed when I heard that those tools could resize a partition. I thought to myself, boy, they sure must be smart if they can move files around in the partition in order to make room at the end of it. Unfortunately, I was too right. If you've been using your Mac for a while, Disk Utility will report "no space left on device" when you try to resize the partition, even if you have plenty of space. That's because the drive will be a bit fragmented, and it doesn't know how to move the fragments around. Apparently, this is a super common problem. However, it's not so common that Apple will admit to it in their documentation.

There are three solutions. One is to copy your system to a second disk and then copy it back. This basically defrags the partition. The second is to buy a $100 defrag tool. I'm broke right now, so neither of those are acceptable options. The third is to just wipe your hard drive and start from scratch. Ah, I fondly remember the days in college when I had time to do things like that ;)

I think I'm just stuck. I don't have another 12 hours to mess with this, and I'm short on funds to just "make the problem go away." I have this wonderful machine that I love, but, as a lot of Linux enthusiasts will say, it's like wearing a pair of golden handcuffs.

I love how well suspend works, how well the wireless works, how long my battery lasts (almost 5 hours!), and how nice my screen is (I really don't understand why it's so much nicer than every Dell I've ever owned). However, I think I'd like to find a way to make my Mac even more open source. I'm already running MacPorts, and I use open source alternatives for just about everything. (By the way, I must admit that I paid for VMware Fusion and for The Missing Sync.) Perhaps it's just my OCD talking, but I want more!

So I spent two days trying to make my Mac more open source, and I didn't really get anywhere. Hopefully this blog post is helpful somehow to someone. If any of you out there in the Python world have helpful suggestions, I'm all ears :-D

Saturday, August 01, 2009

Rails: Foreign Key Constraints

I like referential integrity. Rails doesn't provide support for database-level foreign key constraints. "Agile Web Development with Rails" uses code such as the following to add foreign key constraints:
t.integer :product_id, :null => false, 
:options => "CONSTRAINT fk_line_item_products REFERENCES products(id)"
That doesn't work for me. The migration ran just fine, but when I looked at the database, the constraint wasn't present. I hate it when things fail silently!

I tried a plugin to add foreign key constraints, but I got frustrated pretty quickly. I fell back to executing plain SQL:
execute %{
CREATE TABLE roles_users (
role_id INT NOT NULL,
user_id INT NOT NULL,

FOREIGN KEY (role_id) REFERENCES roles(id) ON DELETE CASCADE,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
INDEX (role_id),
INDEX (user_id)
) ENGINE = INNODB
}
The nice thing about plain SQL is that I can understand exactly what's going on. There are no layers of abstraction in my way, and if my SQL is wrong, MySQL will complain.

To be fair, MySQL's errors aren't always the most helpful. I encountered a couple of syntax errors. One was because I misspelled a column name. Another was because I put the name of the foreign key constraint in the wrong place. In both cases, I ended up with the following:
ERROR 1025 (HY000): Error on rename of './myapp_development/mymodel' to './myapp_development/#sql2-87-40c' (errno: 152)
Lovely! Oh well, at least it's better than failing silently.

Since I'm trying out test driven development, I knew I had to write a test before I could go mucking around with my schema. It turned out to be fairly hard to test that the database itself enforces a foreign key constraint. After all, Rails loves to try to enforce those constraints itself. Here's the RSpec test I ended up with:
require File.expand_path(File.dirname(__FILE__) + '/../spec_helper')

describe Role do
context "the roles table" do
it "should enforce foreign key constraints in the database" do
verbose = ActiveRecord::Migration.verbose
begin
ActiveRecord::Migration.verbose = false
result = ActiveRecord::Migration.execute("SHOW CREATE TABLE roles_users")
sql = result.fetch_row[1]
[/CONSTRAINT/, /FOREIGN KEY/, /REFERENCES/].each do |phrase|
sql.should =~ phrase
end
ensure
ActiveRecord::Migration.verbose = verbose
end
end
end
end
Ugh, painful. (Apparently, the silence method is only available on controllers. There's a different logger for migrations.)

Oh well. My tests pass, and I have referential integrity.