OnMyList Blog

At OML we use Amazon S3 for storage, when we launched the site Gravatar was very unreliable so we had to roll our own with attachment_fu, and it worked fairly well.

With our redesign we want to make all avatars square coz the site looks better that way, so I added a JavaScript image cropper to help you crop your uploaded pictures into squares. It didn’t take long to write and it worked fine in my local development machine. But I was using file_system in attachment_fu locally instead of S3. (Now in retrospect I should’ve setup my own test bucket on S3)... I switched it to S3 on the beta site and it stopped working, that is, the images don’t get cropped properly! After about 10 hours of tracing through attachment_fu and trying silly things such as writing my own with_image override in s3_backend.rb, in the end I only had to add a single line and a minor change. Here are the before and after code…

before:

  def crop2square(args)
    image2crop = find_or_initialize_thumbnail(:full)
    image2crop.with_image do |img|
      @data = img.crop("#{args[:width]}x#{args[:height]}+#{args[:x1]}+#{args[:y1]}")
    end
    attachment_options[:thumbnails].each { |suffix, size| create_or_update_thumbnail(@data, suffix, *size) }
  end

after:

  def crop2square(args)
    image2crop = find_or_initialize_thumbnail(:full)
    image2crop.temp_path = image2crop.create_temp_file
    image2crop.with_image do |img|
      @data = img.crop("#{args[:width]}x#{args[:height]}+#{args[:x1]}+#{args[:y1]}")
    end
    attachment_options[:thumbnails].each { |suffix, size| create_or_update_thumbnail(@data.path, suffix, *size) }
  end

I don’t really feel like explaining why this works right now, and if you’re having the same problem I’m sure you won’t care either and you just want to copy and paste the code and move onto your next problem.

Hope this helps…

Ray and I spent quite a while to get everything in our Facebook app to work. There were some blog posts about how get invites to work but they are outdated (and they involve some nasty FQLs).

One thing that I did was to create a beta fb application to test all this stuff out with our beta server, this helps a lot.

So in case you’re going crazy adding invites to your fb app, here’s how we did it. Keep in mind that the fb platform changes rapidly so by the time you read this there is probably a better way to do invites.

In your canvas partial, add a link to http://apps.facebook.com/[yourappname]/invite

here’s how our invite method looks like:

  def invite
    @appname = ENV['RAILS_ENV'] == 'production' ? 'onmylist' : 'onmylist_beta'
    @appid = ENV['RAILS_ENV'] == 'production' ? 12345678 : 01234567

    if fbsession.is_valid?
      @friends2exclude = fbsession.friends_getAppUsers.uid_list        
    end

    render :partial => "select_fbml" 
  end

note: app id’s are not real! :)

Here’s the select_fbml partial:

<fb:fbml>
<fb:subtitle>Invite your friends to join OnMyList</fb:subtitle>

<div style="background-color:#f2fbdc;padding-bottom:0;margin-bottom:0">

<% join_button = CGI::escapeHTML "<fb:req-choice url='http://www.facebook.com/apps/application.php?id=#{@appid}' label='Join OnMyList'>" %>

<fb:request-form 
method="POST" 
invite="true" 
type="OnMyList" 
action="http://apps.facebook.com/<%= @appname %>/"
content="Let's join OnMyList so you can list your pants off! <%= join_button %>">

<fb:multi-friend-selector
rows="5"
exclude_ids="<%= @friends2exclude.join(',') %>"
showborder="false" 
actiontext="Here are your friends who don't have OnMyList. Invite them now!">
</fb:request-form>

</div>
</fb:fbml>

the exclude_ids param is to exclude your friends who already have the app installed.

After going live for about a week it became obvious that we should implement caching to help performance. I experimented with Rails’ page caching, but in Rails 1.2 the expire_cache helper does not deal with custom routes very well. Also, our pages have view counts, ratings, etc, that change way too often for us to be able to cache the entire page.

The next thing I looked at was caching the MySQL queries, this seemed easier than implementing fragment caching, coz I found a CachedModel tutorial and it looked very good. Took me about an hour to get everything with CachedModel working, I then looked at the verbose output of memcached and my development.log, it didn’t seem to be caching very much. After some more googling it turned out that CachedModel only caches find() queries, but we use custom finders in our models almost exclusively. To get it to cache more I had to manually cache and evict objects, which was kindda not fun.

So back to more googling, and I found a RailConf 2007 presentation by Chris Wanstrath on cache_fu, it was appropriately titled Kickin’ Ass with cache_fu. It seemed cool, and he talked about creating wrappers for custom finders, which was exactly what I needed. So I spent like an hour on it and moved from CachedModel to cache_fu.

I was all excited that it was so easy, almost too easy right? Yeah, I restarted my development server in Locomotive, loaded the main page, and was greeted with a “NoMethodError: protected method” error. It was pretty weird, I spent quite a while trying to figure it out by going through the cache_fu code, still couldn’t figure out it. I emailed Chris, who was extremely helpful and replied to my mails in lightning fast speed (thanks Chris!), he had never seen an error like that before either. After a while I went ahead and checked out cache_fu on our production server and ran the test suites that comes with it, it worked! Chris suspected that the problem had to do with Locomotive, I went ahead and installed a few gems to my global environment and ran the tests outside of the Locomotive sandbox, it worked! So after comparing the installed gem lists between Locomotive and MacPorts’ Ruby installation, I narrowed it down to the Ruby-MemCache gem. Uninstalled that, everything worked! So I thought I’d blog about this in case someone run into the same problem in the future. I have no idea when I installed that gem, I guess the name looked good so I installed it.

cache_fu is pretty neat, I created finders that check if there are cached version of the data and I’m using these new finders in our controllers, for example, here’s how the custom finder that locates recently updated lists looks like (I’ve removed some irrelevant code):

def List.recently_updated(category, find_conditions, 
  sort_column, sort_direction, limit, offset)

  find_conditions += 
    " AND (lists.is_private=0 OR lists.is_private IS NULL)"
  lists = List.find(:all, :conditions => find_conditions,
    :order => "updated_at DESC", :limit => limit, 
    :offset => offset)  

  lists = List.cached_perform_roster_sort(lists, 
    sort_column, sort_direction)

  lists
end

The cached version of it looks like this:

def List.cached_recently_updated(category, find_conditions, 
  sort_column, sort_direction, limit, offset)

  get_cache('recently_updated:' + category.to_s +
    conditions.to_s + sort_direction.to_s + 
    limit.to_s + offset.to_s) do

    recently_updated(category, find_conditions, 
      sort_column, sort_direction, limit, offset)
  end
end

pretty clean and simple! :) We’re testing the version of our site that uses memcached right now, if all goes well we will deploy it to the live site and hopefully you will enjoy a nice speedup.

Now we just have to tune the caching a bit, probably will add fragment caching to various places. Next big thing for me is to scale, right now we’re considering EC2 and Joyent.

At OML we use Amazon’s S3 for profile pics storage, we switched from local disk to S3 for scalability reasons when we need to move to multiple app servers or Amazon EC2. To implement S3 with attachment_fu was fairly straight-forward. However, we ran into a lot of problems with the aws-s3 gem. Basically the S3 backend, which uses aws-s3, failed with a broken pipe.

I did a bit of googling on this and found out the author of aws-s3 attempted to fix this in the latest subversion snapshot (# attempts changed to 10 by yours truly):

rescue Errno::EPIPE, Timeout::Error, Errno::EPIPE, Errno::EINVAL
    @http = create_connection
    attempts == 10 ? raise : (attempts += 1; retry)
end

So I checked out the latest SVN snapshot as a plugin, but it didn’t quite fix it, from time to time I still got the same error. So I ended up retrying it a couple more time in my own rescue block, if it still fails, then catch the exception and fail gracefully.

It seems to work for me, feel free to post comments if you are still having problems with profile pics… It would be great if you would also email the pics you tried to upload to feedback@onmylist.com

Thanks! Now go list your pants off!

We decided to open up the site to public last night, we only sent out a handful of email invites - only to people who wouldn't get too upset if for some unforeseen reasons we have to clear the database or lose some profile images and what not... So far so good...

Yesterday was a long day, we fixed and implemented a large number of the remaining quirks and features. Jeff re-worked the list creation and edit page, which will be re-worked yet again very soon to make them a bit more user-friendly. I decided to change profile image storage scheme from file system with file_column to Amazon S3 with attachment_fu, there are still some quirks with profile image uploads, I am looking into that. (I think if you upload a pic that is too small it doesn't work). Moving from file_column to attachment_fu with S3 was probably a crazy thing to do on launch day but I didn't want to have to deal with migrating images from one plugin to another later on.

After we went live and cleared the test lists from the database there were a few small problems like controller actions expecting at least one list, Jeff took care of those quickly. We had a big scare when we found out Ferret had some serious issues with concurrency. Basically, when more than one process try to update its index at the same time the index gets really screwed up. I moved to a Rrb server architecture for acts_as_ferret and it fixed that problem.

We have a ton of things to work on (yes we have a good number of lists of action items!), now back to Rails hacking...