Adding option columns without killing Active Record

I just finished implementing the new notification system for our site, which involved adding 15 different new options that our users can customize regarding how they are messaged from crusher. We’ve learned that the more columns you have in a model, the more memory it takes up and eats away resources. Given that I really didn’t want to be adding 15 new columns to the user table. What else can you do though? Then I thought – hey, why not just have a single integer that holds the 15 new options and store the value in base 3. Then have each position in the integer represent one of the options. I went down this route and it ended up working really well (I think). I have one tiny int that is in base 6, and one medium int that is in base 3 and that holds all the options we need!  Here is the code I ended up writing to pull out the base 3 values :

  def base_three_message_digest_value
    self.message_digest_value.to_s(3)
  end

  # get out a specific position in a number for base 3
  def get_message_digest_value_for_position(position)
    if base_three_message_digest_value.length > position then
      value_position = base_three_message_digest_value.length - position - 1
      base_three_message_digest_value[value_position..value_position].to_i
    else
      0
    end
  end

  # set a specific position in a number for base 3
  def set_message_digest_value_for_position(position, value)
    if base_three_message_digest_value.length > position then
      value_position = base_three_message_digest_value.length - position - 1
      new_value = base_three_message_digest_value
      new_value[value_position] = value.to_s
      self.message_digest_value = new_value.to_i(3)
    else
      self.message_digest_value = self.message_digest_value + (value.to_i * (3 ** position))
    end
  end

then to pull out users with a specific value I do something like this :

(SUBSTR(CONV(message_digest_value,10,3),LENGTH(CONV(message_digest_value,10,3)) – 7,1)) > 0

Has anyone else out there done this? Am I missing something, or was this actually a good to way to help our user model stay small and reduce the number of db columns? So far all seems great.

More performance plus a cast and a :^)

Don’t try to gently glide your bike over MUNI tracks, because one time you will mess up and be allowed the pleasure of typing one handed (see picture). Perpendicular is the way to go.

bad :

=== -/:^)\-		/__:^(_.,’==
	..crash..

good :

||				||
||	-/:^)\-		-/:^)\-	||
||		..no problem..		||

I’m also hoping for the universal health care here in SF for all us uninsured earth passengers.

:^) is better than : ) .. and when necessary use the clown hat because Cartman did <@:^)

Lastly .. rails performance boosts. Moral of the story is use :select so your objects aren’t bloated, avoid functions that filter records in a has_many relationship, and don’t be shy about selecting out things from joins to avoid code like page.owner.name which will do an extra query to grab the entire user record just for a name.

We had some code like user_page.invitations.yes_replies. Nice and clean, but all invitations are pulled, memory is gobbled away, and then filtered down to the yes replies. Fine for parties with 50 guests, but 1000 guests and monit starts doing more bouncing than we like to see.

Need to move S3 Buckets or Accounts?

So when crusher first started last year our engineer who was experimenting with S3 decided to open up his own account and use his credit card. Then crush3r.com went live using S3 and his account. For the past 7 months he has been paying our S3 bill, which has been no more than a couple dollars. Then he started doing other projects using his S3 account, so we couldn’t take his account for crusher use. So we had this task of taking over our crusher bucket. Which is what we did last week. I found that there were no good tools for downloading entire accounts, or uploading, etc .. so I wrote some code using the aws-s3 gem.  It’s pretty rough code, because I didn’t spend more than an hour working on it, and I just used script/console to run and didn’t write the code for any other use other than this one-time port.
Our key structure is formatted as such “xxxx/xxxx/file_name.blah”, so every key is guaranteed to have 2 directories.

I substituted all the crush3r specific information with CAPS.  Go, downloads everything to a local machine, and then put uploads it all.  In case it can help you with a starting point, here is the code :

  @@root = "LOCAL_FILE_DIR"
  @@new_bucket = "NEW_BUCKET_NAME"
  def self.go
    require 'aws/s3'
    AWS::S3::Base.establish_connection!(
      :access_key_id => "OLD_ACCESS_KEY_ID",
      :secret_access_key => "OLD_SECRET_ACCESS_KEY"
    )
    crusher_bucket = AWS::S3::Bucket.find("OLD_BUCKET_NAME")
    objects = crusher_bucket.objects
    begin
      objects.each do |obj|
        begin
          dirs = obj.key.split("/")
          root = @@root
          (0..(dirs.size-2)).each do |pos|
            FileUtils.mkdir root + dirs[pos] unless File.directory?(root + dirs[pos])
            root = root + "/" + dirs[pos] + "/"
          end
          file_name = "#{@@root}#{obj.key}"
          unless File.exists?(file_name)
            puts "WRITING : #{file_name}"
            open(file_name, 'w') do |file|
              obj.value do |chunk|
                file.write chunk
              end
            end
          else
            # puts "SKIPPING : #{file_name}"
          end
        rescue
          puts "#{obj.key} is messed up"
        end
      end
      objects = crusher_bucket.objects(:marker => objects.last.key)
    end while objects.size > 0
  end

  def self.put
    require 'aws/s3'
    AWS::S3::Base.establish_connection!(
      :access_key_id => "NEW_ACCESS_KEY_ID",
      :secret_access_key => "NEW_SECRET_ACCESS_KEY"
    )
    dirs = ["ARRAY_LIST_OF_LOCAL_DIRS"]
    dirs.each do |dir_name|
      names = Dir.entries(dir_name)
      names.delete_at(0) if names[0] == "."
      names.delete_at(0) if names[0] == ".."
      names.delete_at(0) if names[0] == ".DS_Store"
      left_path = dir_name.gsub(/LOCAL_FILE_DIR/, "")
      names.each do |name|
        begin
          key_name = left_path + name
          unless AWS::S3::S3Object.exists?(key_name, @@new_bucket) then
            AWS::S3::S3Object.store(key_name, open(dir_name + name), @@new_bucket, :access => :public_read)
            puts "WRITING : #{key_name}"
          else
            # puts "SKIPPING : #{key_name}"
          end
        rescue
          puts "MESSED UP : #{key_name}"
        end
      end
    end
  end

Memory over usage is a standard? Really?

So we’ve finally transitioned over to our new host, EngineYard, and we ran in to some slight snags.  One of them being that our mongrels are overly hungry for memory.  I come from the C++ / C# world so normally I think the solution is to go straight to profiling tools and figure out what’s going on.  However I was presented with a different solution for our problem that I guess is fine, it just is not something I’m used to.  That is, when a mongrel becomes to large, simple kill it.  Why not?  So if we have a mongrel that gets out of hand there are a variety of different tools configured on our site that will tell it to die and bring up a new one .. and many people do this .. so problem solved? :-/

Flash uploader, why not?

We all knew that we needed some sort of uploader to help with bulk uploading images in our app. The problem is, my engineering experience doesn’t involve any technology I could use to write a component myself. Which would be fine at a large company, because I would take a month or two, decide on what technology was best, do some prototyping and then build a component. When our team consists of 3 people that just isn’t an option.

Instead I did some research for a couple hours. Decided to go with flash instead of java. Found a five year old company that made a flash uploader for $39. Built the feature out using the demo version. Shared it with the team to experiment, we all liked it, and that was it. Now there are most likely better solutions out there, but with around 2 – 3 days of time we were able to get a great feature that we are happy with that will move us ahead in helping make photo sharing easier on our site.

If any of you RoR people are going to try and use this uploader there were a couple things I had to do:

The uploader comes up after an ajax request so it isn’t on the intial page. IE did not care for this and threw a bunch of JS errors. This could have been because of the JS hooks provided by the flash component. However, hosting the uploader in an iframe fixed that problem and allowed me to continue on.

After an image is uploaded to the server there is a JS hook that is called on the client. I used this hook to send out an ajax request asking for the image that was just uploaded. This made it so that as each image is uploaded a thumbnail will appear in view on the client.

FireFox on the mac causes the uploader to disappear if opacity is used.  So, when the uploader is brought to view I dynamically disable opacity and restore it when uploading is finished.

If anyone has more experience with this I’d certainly like to hear about it. Picture is below, and use the site and try it out for yourself

RoR? Don’t forget about script/console :-)

When programming javascript Firebug has a console that lets you type in javascript and fire it against an open window. The Tandy TRS-80 command line let me type in basic programs on the fly. I also have faint recollections of having a command window that came in handy with other languages. However, none of these ever proved incredibly useful for me, or left any strong imprint on my impressionable mind.

That is, until I started working with RoR and was introduced to the script/console. Hello “script/console”, nice to meet you At first it seemed mildly useful, but the other day after a minor hiccup it proved itself extremely helpful.

The biggest thing to make sure and not overlook is the ability to execute commands like this:

>>user_list = User.find(:all)
>>user_list.each{ |u| u.update_attributes(:is_crazy, true) if u.has_duplicate_phone_numbers? }

I’ve done a lot of work with databases, but I am by no means a SQL guru. These 2 lines of code took me under a minute to write and there aren’t enough users in our system to have to worry about the execution time. The hiccup we had the other day involved sending out our monthly product mailer. We managed to have 2 servers that were asked to create mailer messages, so everyone started getting 2 of our product mailers! Eek!

Not the end of the world, but not so fun. We shut down the mailer. 1/3 of the mail did not get sent at all, a handful of people only got one mail, and the rest received 2 mails. So, I needed to enable only 1 mailer for the 1/3 of the people who hadn’t received any mails, and turn the rest of the records to a failed status. Thank you script/console. Literally under 2 minutes I was able to do some command line work and bring the mailer back up. I’m slowly moving myself away from directly doing sql, and loving every inch away I get.

You can see our admin interface – the fails are the duplicate mailers that I automatically failed:

More fun with caching and memcached

For the past couple months we’ve been running with our home page and an example page cached. Which gave us a terrfic boost – it was somewhere around 10x more requests/sec on the home page. After running with this for a while we decided to dedicate a week to performance. Specifically trying to speed up event pages, listing directories, users stuff pages, and the contact page. If your mind is ready to sink in to the details, read on!

Switching to memcached sessions
Had to go through some effort to make sure during our rolling upgrade our users don’t lose their sessions. I guess we could always schedule downtime, but where’s the fun in that?

Using the model cache portion of intelligent fragment cache
This plug-in intelligently maps models to fragment caches so that when any save or delete occurs on a specific model the correct fragment caches are invalidated. There is one shortcoming that we came across. In our contact page, if a user adds a new contact the cache is not invalidated because that new contact was not associated with the original fragment. So I needed to add code in the after_create method of our contact model to invalidate the proper key.

after_create :invalidate_fragments def invalidate_fragments
return unless ActionController::Base.perform_caching
ActionController::Base.fragment_cache_store.delete(“/account/contacts.body/#{self.owner_id}”)
end

Then I also realized it wasn’t clear to me when a model object was “touched” and therefore associated with a fragment. So there were some instances where I had to do things like this:

@component.touch if ActionController::Base.perform_caching

Aside from that all seemed fine and everything is happily caching now.

Grr to caching content with small custom messaging fragments
Those partial we have where it says something like “Russell, hello!” have made it difficult to effectively apply caching. One more specific area in Crusher with custom messaging is the guest list. When you invite guests to a party and show them on your page we display a list of all those people along with their pictures, responses, etc… Each one of those is a partial and if you’re party is big it takes a bit of time to load. Each anonymous user when they visit the page is given an “add photo” link that overs over their default mug shot. So, what to do? I haven’t spent enough time investigating this yet, but I’m thinking that I can leverage inline CSS outside of the cached partial to make sure the proper display is shown. If not that, there may be some possible Javascript trickery, but I try to avoid that stuff.

Excessive calls to the DB with iterating over records
I feel like everyone knows about this one. We did some major clean-ups to ensure that when we iterate over records we weren’t making those single DB calls for every record. This involved using the :include portion of the find methods from active record. The way I debugged this was by watching my console for DB queries, and continually optimizing until the only one left was the big one at the beginning of the request processing. I’m not a mysql guru, so maybe there are things we can do with views or stored procedures to speed things up more, but I haven’t had time to investigate.

Selenium fun with Internet Explorer and testing elements that toggle

Selenium may seem like a big overhead, but when you don’t have a dedicated QA team like us, it is a life saver. Between that, and our functional, unit, and integration tests we manage to cover testing most of the app. However, since it uses JavaScript, and each browser is implemented different, there are some tricks that are required in order to get cross-browser selenium running.

One of them that we use most of the time allows us to enable show/hiding of browser elements and test through selenium that it works.

The trick is to define a css class :

.hidden { display:none; }

Locate the content that needs to be toggled :

<div id=”toggle_me”> content </div>

Then use this JS onclick for your element that initiates the toggle. Since we use rails, we use prototype, which has a terrific helper function:

$(“toggle_me”).toggleClassName(“hidden”);

The reason it needs to be done that way is because Internet Explorer doesn’t fully support attributes. So CSS like this is invalid in IE:

DIV[style="display:inline;"] { background: yellow; }

Then in selenium the checks will look something like this

check hidden :

verify_element_present “css=#hidden_options.hidden”

check visible :

verify_element_not_present “css=#hidden_options.hidden”

That’s it! This simple thing has saved us lots of pain in making sure we can run Selenium in IE. If anyone else has some tips, I’d love to hear them. We’ve been having problem after problem trying to get selenium passing using Safari.

Some simple tweaks for auto-complete

Ah, yes .. the lovely auto-complete. I can’t imagine using any program for sending invitations that doesn’t have a fast way to locate your contacts. Which is exactly why, if you’re a registered user, when you are inviting guests we use auto-complete to help you quickly locate your contacts. Furthermore we allow tags to be assigned to contacts, and those tags show up in the auto-complete as well. So if you had a friend named “Joysake Muroo <jmuroo@jjjmmdd.com>” and a tag “joyofsakefriends” when you type in “sake” both of them will show up. Alright, so that’s the fun part.

We had 2 less fun things (problems) that I found annoying. The first problem was if you are a keyboard user and on FireFox, and used the up and down arrows in the auto-complete box when you hit the up arrow the entire window would jump. Thanks for this scriptaculous So after one evening of getting fed up with this I planned to fix it. I found a great simple solution.

Open up controls.js, go to the markPrevious function and change the last line to use false instead of true when calling scrollIntoView.

That’s it. I assume this line is used to support some corner case that doesn’t effect us. I tested all our auto-complete boxes against all our supported browsers and all is good. Problem #1 solved.

Problem #2. We also enable auto-complete for registered users to invite other people to a party they’ve been invited to. The problem with this is that the invite others box comes up through an ajax request. This is a problem because no matter what I did I wasn’t able to get the auto-complete javascript to fire when embedded in a script tag with-in the partial. There are multiple ways we could have fixed this. One being, don’t use ajax, and simply toggle the visibilty of the box when “invite others” is clicked, but I wanted to avoid loading the script if the person was just viewing the page. The solution we went with was when the user clicks in the auto-complete textbox it loads at that moment (onfocus). I’m not so excited about this one, but it has been working fine for some weeks now.

So .. go play with our auto-complete and see what you think .. and if you like it then get yourself some scriptaculous

Rapleaf and Upscoop are the latest spam king and queen

Once upon time there was this nice little site called Upscoop, that was pretty interesting. It would scour the e-mail addresses in your gmail/yahoo/etc.. contact list. Then it would show you all the social sites that your contacts belong to. It was pretty fun, and a good way to see what sites people you know are actually using.

This morning my entire gmail contact list was spammed via Rapleaf / Upscoop.

I am sorry to all those people that have e-mailed me via gmail and received this appalling SPAM since I used Upscoop.

This is one reason why we’ve tried our best to make sure that we e-mail people when it is expected. Like if you invite someone to a party, we send an e-mail. We’d never SPAM your guest list. This seems like simple common decency. I wouldn’t want to put this feeling I have on our users.

I understand that Rapleaf and Upscoop are start-ups, but this is just common decency. C-O-M-M-O-N .. D-E-C-E-N-C-Y. They even say on their home page : “It is more profitable to be ethical.” ..um, ha ha?

Ok, not that this was engineering related or anything, but I just had to get that out there.

I would suggest avoiding both of these sites.

Update:

Since then rapleaf has issued a formal apology. Should we forgive blatant spammers? Our court systems don’t let people off with apologies, I’m not sure that spamming is one of those forgivable offenses. Here’s the blog post : http://blog.rapleaf.com/2007/09/06/start-ups-privacy-and-being-wrong/