More fun with caching and memcached

For the past couple months we’ve been running with our home page and an example page cached. Which gave us a terrfic boost – it was somewhere around 10x more requests/sec on the home page. After running with this for a while we decided to dedicate a week to performance. Specifically trying to speed up event pages, listing directories, users stuff pages, and the contact page. If your mind is ready to sink in to the details, read on!

Switching to memcached sessions
Had to go through some effort to make sure during our rolling upgrade our users don’t lose their sessions. I guess we could always schedule downtime, but where’s the fun in that?

Using the model cache portion of intelligent fragment cache
This plug-in intelligently maps models to fragment caches so that when any save or delete occurs on a specific model the correct fragment caches are invalidated. There is one shortcoming that we came across. In our contact page, if a user adds a new contact the cache is not invalidated because that new contact was not associated with the original fragment. So I needed to add code in the after_create method of our contact model to invalidate the proper key.

after_create :invalidate_fragments def invalidate_fragments
return unless ActionController::Base.perform_caching
ActionController::Base.fragment_cache_store.delete(“/account/contacts.body/#{self.owner_id}”)
end

Then I also realized it wasn’t clear to me when a model object was “touched” and therefore associated with a fragment. So there were some instances where I had to do things like this:

@component.touch if ActionController::Base.perform_caching

Aside from that all seemed fine and everything is happily caching now.

Grr to caching content with small custom messaging fragments
Those partial we have where it says something like “Russell, hello!” have made it difficult to effectively apply caching. One more specific area in Crusher with custom messaging is the guest list. When you invite guests to a party and show them on your page we display a list of all those people along with their pictures, responses, etc… Each one of those is a partial and if you’re party is big it takes a bit of time to load. Each anonymous user when they visit the page is given an “add photo” link that overs over their default mug shot. So, what to do? I haven’t spent enough time investigating this yet, but I’m thinking that I can leverage inline CSS outside of the cached partial to make sure the proper display is shown. If not that, there may be some possible Javascript trickery, but I try to avoid that stuff.

Excessive calls to the DB with iterating over records
I feel like everyone knows about this one. We did some major clean-ups to ensure that when we iterate over records we weren’t making those single DB calls for every record. This involved using the :include portion of the find methods from active record. The way I debugged this was by watching my console for DB queries, and continually optimizing until the only one left was the big one at the beginning of the request processing. I’m not a mysql guru, so maybe there are things we can do with views or stored procedures to speed things up more, but I haven’t had time to investigate.

Selenium fun with Internet Explorer and testing elements that toggle

Selenium may seem like a big overhead, but when you don’t have a dedicated QA team like us, it is a life saver. Between that, and our functional, unit, and integration tests we manage to cover testing most of the app. However, since it uses JavaScript, and each browser is implemented different, there are some tricks that are required in order to get cross-browser selenium running.

One of them that we use most of the time allows us to enable show/hiding of browser elements and test through selenium that it works.

The trick is to define a css class :

.hidden { display:none; }

Locate the content that needs to be toggled :

<div id=”toggle_me”> content </div>

Then use this JS onclick for your element that initiates the toggle. Since we use rails, we use prototype, which has a terrific helper function:

$(“toggle_me”).toggleClassName(“hidden”);

The reason it needs to be done that way is because Internet Explorer doesn’t fully support attributes. So CSS like this is invalid in IE:

DIV[style="display:inline;"] { background: yellow; }

Then in selenium the checks will look something like this

check hidden :

verify_element_present “css=#hidden_options.hidden”

check visible :

verify_element_not_present “css=#hidden_options.hidden”

That’s it! This simple thing has saved us lots of pain in making sure we can run Selenium in IE. If anyone else has some tips, I’d love to hear them. We’ve been having problem after problem trying to get selenium passing using Safari.

Some simple tweaks for auto-complete

Ah, yes .. the lovely auto-complete. I can’t imagine using any program for sending invitations that doesn’t have a fast way to locate your contacts. Which is exactly why, if you’re a registered user, when you are inviting guests we use auto-complete to help you quickly locate your contacts. Furthermore we allow tags to be assigned to contacts, and those tags show up in the auto-complete as well. So if you had a friend named “Joysake Muroo <jmuroo@jjjmmdd.com>” and a tag “joyofsakefriends” when you type in “sake” both of them will show up. Alright, so that’s the fun part.

We had 2 less fun things (problems) that I found annoying. The first problem was if you are a keyboard user and on FireFox, and used the up and down arrows in the auto-complete box when you hit the up arrow the entire window would jump. Thanks for this scriptaculous So after one evening of getting fed up with this I planned to fix it. I found a great simple solution.

Open up controls.js, go to the markPrevious function and change the last line to use false instead of true when calling scrollIntoView.

That’s it. I assume this line is used to support some corner case that doesn’t effect us. I tested all our auto-complete boxes against all our supported browsers and all is good. Problem #1 solved.

Problem #2. We also enable auto-complete for registered users to invite other people to a party they’ve been invited to. The problem with this is that the invite others box comes up through an ajax request. This is a problem because no matter what I did I wasn’t able to get the auto-complete javascript to fire when embedded in a script tag with-in the partial. There are multiple ways we could have fixed this. One being, don’t use ajax, and simply toggle the visibilty of the box when “invite others” is clicked, but I wanted to avoid loading the script if the person was just viewing the page. The solution we went with was when the user clicks in the auto-complete textbox it loads at that moment (onfocus). I’m not so excited about this one, but it has been working fine for some weeks now.

So .. go play with our auto-complete and see what you think .. and if you like it then get yourself some scriptaculous

Rapleaf and Upscoop are the latest spam king and queen

Once upon time there was this nice little site called Upscoop, that was pretty interesting. It would scour the e-mail addresses in your gmail/yahoo/etc.. contact list. Then it would show you all the social sites that your contacts belong to. It was pretty fun, and a good way to see what sites people you know are actually using.

This morning my entire gmail contact list was spammed via Rapleaf / Upscoop.

I am sorry to all those people that have e-mailed me via gmail and received this appalling SPAM since I used Upscoop.

This is one reason why we’ve tried our best to make sure that we e-mail people when it is expected. Like if you invite someone to a party, we send an e-mail. We’d never SPAM your guest list. This seems like simple common decency. I wouldn’t want to put this feeling I have on our users.

I understand that Rapleaf and Upscoop are start-ups, but this is just common decency. C-O-M-M-O-N .. D-E-C-E-N-C-Y. They even say on their home page : “It is more profitable to be ethical.” ..um, ha ha?

Ok, not that this was engineering related or anything, but I just had to get that out there.

I would suggest avoiding both of these sites.

Update:

Since then rapleaf has issued a formal apology. Should we forgive blatant spammers? Our court systems don’t let people off with apologies, I’m not sure that spamming is one of those forgivable offenses. Here’s the blog post : http://blog.rapleaf.com/2007/09/06/start-ups-privacy-and-being-wrong/