def base_three_message_digest_value
    self.message_digest_value.to_s(3)
  end

  # get out a specific position in a number for base 3
  def get_message_digest_value_for_position(position)
    if base_three_message_digest_value.length > position then
      value_position = base_three_message_digest_value.length - position - 1
      base_three_message_digest_value[value_position..value_position].to_i
    else
      0
    end
  end

  # set a specific position in a number for base 3
  def set_message_digest_value_for_position(position, value)
    if base_three_message_digest_value.length > position then
      value_position = base_three_message_digest_value.length - position - 1
      new_value = base_three_message_digest_value
      new_value[value_position] = value.to_s
      self.message_digest_value = new_value.to_i(3)
    else
      self.message_digest_value = self.message_digest_value + (value.to_i * (3 ** position))
    end
  end

then to pull out users with a specific value I do something like this :

Has anyone else out there done this? Am I missing something, or was this actually a good to way to help our user model stay small and reduce the number of db columns? So far all seems great.

bad :

good :

I’m also hoping for the universal health care here in SF for all us uninsured earth passengers.

:^) is better than : ) .. and when necessary use the clown hat because Cartman did <@:^)

Lastly .. rails performance boosts. Moral of the story is use :select so your objects aren’t bloated, avoid functions that filter records in a has_many relationship, and don’t be shy about selecting out things from joins to avoid code like page.owner.name which will do an extra query to grab the entire user record just for a name.

We had some code like user_page.invitations.yes_replies. Nice and clean, but all invitations are pulled, memory is gobbled away, and then filtered down to the yes replies. Fine for parties with 50 guests, but 1000 guests and monit starts doing more bouncing than we like to see.

I substituted all the crush3r specific information with CAPS.  Go, downloads everything to a local machine, and then put uploads it all.  In case it can help you with a starting point, here is the code :

  @@root = "LOCAL_FILE_DIR"
  @@new_bucket = "NEW_BUCKET_NAME"
  def self.go
    require 'aws/s3'
    AWS::S3::Base.establish_connection!(
      :access_key_id => "OLD_ACCESS_KEY_ID",
      :secret_access_key => "OLD_SECRET_ACCESS_KEY"
    )
    crusher_bucket = AWS::S3::Bucket.find("OLD_BUCKET_NAME")
    objects = crusher_bucket.objects
    begin
      objects.each do |obj|
        begin
          dirs = obj.key.split("/")
          root = @@root
          (0..(dirs.size-2)).each do |pos|
            FileUtils.mkdir root + dirs[pos] unless File.directory?(root + dirs[pos])
            root = root + "/" + dirs[pos] + "/"
          end
          file_name = "#{@@root}#{obj.key}"
          unless File.exists?(file_name)
            puts "WRITING : #{file_name}"
            open(file_name, 'w') do |file|
              obj.value do |chunk|
                file.write chunk
              end
            end
          else
            # puts "SKIPPING : #{file_name}"
          end
        rescue
          puts "#{obj.key} is messed up"
        end
      end
      objects = crusher_bucket.objects(:marker => objects.last.key)
    end while objects.size > 0
  end

  def self.put
    require 'aws/s3'
    AWS::S3::Base.establish_connection!(
      :access_key_id => "NEW_ACCESS_KEY_ID",
      :secret_access_key => "NEW_SECRET_ACCESS_KEY"
    )
    dirs = ["ARRAY_LIST_OF_LOCAL_DIRS"]
    dirs.each do |dir_name|
      names = Dir.entries(dir_name)
      names.delete_at(0) if names[0] == "."
      names.delete_at(0) if names[0] == ".."
      names.delete_at(0) if names[0] == ".DS_Store"
      left_path = dir_name.gsub(/LOCAL_FILE_DIR/, "")
      names.each do |name|
        begin
          key_name = left_path + name
          unless AWS::S3::S3Object.exists?(key_name, @@new_bucket) then
            AWS::S3::S3Object.store(key_name, open(dir_name + name), @@new_bucket, :access => :public_read)
            puts "WRITING : #{key_name}"
          else
            # puts "SKIPPING : #{key_name}"
          end
        rescue
          puts "MESSED UP : #{key_name}"
        end
      end
    end
  end

Instead I did some research for a couple hours. Decided to go with flash instead of java. Found a five year old company that made a flash uploader for $39. Built the feature out using the demo version. Shared it with the team to experiment, we all liked it, and that was it. Now there are most likely better solutions out there, but with around 2 – 3 days of time we were able to get a great feature that we are happy with that will move us ahead in helping make photo sharing easier on our site.

If any of you RoR people are going to try and use this uploader there were a couple things I had to do:

The uploader comes up after an ajax request so it isn’t on the intial page. IE did not care for this and threw a bunch of JS errors. This could have been because of the JS hooks provided by the flash component. However, hosting the uploader in an iframe fixed that problem and allowed me to continue on.

After an image is uploaded to the server there is a JS hook that is called on the client. I used this hook to send out an ajax request asking for the image that was just uploaded. This made it so that as each image is uploaded a thumbnail will appear in view on the client.

FireFox on the mac causes the uploader to disappear if opacity is used.  So, when the uploader is brought to view I dynamically disable opacity and restore it when uploading is finished.

If anyone has more experience with this I’d certainly like to hear about it. Picture is below, and use the site and try it out for yourself

That is, until I started working with RoR and was introduced to the script/console. Hello “script/console”, nice to meet you At first it seemed mildly useful, but the other day after a minor hiccup it proved itself extremely helpful.

The biggest thing to make sure and not overlook is the ability to execute commands like this:

>>user_list = User.find(:all)
>>user_list.each{ |u| u.update_attributes(:is_crazy, true) if u.has_duplicate_phone_numbers? }

I’ve done a lot of work with databases, but I am by no means a SQL guru. These 2 lines of code took me under a minute to write and there aren’t enough users in our system to have to worry about the execution time. The hiccup we had the other day involved sending out our monthly product mailer. We managed to have 2 servers that were asked to create mailer messages, so everyone started getting 2 of our product mailers! Eek!

Not the end of the world, but not so fun. We shut down the mailer. 1/3 of the mail did not get sent at all, a handful of people only got one mail, and the rest received 2 mails. So, I needed to enable only 1 mailer for the 1/3 of the people who hadn’t received any mails, and turn the rest of the records to a failed status. Thank you script/console. Literally under 2 minutes I was able to do some command line work and bring the mailer back up. I’m slowly moving myself away from directly doing sql, and loving every inch away I get.

You can see our admin interface – the fails are the duplicate mailers that I automatically failed:

Switching to memcached sessions
Had to go through some effort to make sure during our rolling upgrade our users don’t lose their sessions. I guess we could always schedule downtime, but where’s the fun in that?

Using the model cache portion of intelligent fragment cache
This plug-in intelligently maps models to fragment caches so that when any save or delete occurs on a specific model the correct fragment caches are invalidated. There is one shortcoming that we came across. In our contact page, if a user adds a new contact the cache is not invalidated because that new contact was not associated with the original fragment. So I needed to add code in the after_create method of our contact model to invalidate the proper key.

after_create :invalidate_fragments def invalidate_fragments
return unless ActionController::Base.perform_caching
ActionController::Base.fragment_cache_store.delete(“/account/contacts.body/#{self.owner_id}”)
end

Then I also realized it wasn’t clear to me when a model object was “touched” and therefore associated with a fragment. So there were some instances where I had to do things like this:

@component.touch if ActionController::Base.perform_caching

Aside from that all seemed fine and everything is happily caching now.

Grr to caching content with small custom messaging fragments
Those partial we have where it says something like “Russell, hello!” have made it difficult to effectively apply caching. One more specific area in Crusher with custom messaging is the guest list. When you invite guests to a party and show them on your page we display a list of all those people along with their pictures, responses, etc… Each one of those is a partial and if you’re party is big it takes a bit of time to load. Each anonymous user when they visit the page is given an “add photo” link that overs over their default mug shot. So, what to do? I haven’t spent enough time investigating this yet, but I’m thinking that I can leverage inline CSS outside of the cached partial to make sure the proper display is shown. If not that, there may be some possible Javascript trickery, but I try to avoid that stuff.

Excessive calls to the DB with iterating over records
I feel like everyone knows about this one. We did some major clean-ups to ensure that when we iterate over records we weren’t making those single DB calls for every record. This involved using the :include portion of the find methods from active record. The way I debugged this was by watching my console for DB queries, and continually optimizing until the only one left was the big one at the beginning of the request processing. I’m not a mysql guru, so maybe there are things we can do with views or stored procedures to speed things up more, but I haven’t had time to investigate.

One of them that we use most of the time allows us to enable show/hiding of browser elements and test through selenium that it works.

The trick is to define a css class :

.hidden { display:none; }

Locate the content that needs to be toggled :

<div id=”toggle_me”> content </div>

Then use this JS onclick for your element that initiates the toggle. Since we use rails, we use prototype, which has a terrific helper function:

$(“toggle_me”).toggleClassName(“hidden”);

The reason it needs to be done that way is because Internet Explorer doesn’t fully support attributes. So CSS like this is invalid in IE:

DIV[style="display:inline;"] { background: yellow; }

Then in selenium the checks will look something like this

check hidden :

verify_element_present “css=#hidden_options.hidden”

check visible :

verify_element_not_present “css=#hidden_options.hidden”

That’s it! This simple thing has saved us lots of pain in making sure we can run Selenium in IE. If anyone else has some tips, I’d love to hear them. We’ve been having problem after problem trying to get selenium passing using Safari.

We had 2 less fun things (problems) that I found annoying. The first problem was if you are a keyboard user and on FireFox, and used the up and down arrows in the auto-complete box when you hit the up arrow the entire window would jump. Thanks for this scriptaculous So after one evening of getting fed up with this I planned to fix it. I found a great simple solution.

Open up controls.js, go to the markPrevious function and change the last line to use false instead of true when calling scrollIntoView.

That’s it. I assume this line is used to support some corner case that doesn’t effect us. I tested all our auto-complete boxes against all our supported browsers and all is good. Problem #1 solved.

Problem #2. We also enable auto-complete for registered users to invite other people to a party they’ve been invited to. The problem with this is that the invite others box comes up through an ajax request. This is a problem because no matter what I did I wasn’t able to get the auto-complete javascript to fire when embedded in a script tag with-in the partial. There are multiple ways we could have fixed this. One being, don’t use ajax, and simply toggle the visibilty of the box when “invite others” is clicked, but I wanted to avoid loading the script if the person was just viewing the page. The solution we went with was when the user clicks in the auto-complete textbox it loads at that moment (onfocus). I’m not so excited about this one, but it has been working fine for some weeks now.

So .. go play with our auto-complete and see what you think .. and if you like it then get yourself some scriptaculous

This morning my entire gmail contact list was spammed via Rapleaf / Upscoop.

I am sorry to all those people that have e-mailed me via gmail and received this appalling SPAM since I used Upscoop.

This is one reason why we’ve tried our best to make sure that we e-mail people when it is expected. Like if you invite someone to a party, we send an e-mail. We’d never SPAM your guest list. This seems like simple common decency. I wouldn’t want to put this feeling I have on our users.

I understand that Rapleaf and Upscoop are start-ups, but this is just common decency. C-O-M-M-O-N .. D-E-C-E-N-C-Y. They even say on their home page : “It is more profitable to be ethical.” ..um, ha ha?

Ok, not that this was engineering related or anything, but I just had to get that out there.

I would suggest avoiding both of these sites.

Update:

Since then rapleaf has issued a formal apology. Should we forgive blatant spammers? Our court systems don’t let people off with apologies, I’m not sure that spamming is one of those forgivable offenses. Here’s the blog post : http://blog.rapleaf.com/2007/09/06/start-ups-privacy-and-being-wrong/