LoLoCoJr

While cleaning up some code I ran across some obscure code of mine from my Ruby youth. I remembered reading about an ultra cool Ruby tool the other day so I decided to give my code a good flogging. The victim for tonight is a method used to slice ID numbers into chunks of at most 4 characters long. This is required to overcome the typical Linux file system limitation of at most 32000 entries per directory. In this case the project stores roughly a million thumbnail images on disk. We start with the original piece of code:

1
2
3
4
5
6
7
8

def splice_number(number, part_size = 4)
  n = number.to_s
  r = []
  return r if n.size.zero?
  (n.size / part_size).times { |t| r << n[(t*part_size)..((t+1)*part_size-1)] }
  r << n[-(n.size % part_size)..n.size] if (n.size % part_size) > 0
  r
end

Ah yes, WTF was I thinking when I wrote this? Who cares, it seemed very clever then! What does Flog think about this?

Total score = 28.35

none#splice_number: (28)
     7: size
     3: *
     2: %
     2: []
     2: <<
     1: +
     1: -@
     1: -
     1: /
     1: lit_fixnum
     1: zero?
     1: >
     1: to_s
     1: times

Pretty good score. Now it’s time for some torturing. Say hello to my little friend: unpack!

1
2
3
4

def splice_number(number, part_size = 4)
  n = number.to_s
  n.unpack("a#{part_size}" * (n.size / part_size) + ((n.size % part_size == 0) ? "" : "a*"))
end

Flog?

Total score = 13.05

none#splice_number: (13)
     3: size
     1: %
     1: /
     1: ==
     1: *
     1: +
     1: to_s
     1: unpack
     0: lit_fixnum

Yeow, well over half the pain gone!! Perhaps we can still improve by being less clever?

1
2
3
4
5
6

def splice_number(number, part_size = 4)
  n = number.to_s
  r = n.unpack("a#{part_size}" * (n.size / part_size) + "a*")
  r.delete("")
  r
end

More lines, but less code! Hmm?

Total score = 9.25

none#splice_number: (9)
     1: size
     1: /
     1: *
     1: +
     1: delete
     1: to_s
     1: unpack
     0: lit_fixnum

Update: Okay, couldn’t help myself, last blow:

1
2
3
4
5
6
7
8

def splice_number(number, part_size = 4)
  n = number.to_s
  p = "a#{part_size}"
  t = n.size / part_size
  r = n.unpack(p * t + "a*")
  r.delete("")
  r
end

Total score = 8.05

none#splice_number: (8)
     1: *
     1: size
     1: +
     1: delete
     1: to_s
     1: /
     1: unpack
     0: lit_fixnum

After the final botched Typo upgrade I gave up on it and am now serving this blog from Mephisto (0.7.3). The migration went fine. Luckily I still had a copy from the old Typo database structure since the Typo SVN version is not compatible with the migrations scripts from Mephisto. So far everything seems to work fine, except for the Sections. But I can now tweak Mephisto without fear. Typo has become such a huge opaque monster making changes felt like gambling. The last Typo version I checked out had over 35.000 files in the trunk!

The only downside I see so far with using Mephisto is that it's even more obscure than Typo so there is no ready-made migration path to anything else :-)

With the migration completed I will start posting articles again..<ali_g_accent>for real<ali_g_accent>

Update: Mephisto's SPAM filtering (through Akismet) actually works and the approval process is very well done. I also like the Overview section.

Extracting HTML from Firefox DOM

We first started out with attempting to rewrite the table heavy design to CSS. We soon realised that this was a project on itself so we decided to work with what we have. Since most of the HTML code is intermingled with PHP logic it was really not an option to examine the numerous .php files and cut ‘n paste what we needed. We basically used the excellent Firefox View Formatted Source extension to take snapshots of the DOM tree of rendered pages (look here if you’re on FF2).

Writing down business logic

The original project never had functional or technical specifications so I scheduled 3 sessions spread over 2 weeks with the (back-end) users of the system. We identified the business logic and rules of the system as best as we could during these (1-2 hour) sessions. The whole business logic was then written again from scratch in Ruby. I figured it would have take much longer if I had examined the PHP code myself. This might not be an option for you though. Anyway, during these 2 weeks at the end of each day a small number of functionalities would be delivered for testing. In practice the user(s) would only test functionality and provide feedback every 3-4 days (busy schedules on both sides, so noone enforced this). Could have been better.

TDD, or rather WTAD (Write Test After Developing)

It was really really hard to bring up the discipline of writing the Test code before Developing the functionality. In practice tests were only written after something was seen working already. What do you do, honestly? :)

The final part will contain details about the deployment (Mongrels, W00t!) and also some interesting statistics on how many lines were left after the rewrite. Stay tuned..

In recent weeks I was busy converting a fairly large PHP application to Rails. The existing PHP application is about 65.500 lines of intermingled PHP and HTML/CSS code. Yep, a classic PHP application without any database abstraction layer, no templating, no MVC. This is why I dubbed it “large”, but replacing that with “crappy” would be fine too :)

I divided the project in a couple of deliverables:

1. Data model analysis, redesign, data migration
2. HTML conversion to layout/partials
3. Business logic analysis and conversion to Ruby
4. Integration with external web application
5. Testing
6. Production deployment

Since I wanted to get something visible as soon as possible and with semi-live data the first thing I did was analyze the current data model. The model consisted of 46 tables. 8 tables were discarded right away (code/data rot). After further analysis the table count was reduced to 23. Next up was converting and migrating the current data set to the new data model. The PHP site was using Mysql 4.23 while the Rails version would be using PostgreSQL 8.1. This was BTW also the first Rails project where I religiously used migrations and migrations absolutely rock!. A total of 53 migration scripts were generated during the course of development. Back to conversion; Instead of writing CSV/YAML exporter/importer scripts I used the following mechanism to import/convert legacy data objects:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# Read in Rails config
config = Rails::Configuration.new

# Set up the class for the table/objects we want to convert
# Of course the legacy database does not follow Rails conventions
# so we always use table_name to fix this
class OldVenue < ActiveRecord::Base
  def self.table_name
    "adressen"
  end
end

# Set up a connection to the legacy MySQL database
ActiveRecord::Base.establish_connection config.database_configuration["old_production"]

# Save the connection
mysql_connection = OldVenue.connection

# Establish a connection to our new shiny PostgreSQL database
ActiveRecord::Base.establish_connection config.database_configuration["production"]

# But restore the mysql connection for the legacy ActiveRecord class
OldVenue.connection = mysql_connection

The above code allows the conversion script to access both MySQL and PostgreSQL databases simultaneously.This means I can do something like:

1
2
3
4
5
6
7
8
9
10

old_venues = OldVenue.find(:all)

for o in old_venues do
  n = Venue.new
  n.id = o.id
  n.name = o.naam
  n.zip = o.postcode
  # More field assignments
  n.save
end

..Instant data migration from MySQL to PostgreSQL without messy CSV/YAML export/import!