Clockwise

Recently I took over hosting duties for a phpbb forum for a group of friends, and friends of friends. It’s not a large forum, but it isn’t small either. The database with search index was about half a gig.

Thankfully the conversion process can be run alongside the existing forum. Granted you should disable the old forum when you are upgrading, but to test the process out it is handy to not have to take the board down. Unfortunately for me the conversion process that comes with PHPBB 3.0.1 was consistently failing. Turning on all the php error reporting resulted in nothing as to why the script was failing. After some searching I read some people having problems with 3.0.1’s conversion had success upgrading to 3.0.0. The process wasn’t flawless, needing to be babysat for the occasional 500 error that thankfully can be overcome with a simple reload of the script. After a couple of hours it could at least complete the process.

I did try immediately upgrading to 3.0.1, but the update package didn’t like something and ended up leaving the board in an unusable state. Attempts to manually update things couldn’t save it. So the process was started over again and left at version 3.0.0 for the time being.

This information might be useful to someone else having problems upgrading a version 2 phpbb forum to a version 3. Good luck, dealing with their update process is a painful reminder of how much nicer it is to migrate a Rails app to a newer version.

I’ve spent many years developing websites. The most successful projects have always had a good foundation design done before the coding started. 37 Signals talks about this in their book Getting Real in chapter 9 Interface First. The interface gives you something concrete to think about and to work towards.

Recently I had a small project come my way, that was basically a simple feed reader. I started by sketching out my interface. The target was simplicity and not a lot of development time. Unfortunately I neglected one vital part of the application in the interface design phase, the requirement that the user be able to sort the feeds.

I dove into the code, setting up the Ruby on Rails project. I found a gem that would simplify reading in all the different feed types and letting me treat them the same. I got the interface in and about 90% of the project coded within an hour and a half. Then my poor interface planning skills hit trying to implement the sort. There were certainly easier and more elegant ways to tackle the problem, but not in the interface I had designed.

The decision became to bolt on the sorting requirement or to rethink my interface. I was already hovering around the expected amount of effort put in. I spent some time toying with some different interface ideas, but nothing came that solved the problem without requiring significant work and making the application more than it was intended to be. So I bent the framework to my purposes and called it version 0.1.

I temporarily had it running on this server. As a coding exercise I might do more with this project and expand it out with a better interface. The interface is definitely the thing that needs to be overhauled and the next would be reworking the tests that are mostly untouched from their generated state.

When I first started coding with Ruby regularly my code was not always The Ruby Way. One task I’ve come back to time and time again is importing data from a text file into a database. Though I can’t recall the specific code used, I believe my first attempt loaded the whole file into memory before running the INSERTstatements. This was needlessly expensive on the machine doing the processing and slow. I had started the process before leaving work and on my commute home puzzled out the better way to do it. The new version that read a line in, inserted the data in the database and then went on to the next line forgetting about the previous took much less time to execute and worked wonderfully.

Later I took over primary development of a large set of Ruby scripts that more or less managed importing and exporting data from MySQL using flat files. This system communicated with corporate headquarters and individual stores about everything, customers, orders, products, special pricing agreements, etc. Processing time varied on file size, but most files took a couple of minutes to process even on new hardware and the largest set of files, a potentially daily input of data, could take up to an hour to get through. Now there was some processing with this data to populate some fields and sometimes joins were necessary to set things up right.

However the client was looking to be able to take this process from a handful of sites to potentially hundreds. The system’s architecture was changed so we could throw more hardware to scale. I also looked for optimizations that could be made. It made sense to do things they way they were, Ruby read in a line of data, performed any calculations or string concatenations and then inserted the data in the database. There was something faster though. MySQL’s LOAD DATA INFILE was brought in to do all the heavy lifting of getting data from the flat files into the database. Then aSQL query could be run to calculate any fields that needed to be calculated. This cut processing time overall for the most common file from minutes to seconds. It was a good reminder of how usually if your database server can do it, it’ll probably do it faster than whatever language you’re working with.