Tuesday, November 4, 2008

iPredict Server Upgrade

iPredict was slammed with traffic around 8:00pm last night as a result of our appearance on Tv3, resulting in one of our web servers crashing and affecting some users.

This was a RAM problem and occurred in spite of testing in March which we thought guaranteed performance on the night. We were wrong.

We have ordered more memory for our servers to be here in time for election night this Saturday and will continue testing in the meantime. iPredict staff will be working right through election night to keep the site running.

I'll keep you posted on progress ahead of the weekend.

3 comments:

PhiRatE said...

Performance testing is a never-ending process of bottleneck discovery. Having done a few heavy sites now the fundamental rule appears to be: you can never be prepared enough.

Fortunately you have a relatively good write:read ratio so caching should be moderately effective for you, and obviously you can deploy stuff like the charts off to static servers if you need to. Amazon S3 is pretty useful for that if you find yourself short of servers and bandwidth/processor limited, but normally static delivery itself is sufficient.

Matt Burgess said...

Thanks PhiRatE, I'll run that past our developers. We do cache our charts which helps a lot but the S3 stuff sounds great.

PhiRatE said...

Sweet as. Let me know if you need some code for it, I've got some command-line python script stuff that does a simple S3 upload and sets all the properties etc. Happy to provide some advice on high-performance configs for linux/apache/postgres as well if you need it.