So last Wednesday I submitted Jobs Tractor to hacker news. I was hoping to get an answer to the question ‘Is this useful? could it be useful?’ before I committed any more time to the project. The response on hacker news, twitter and email was very much Yes. So after a little too much excitement (editor@techcrunch.com hasn’t responded to my email yet!) I settled down this weekend to fill in some of the gaps.

The main thing missing was that although I had a form on the site to allow people to sign up for  a weekly email of jobs near them I’d not implemented the associated emailing functionality. I’m planning another blog post about how my attitude to building things has changed from earlier this year but this is a key part of it, put the form up and wait for someone to fill it in. You can worry about the rest of the work after and more importantly if that happens.

It’s taken quite a bit of time to get all this working over the weekend, some good resources I can recommend if you’re doing something similar:

1. Jeff Atwood’s exceptional post on the subject of sending emails
2. Amazon SES, although Postmark was also highly recommended to me. The reason I went with Amazon is that the emails I am sending are classed as Bulk as opposed to Transactional. Transactional means the user has done something which you are emailing them about, Bulk means they’ve opted into a mailing list.
3. This excellent Amazon SES wrapper for node from Jim Jenkins
4. These email templates from the fine people at Mailchimp of which I adapted as a jade template.

That’s pretty much been the sum of it for the weekend. I had to fix a bunch of incorrectly geocoded sign ups and fix the bug which had allowed them to happen. There was a lot of looking at emails I was about to send out and looking through the emails and associated locations I’d collected. Certainly the downside of launching a less than 100% baked product but worth it I think and hopefully none of the people receiving emails tomorrow will be any the wiser.

One interesting challenge was working out how best to structure the email service to avoid sending too few or (worse) too many emails to people. I decided two things, the first being that this is my new interview question for developers. The second that the solution was to mark each email in the mailing list with the date it should next be emailed to. To check hourly if any of those dates are in the past and if so compile the email and put it in a separate queue for sending. In parrallel there is a check every few minutes for entries in that queue, entries are removed and an attempt to email them made. Failure to send results in them being put in a failed email store I can inspect later. Removing them from the queue before I try to send them means there should be zero chance of me accidentally sending the same email multiple times. As anyone who’s ever worked with programmatic emailing knows, a computer can send a lot of emails very quickly if you cock that bit up.