Is the host flexible in its server types, supported languages, and features?
If I had a dollar for every time a host told me that they couldn’t add a PHP/PEAR library or wouldn’t allow mod_rewrite, I’d be a rich man. Projects don’t always go as planned and flexibility within your web host is a key component in picking where the site will live. Flexibility generally means a slightly higher price; pay it, because you may pay a steeper price if you don’t.