A
hydrophone is another thing on my potential shopping list at the moment, to complement my underwater camera. Again, pretty cheap at £30. Since this is quite niché, I think if you want higher quality you end up looking at much much more expensive devices.
FWIW, as a graduate of 'Sonic Arts' at Middlesex, I have more experience of the unusual side of things than I do of actually recording normal music.

sadly, I hear that course is being shut down.
I wish everyone happy listening, playing, tweaking... mostly listening. Re the comment earlier about carrying a recorder around routinely: even if you don't gather useful sounds for music, this can completely transform your perception of the environment. Note to self: do more of this.
For anyone interested in field recording of nature; it may be worth setting up equipment and leaving it to record for a while, so that any person-shy creatures are not scared away and you don't have to stay unnaturally still for prolonged periods. I've not tried this myself, I read it at
http://www.franciscolopez.net/amazon.html ... guess things are a bit different in the Amazon, though.