We don't have many neighbors out my way, but I do check the houses out everytime I go by. Silly really, it's not like I don't already know what's there, LOL.
But when I go into town I do check out all the houses and stuff. I don't think it's rude. It's either thoughts of admiration or boye! they need to mow their yard, etc. LOL
My favorite is passing farm houses w/ horses. Not only do I check out their homes, landscaping, and colors, but their horses and any other living thing they might have.