I don't think it's rude either. I always prefer the doorbell over knocking anyway. The letterbox is for letters. If it was supposed to be used to knock on someone's door, they would have called it...