I'm pretty OK with that. They're good people, but they lead vastly different lives. That makes it hard to really get to know them. Basically, they're friendly strangers that happen to live in the same location.
We forgot their names. They introduced themselves, of course, but if you never use the names, you forget them.
Until we got a christmas card in our letterbox from the right-hand neighbour. She forgot our names as well (the card was adressed to our house number), but she signed it with her name. So now we have a written reminder of that! We put a card in her letterbox in return, signed with our names. We did the same for the other neighbours, and we got a card back from them as well.
It allows everybody to ask for the names of everyone else, without having to admit the embarassing fact that you forgot it during a conversation. Ideal. :)