Purity and cleanliness are important values in Japanese culture. For example, in Japan people remove their shoes before entering a house. I believe this practice to be rooted in the focus on purity in the Shinto religion. If you come to think about it, it makes sense from a practical viewpoint as well. You leave all the dirt, both visible and invisible, at the doorstep.
Japanese homes have a special area to make the transition from outside to inside, called a genkan. Located at the front door, it is slightly lower than the rest of the house. Shoes are left in the genkan and usually there are slippers for residents and guests alike. Or else people just wear socks indoors.
But the quest for purity doesn’t stop at the genkan. The Japanese take it even further with special toilet slippers. These are slippers to be worn in the toilet only. They are placed at the toilet entrance and whenever you want to use the toilet, you change from your regular indoor slippers to the special toilet slippers. Again it makes sense when you think about it. Why would you go through all that trouble to keep your floors clean and pure, to then spread all the (invisible) toilet nastiness all over the house by walking around.
A few extra things to note about the use of slippers:
Do not forget to leave your toilet slippers at the toilet door after you are done. Stories of forgetful foreigners walking all over the house in toilet slippers are ubiquitous.
When you remove your shoes at the genkan, make sure you don’t step on the elevated part and take them off there, but actually remove them in lower area of the genkan. It is something I used to do a lot in the beginning, but it kind of defeats the purpose of the genkan.
Taking off your shoes before entering a house applies to traditional Japanese homes, restaurants and hostels. In a Western style hotel, for example, people keep their shoes on, even in their room. In some restaurants, you should also keep your shoes on. So the rules about where to take off your shoes may differ a little according to the situation. But no need to panic. If you just try to keep the shoe issue in mind and pay attention to what other people are doing, you should be fine.
Do not step on tatami wearing slippers. I discovered this rule the hard way, after having been severely scolded for stepping on tatami with slippers when I was visiting a temple. I imagine tatami are delicate and the slippers might damage them. You may only step on tatami barefoot or with socks. Socks are preferred because it is more hygienic.
Finally, if you do mess up the slipper thing and someone gets angry at you, try a sincere and humble apology to defuse the situation. Depending on how angry they are, you might try a bow and sumimasen deshita. If they are positively fuming, whip out your deepest bow, hold it for a while and say mōshiwake gozaimasen deshita. Good luck!