Because the lunch and dinner services are different. Because we later vacationed in Japan with a couple that we are friends with and they wanted to go and were more comfortable going with us (both of these places are an intense experience, to put it mildly, though one is a more comfortable atmosphere than the other). Because of that intense experience and the atmosphere that it provides.

Because it's more difficult still to comfortably navigate those smaller towns and regions, outside of Tokyo or Osaka or Kyoto, etc..., without any sort of handle on the language. That alone implies a few things. You or someone you're traveling with knows the language well enough to navigate those smaller towns and regions and that it's very likely not your first or even second trip to the country.

We didn't come to understand that sushi in a place like Shimoda could be better than what you'd find at a Michelin rated sushi restaurant until we were able to find ourselves in Shimoda having that sushi and we weren't comfortable even going to a place like Shimoda until our third trip to Japan.

edit/ Because my wife and I are just as vulnerable as everyone else that is being talked about here for falling prey to "top 10 lists" and the like.