For reference, sound travels 0.343 meters per millisecond, so 10ms latency is the equivalent of sitting 3.4 meters from a speaker and playing your keyboard.
Not sure what you are saying, the time it takes after you hit a key for the sound to reach your ears is 10ms when sitting at that distance. It's the same time it takes if you use headphones and use a plugin with a 10ms latency. If anything the speaker situation could be worse since you may add room reverbation to the sound as well. If this is an issue or not depends on the player and how sensitive they are to this.
