Well, ssh sometimes certainly feels sluggish to me. If you have your aliases configured and are typing at a high speed, you should be basically limited by your fingers.
Terminal use at this level of focus, is inseperable from thinking. You do not think to compose a command nor contemplate to move your fingers to type and execute, it just happens.
Therefore I think latency is also severely important at the terminal level.
This is not an apology for high latency, although, on the other hand, typing at the speed of thought is the main cause for rm -rf / and other similar mishaps.
Terminal use at this level of focus, is inseperable from thinking. You do not think to compose a command nor contemplate to move your fingers to type and execute, it just happens.
Therefore I think latency is also severely important at the terminal level.