I just tried this on my stock Varnish config and the site was still responding to requests. So, Varnish can be added to the list of 'not affected.'