But is the code you're speaking of the security-specific code, or the network-specific code? If you attack a networking protocol, what are you attacking if not the associated cryptography? A networking protocol without any cryptography doesn't really need to be attacked because it's already wide open.
You introduce the confidentiality, authentication and integrity (which is really just going to be part of the authentication) through cryptography and security-specific code.
I'm not saying people don't attack networking protocols, or that there is no incentive for doing so. I'm saying that the portion that plays the defensive role is in the security software, not the overlying networking protocol.
Anything that accepts data from potentially-malicious sources is security-critical if it's itself privileged or produces data/events that go to something else privileged. Such analysis is usually used to determine the "Trusted Computing Base" (TCB) of a system that encompasses everything that might be used to break the security policy. All people who do real, security engineering keep it in mind, minimize it, strengthen it, and document what's left.
Here's an example from high-assurance field of what that looks like where they prototype an architecture for VPN's that minimizes TCB of key components:
Also shows their TCB is a lot bigger than their tiny kernel. Fortunately, there's been high-assurance developments to solve most of those things. I've seen robust, networking stacks but don't think they're high-assurance yet. If they're complex and done in C, here be dragons.
I was thinking of attacking the implementation of whatever processes attacker controlled input. If sending 100 pings in a row triggers a stack overflow, this will interest the attacker.
I would say that security specific code would be any code that directly processes the untrusted inputs. Parsers and decoders, mostly. You may want to isolate them from the rest of the program (ideally a separate address space), and make sure the interface between them and the rest of the program is much simpler than the input they are processing.
At least, you want to ensure such code either produces sound output for the rest of the program or fails cleanly. The attacker will have a harder time exploiting a bug deeper in to the program, I think —hope.
You introduce the confidentiality, authentication and integrity (which is really just going to be part of the authentication) through cryptography and security-specific code.
I'm not saying people don't attack networking protocols, or that there is no incentive for doing so. I'm saying that the portion that plays the defensive role is in the security software, not the overlying networking protocol.