Except Rhaegar and Lyanna are only married in crackpot theories without textual evidence, and the three KG are already protecting Lyanna and her unborn bastard while Aerys and his heirs are alive. The actual king was living, the potential king was living, and the potential kings real heir was living. Lyannas unborn child was and is not an heir nor does it make sense to protect him because he might be the "potential" king when Aegon is alive.
Those are different instances. After Rhaegar leaves for KL and the three KG stay, they most possibly stay on his order to protect Lyanna and her unborn baby. If Aerys didn't explicitely order them back, they are free to follow Rhaegar's order in this as the highest ranking member of the royal family available and they are not in dereliction of duty as their primary duty to guard the king is done by the remaining four and finally by Jaime when Rhaegar takes Martell, Darry and Selmy to Trident.
Then, however, Rhaegar dies and not much later also Aerys and Aegon. From that moment on, if Viserys is king, Dayne, Whent and Hightower are in dereliction of duty because their new king is without protection of his Kingsguard. No matter what orders they had from whoever, by remaining at ToJ, they are breaking their vows - only they claim very resolutely that they keep their vows and they emphasize their KG status. These are contradicting fact, so if they claim that they are not breaking their oaths, then Viserys is not king. The only way Viserys may not be king is if a legit male child of Rhaegar's is alive (and present at ToJ, so that the KG stay there). A child is legit either by his parents' marriage or by the king's decree, so either Aerys must have issued a decree, or Rhaegar and Lyanna got married. As for the decree, there is not a single hint at the existence of anything like that; the option cannot be entirely ruled out but given the relationship between Rhaegar and Aerys, it is not very plausible. As for the marriage, there are established precedents which make it possible, and it is never stated that the practice became illegal (there is not even a mention of it being a sin against the gods - unlike the incest, which is very much perceived as abominable, yet currently tolerated in the royal family without much fuss).
Now, can you point me to the part of argumentation which seems crackpotty to you?