It isn't that Ned was a failure at The Game of Thrones so much as that he didn't actually play the game at all. No one in his right mind brings in a Ned Stark to "out intrigue" a Varys. As a ruler in the North, the Game of Thrones was essentially ineffectual against Ned. If Ned was vulnerable to the game, Roose Bolton would either be sitting in Winterfell or running it as a puppet government and not cutting out the tongues of smallfolk in fear that they might run to Ned Stark.
Since Ned wasn't vulnerable to the Game as a ruler in Winterfell, wasn't made Hand for his prowess as a player, and never really engaged in the game, is viewing his failure as a player the proper way to look for his tragic flaw? If not what was it? I suspect it is the attempted redemption of Robert.
Consider the Varys/Illyrio conversation:
“The gods alone know,” the first voice said.
Stannis Baratheon and Lysa Arryn have fled beyond my reach, and the whispers say they are gathering swords around them. The Knight of Flowers writes Highgarden, urging his lord father to send his sister to court. The girl is a maid of fourteen, sweet and beautiful and tractable, and Lord Renly and Ser Loras intend that Robert should bed her, wed her, and make a new queen. Littlefinger… the gods only know what game Littlefinger is playing. Yet Lord Stark’s the one who troubles my sleep. He has the bastard, he has the book, and soon enough he’ll have the truth. And now his wife has abducted Tyrion Lannister, thanks to Littlefinger’s meddling. Lord Tywin will take that for an outrage, and Jaime has a queer affection for the Imp. If the Lannisters move north, that will bring the Tullys in as well.
Ned and Robert argue in just about every scene they are together.
Ned’s mouth tightened in anger. “Nor will I. Leave it be, Robert, for the love you say you bear me.
The king was not pleased. He took his arm from around Ned’s shoulders. “Jon’s service was the duty he owed his liege lord. I am not ungrateful, Ned. You of all men ought to know that
“Robert, you cannot mean this,” Ned protested.
The king was in no mood for more argument. “Enough, Ned, I will hear no more.
Given Robert's absence it seems that as Hand Ned was technically free to effectively act as King.
He chose not to.
I think Ned wanted to pull Robert back into his rightful role as King and redeem his friend. Here is where we see this and see the Robert whom Ned is looking for.
“With all my heart,” Ned said.
“Let me tell you a secret, Ned. More than once, I have dreamed of giving up the crown. Take ship for the Free Cities with my horse and my hammer, spend my time warring and whoring, that’s what I was made for. The sellsword king, how the singers would love me. You know what stops me? The thought of Joffrey on the throne, with Cersei standing behind him whispering in his ear. My son. How could I have made a son like that, Ned?”
“He’s only a boy,” Ned said awkwardly. He had small liking for Prince Joffrey, but he could hear the pain in Robert’s voice. “Have you forgotten how wild you were at his age?”
“It would not trouble me if the boy was wild, Ned. You don’t know him as I do.” He sighed and shook his head. “Ah, perhaps you are right. Jon despaired of me often enough, yet I grew into a good king.” Robert looked at Ned and scowled at his silence. “You might speak up and agree now, you know.”
“Your Grace…” Ned began, carefully.
Robert slapped Ned on the back. “Ah, say that I’m a better king than Aerys and be done with it. You never could lie for love nor honor, Ned Stark. I’m still young, and now that you’re here with me, things will be different. We’ll make this a reign to sing of, and damn the Lannisters to seven hells. I smell bacon. Who do you think our champion will be today? Have you seen Mace Tyrell’s boy? The Knight of Flowers, they call him. Now there’s a son any man would be proud to own to. Last tourney, he dumped the Kingslayer on his golden rump, you ought to have seen the look on Cersei’s face. I laughed till my sides hurt. Renly says he has this sister, a maid of fourteen, lovely as a dawn…”
This was the boy he had grown up with, he thought; this was the Robert Baratheon he’d known and loved. If he could prove that the Lannisters were behind the attack on Bran, prove that they had murdered Jon Arryn, this man would listen. Then Cersei would fall, and the Kingslayer with her, and if Lord Tywin dared to rouse the west, Robert would smash him as he had smashed Rhaegar Targaryen on the Trident. He could see it all so clearly.
The "now that you're here with me" echoes his original request for Ned to be Hand.
That was your moment. All you needed to do was climb those steps, and sit. Such a sad mistake.”
“I have made more mistakes than you can possibly imagine,” Ned said, “but that was not one of them.”
The heart of their conflict plays out in the Dany assassination argument.
“Gods,” the king swore, the word exploding out of him as if he could barely contain his fury. “You mean it, damn you.”
“I will not be part of murder, Robert. Do as you will, but do not ask me to fix my seal to it.”
He laid it on the table in front of the king, saddened by the memory of the man who had pinned it on him, the friend he had loved. “I thought you a better man than this, Robert. I thought we had made a nobler king.”
Ned married his fate to Robert's by choosing to try and redeem his friend. As foolish of a move as telling Cersei he knew of the incest was as far as the Game of Thrones it really didn't matter. For Ned it was about redeeming Robert (and Robert not killing children has added importance given Jon) not about securing power. If Robert hadn't been mortally wounded by the boar, or if Barristan had intervened Ned's telling Cersei wouldn't have mattered. Ned's "huge mistakes" are only mistakes because of the timing, luck, and random chance surrounding Robert's death. Different luck or timing and Ned still wins despite these "mistakes." Ned's fate was married to Robert's regardless of these seemingly inept choices.
Ned's truly fatal mistake was not telling Robert about the incest on his deathbed. Ned's choice to not seize the institutions of power as a ruler when he first arrived meant all his actual power was derived purely through Robert. His choice to try and redeem Robert over ruling as Hand the way he ruled in Winterfell removed the armor an honorable leader has to shield him from the game. After Robert dies Ned is doomed regardless. He may have faced a less unpleasant fate at the hand of Renly but he would have been equally betrayed and equally powerless to stop it. His last hope was to use the power derived through Robert to seize control while Robert still drew breath. It is not being too honorable for the game, but being too honorable to make a dying friend own his mistakes that truly seals Ned's fate.