It's mentioned several times. I can quote the occurences in the three first books :
Game 23, Jorah to Dany :
"The common people pray for rain, healthy children, and a summer that never ends," Ser Jorah told her. "It is no matter to them if the high lords play their game of thrones, so long as they are left in peace." He gave a shrug. "They never are."
Game 45, Cersei to Ned (and Ned remembers it twice) :
"You should have taken the realm for yourself. It was there for the taking. Jaime told me how you found him on the Iron Throne the day King's Landing fell, and made him yield it up. That was your moment. All you needed to do was climb those steps, and sit. Such a sad mistake."
"I have made more mistakes than you can possibly imagine," Ned said, "but that was not one of them."
"Oh, but it was, my lord," Cersei insisted. "When you play the game of thrones, you win or you die. There is no middle ground."
Game 58, Varys to Ned :
"The High Septon once told me that as we sin, so do we suffer. If that's true, Lord Eddard, tell me... why is it always the innocents who suffer most, when you high lords play your game of thrones? Ponder it, if you would, while you wait upon the queen."
Game 71, Stevron Frey to the northern lords :
"My lord father would urge caution," aged Ser Stevron said, with the weaselly smile of a Frey. "Wait, let these two kings play their game of thrones. When they are done fighting, we can bend our knees to the victor, or oppose him, as we choose. With Renly arming, likely Lord Tywin would welcome a truce… and the safe return of his son. Noble lords, allow me to go to him at Harrenhal and arrange good terms and ransoms…"
Nothing in Clash
Storm 24, the Liddle to Bran :
"The Bastard's boys, aye. He was dead, but now he's not. And paying good silver for wolfskins, a man hears, and maybe gold for word of certain other walking dead." He looked at Bran when he said that, and at Summer stretched out beside him. "As to that Wall," the man went on, "it's not a place that I'd be going. The Old Bear took the Watch into the haunted woods, and all that come back was his ravens, with hardly a message between them. Dark wings, dark words, me mother used to say, but when the birds fly silent, seems to me that's even darker." He poked at the fire with his stick. "It was different when there was a Stark in Winterfell. But the old wolf's dead and young one's gone south to play the game of thrones, and all that's left us is the ghosts."
Storm 61, Littlefinger to Sansa :
"Why should I wish [Joffrey] dead?" Littlefinger shrugged. "I had no motive. Besides, I am a thousand leagues away in the Vale. Always keep your foes confused. If they are never certain who you are or what you want, they cannot know what you are like to do next. Sometimes the best way to baffle them is to make moves that have no purpose, or even seem to work against you. Remember that, Sansa, when you come to play the game."
"What... what game?"
"The only game. The game of thrones."