I will say, as a Stannis fan, it really pisses me off that he 1.burned the statues of the seven, 2. allowed his men to attack the sept at Dragonstone, 3. burned the godswood in Storm's End, and left Selyse unpunished for burning 4. Sunglass and the Rambton boys. Atheist or not, if he allows his men to attack other people's faiths, his reign is going to be bloody one.
I'm hoping he 5. doesn't burn the godswood in Winterfell.
1, 2 & 3 was Stannis destroying his own property. It may be insensitive to others beliefs, but I don't think it was wrong.
4. This is a issue, I agree. The Rambton were traitors, they took up arms against him when he wanted to destroy his own property. Sunglasse is more pitiable as he abstained pacifically. But burning them as Selyse did was pretty terrible.
I understand why Stannis didn't punish her. It undermines his authority for her to have done with without him approving or knowing. So he must pretend it was sanctioned. On top of this the only troops he still has after the Blackwater are largely Red-God Worshipers or Florents (or both).
5. I don't think Stannis would be that foolish. He knows how inflammatory that would be. Likewise he appreciates that his army is drawn from many different faiths when he commands his men to "pray harder" instead of another sacrifice.