Rhaenys took after her mother in appearance, she had brown hair and looked like a Martell. Who's to say Aegon the VI didn't have brown hair?
If Lyanna's babe had Valyrian features, Ned may have swiched the babe with Ashara, who had similar features. Nobody would question her raising a baby with those features.
It might also be why Ned feels such a sense of shame when thinking about Jon, because he never fulfilled that promise. HE gave her babe away instead of raising it even if it was to protect her.
Varys may have helped cover up Ashara's death like Connigton. Both seem to barely be remembered. Only by those they touched emotionally. Ashara also killed herself AFTER Ned's visit.
It also seems strange to me that Edric Dayne would ask Arya about both Wylla and Ashara.
Except it's not that Jon has dark hair but that he has the Stark look--it's more than just his hair that people look at and go "dude's a Stark" but his entire facial structure, which is not going to be shared between the Dornish Elia and the Northern Lyanna
But nevertheless, you are right, the timeline does not work for Lyanna's baby to be swapped with Rhaella's.
Well, at least two weeks (Jaime states that Lord Pyromancer was Hand for two weeks), but yes,