After going ahead late in the second half when centre-back Dennis Lawrence headed home a free-kick, their win was sealed from the penalty spot by Jason Scotland, who netted his 25th goal of the season.
The hotly-disputed penalty was given after sub Febian Brandy had gone down under a tackle from Byron Anthony. With tempers still high, Brandy was red-carded six minutes later after squaring up to home centre-back Danny Coles on the touchline.
He had been in the match just 20 minutes, but they could turn out to be the most vital minutes of Swansea's season, for their 11th away win steadied their promotion bid after they had taken only one point from the previous three games.
The pace that Thomas Butler and Darryl Duffy raised on the wings always caused problems for Rovers' back four. They gave Swansea a more polished look that Rovers, missing David Pipe through injury, could not match.
With one brilliant run Duffy opened up Rovers down the left, but with Scotland crying for the ball, Duffy's shot flew past the far post.
A bigger threat developed from the midfield where the diminutive Andy Robinson sent in a rising, angled shot that Steve Phillips did well to deflect over his bar.
For their part, Rovers managed just one shot, a looping effort from Lewis Haldane which was tipped on to the roof of the net by Dorus de Vries.
Rovers were furious in the first half when they had two penalty appeals turned down. They seemed to have a fair shout when Rickie Lambert was pushed in the back, but referee Ure Rennie was having none of it.
When Haldane went down in the box later in the first half his reward was a yellow card.
Lambert was also booked after Swansea's successful penalty for asking the referee what the difference was between Swansea's award and his own earlier appeal.