The Swans started brightly and dominated possession throughout the entire 90 minutes and the three points brings to an end their record-breaking sequence of drawn matches.
Darren Pratley gave an energetic performance throughout, although it will be Scotland who takes the credit for his double strike.
Pratley went close with a strike from 20 yards that had Burnley goalkeeper Brian Jensen scrambling to his left.
All Burnley could muster in the opening half was a header from Chris McCann after striker Martin Paterson had burst past central defender Ashley Williams.
The visitors went in front on 33 minutes when Burnley's central defender Michael Duff was shown the red card by referee Mike Oliver for hauling back Scotland when the striker was through on goal.
Scotland easily converted the spot kick, striking the ball high into the net and giving Jensen no chance.
The red card was Burnley's sixth of the season and discipline is clearly something which manager Owen Coyle will have to work hard on even accepting that some of those decisions may have been harsh.
Swansea continued to enjoy the vast majority of possession without creating too many clear-cut chances.
Winger Mark Gower spurned two opportunities at the start of the second half, while at the other end the closest that the 10 men came to an equaliser came when Wade Elliott crossed to the far post and substitute Steven Thompson knocked the ball back for McCann to touch it just past the post.
The points were secured six minutes from time when Gower cruised past makeshift right-back Elliott and picked out Scotland to score with some ease.
Striker Paterson went close to pulling a goal back when his cross was deflected onto a post.
After drawing their previous eight games, the three points from this match enhances the Welsh side's prospects of maintaining a play-off challenge in their first season at Championship level.