French striker Eric Sabin's double sent Swansea spinning to their 4th league defeat under new manager Kenny Jackett.
It could have been even worse for the Swans as the Cobblers had a first half goal disallowed and missed a penalty on the hour.
"I am feeling low after another defeat but I do know there are players of quality at this club," said Jackett.
"What I do have to address is the defending which was once again poor today. Our defence must quickly get into the real world and perform much better.
"Despite the defeat however there are plus points and that is what we will be working on through the week.
"The next game cannot come quickly enough and I certainly hope we can get that first victory at Darlington." The Swans looked to have gone behind in the eighth minute when their former loan striker Marc Richards blasted the ball home but referee Andy Penn had spotted an infringement.
In the 24th minute Cobblers boss Colin Calderwood was dismissed to the stands after over-vigorously disputing a decision with the fourth official.
This seemed to spur the visitors on and Sabin opened the scoring in the 24th minute.
Receiving a slide-rule pass from Chris Willmott he ran on a few paces before unleashing a shot which beat home keeper Brian Murphy all ends up.
Nine minutes later, he netted again when he held off a challenge from Jonathan Coates before firing into the corner of the net.
The Swans tried hard to reduce the deficit but Northampton should have increased their lead on the hour and once again it was Sabin who caused the damage.
He was brought down in the area by Stuart Jones but leading scorer Martin Smith stubbed the spot kick wide.
Calderwood was still bemused at his dismissal after the match: "I have spoken to the referee and let's just say that our opinion on the incident still differ.
"As for the game we fully deserved our victory and it keeps us on line for a place in the end of season play-offs."