The Swans were already without injured duo Adebayo Akinfenwa and Darren Pratley and were further weakened after just 20 minutes when experienced defender Kevin Austin went off injured.
It was not the start new manager Roberto Martinez wanted. He was given the job on Friday but took a back seat to allow caretaker-boss Kevin Nugent to take charge of the team for the third time.
The first half was goalless and produced very few chances. In the 19th minute Yeovil seemed destined to take the lead when Marcus Stewart turned quickly and fired low to the right of Andy Oakes. But Oakes, given the jersey in place of Willy Gueret, made a superb reaction save at the expense of a fruitless corner.
Swans striker Lee Trundle can always be relied on for the unexpected and he had their best chance of the first half in the 36th minute. Steve Mildenhall put a mis-directed clearance straight to him and he tried to lob the keeper as he scrambled back to his line but succeeded in getting the ball over 6'4" custodian but wide of the mark.
The second half was two minutes old when Swansea thought they should have had a penalty. Andy Robinson went down under a challenge from Martin Brittain on the corner of the box but referee Tony Bates ruled he had dived.
The goal that won the game came on 56 minutes. Yeovil were awarded a free-kick 35 yards out on the left. Nathan Jones floated the ball into the box and it was flicked on to the far post where Stewart nodded in from close range.
In the dying minutes, Jones came to Yeovil's rescue with a goalline clearance from Robinson, but a Swansea goal at that stage would have been a slight injustice to Yeovil who generally had the better of things.