Williams broke the deadlock in the 68th minute and it was enough to earn Paulo Sousa's men all three points
Coventry manager Chris Coleman decided to leave new loan signing Jack Cork on the bench and named the same starting 11 which had ground out a draw at Doncaster in midweek.
But for the first 10 minutes it was the visitors who were playing the more enterprising and had the home side rocked back on their heels.
Stephen Dobbie, playing up front as a lone striker, had two early opportunities. He just failed to reach a good throughball from Mark Gower and then had an effort well blocked by Martin Cranie inside the six-yard box.
Coventry were then very lucky to survive a penalty claim when Stephen Wright appeared to block a cross with his elbow only for the referee to turn away unimpressed.
Leon Best went close for Coventry in the 10th minute when he ended a well-worked move by putting his shot into the side-netting.
Last season, Aron Gunnarsson's long throws proved to be a potent weapon and it should have helped them take the lead in the 13th minute.
Swansea allowed the ball to bounce and it reared up and hit Andrea Orlandi on the arm. This time referee Crossley pointed to the spot.
Sammy Clingan, who scored four spot-kicks for Norwich last season, stepped up but skewed his effort wide.
Coventry continued to offer more attacking threat despite not really playing flowing football.
But Swansea ended the half on a high when Chad Bond got in front of a defender to meet a left-wing cross but the ball flew wide.
Neither side managed to exert any dominance at the start of the half, although Dorus de Vries had to really stretch to palm a cross from Patrick van Aanholt away from four Coventry players.
It was going to take something special to break the deadlock and it came in the 68th minute.
Leon Britton's spectacular overhead kick looked to be heading for the net but it smacked against a the bar and fell for Williams, who calmly slammed it home past the helpless Keiren Westwood.