Gillingham had led through Dennis Oli's 22nd minute strike, but Bauza levelled matters in the 44th minute before stunning the home side with his second a minute later.
Swansea will now play in the second tier of English football for the first time since 1984 and are now strong favourites for the League One title.
With Gillingham fighting for their lives and Swansea requiring a solitary victory to secure promotion, both sides set out to attack from the outset.
The home side began the brightest and were rewarded for their early pressure when Oli spun and fired home inside the area after Simon King's deflected effort had fallen kindly to him.
The Gills continued to dominate with Oli and Andrew Crofts both missing chances, but as the half drew to a close Swansea turned the game on its head with two goals on 44 and 45 minutes.
Home keeper Derek Stillie could only parry Andy Robinson's speculative shot from 20 yards and, as the keeper appeared to grab the rebound, Bauza poked the ball in for the equaliser as the Gillingham players stood in protest.
Stillie was at fault again for Swansea's second after parrying another Robinson shot to Bauza and allowing the striker to slam home the rebound to send the visiting fans into delirium.
A shell-shocked Gillingham struggled to re-create the rhythm they displayed in the first half and were limited to a couple of half-chances as Swansea protected their lead.
Simeon Jackson narrowly failed to convert Nicky Southall's cross with ten minutes remaining as the home side desperately chased an equaliser to boost their survival hopes.
But Swansea proved more than capable of holding on and, after Jackson fired another chance wide in injury-time, their promotion was confirmed at the final whistle.