The Stutes had reached round three by disposing of Durham City, Swindon Town and Leeds United - in a famous 1-0 second round giantkilling.
Championship outfit City arrived at the Glassworld Stadium having broken a run of eight successive draws with a 2-0 weekend win at Burnley.
A fast-paced opening saw the home side make a bright start, with Swans keeper Dorus de Vries gathering a shot from Antonio Murray as Histon targeted another shock.
However, the visitors hit the front 22 minutes in as Gorka Pintado collected a knock-down from Guillem Bauza, found space 20 yards out and rifled an unstoppable shot past keeper Danny Naisbitt.
Histon responded well with both Josh Simpson and Damien Reeves threatening, before City doubled their lead seven minutes before the break.
Bauza was the goalscorer this time, after Andrea Orlandi fine run and pull-back had split open the home defence.
Histon made a double substitution at half-time in an attempt to turn things around and soon after Daniel Wright headed a Murray cross narrowly wide of target.
However, Thomas Butler almost made it 3-0 from an Orlandi cross as the Swans continued to carve out chances.
Goalscorers and Bauza and Pintado both fired shots over for the visitors, before a goalmouth scramble at the other end finally saw de Vries come to his side's rescue.
Bauza again went close with a header from Orlandi's corner as City continued to hold the upper hand, before Pintado missed a penalty eight minutes from time after Naisbitt had felled sub Nathan Dyer.
Histon then set up a thrilling finale as Simpson pulled a goal back with six minutes left on the clock, netting from close range after Mat Mitchell-King's effort had been blocked.