Striker Baptiste scored the all-important equaliser in Havant's 1-1 draw at Swansea's Liberty Stadium to earn the Hampshire part-timers a replay.
The former Chelsea trainee struck again to help the Blue Square South minnows to leave League One leaders Swansea red -faced.
Life-long Liverpool fan Baptiste secured the tie of his dreams with the cruicial third goal eight minutes before half-time as Havant stunned their lofty opponents by storming in to a 3-0 lead.
Former QPR striker Richard Pacquette gave the non-leaguers a dream start when he headed home Brett Poate's cross as early as the fourth minute.
Poate was sent off in the first game for a horrific two-footed tackle and will be suspended for the trip to Anfield on January 26th.
Plasterer Poate was the provider again in the 25th minute when his centre was rifled in by captain Jamie Collins.
Baptiste stunned the 500 travelling Swansea fans when he slid in at the back post to notch the home side's third in the 37th minute after Pacquette had fired across the six-yard box.
Guillem Bauza gave the Swans hope with a 25-yard stunner two minutes later. But the turning point came in the 40th minute when Leon Britton had a penalty saved by Havant's builder goalkeeper Kevin Scriven.
Jason Scotland cut the deficit further three minutes after the restart when he tapped in from close range after Bauza's shot had struck the post.
But Havant substitute Tom Jordan wrapped up a bumper Anfield pay-day when he headed in Poate's cross in th 65th minute.