The Blues had gone in at the break trailing 2-0 facing a mountain to climb.
But, inspired by the introduction of substitute Jayden Braaf who scored twice, City turned a remarkable game on its head.
Skipper Luke Bolton narrowed the deficit before Braaf scored twice to put us ahead and then Taylor Richards rounded off a night to remember.
With the prize of a place in the quarter-finals at stake, there was plenty of incentive for both teams.
But it was the hosts who seized the early initiative, securing a crucial fourth minute lead which they never relinquished.
From a whipped in corner, dangerous full-back Kgosi Ntlhe clipped the ball back into the box and Jimmy McNulty rose to head home past Daniel Grimshaw into the far corner of the net.
With their tails up, Dale were on top in a dangerous first 20 minutes.
Ntlhe played a superb ball through for striker Matthew Done which was smothered by Grimshaw before, from yet another inviting Ntlhe cross Rory Holden’s shot was blocked by the Blues keeper.
Gradually however City found their rhythm with Taylor Richards seeing his shot blocked by Dale keeper Magnus Norman after a fine one-two with Ian Poveda.
City then conjured up a golden chance to draw level on 24 minutes.
Full-back Jeremie Frimpong launched a lightening raid down the right wing before being brought down in the box to earn the Blues a penalty.
However, Richards could only direct the spot kick narrowly to the side of Norman who gratefully smothered the ball.
And the hosts made City pay for that miss on the half hour by doubling their advantage.
Stephen Dooley played a neat give and go with Oliver Rathbone before drilling a low shot inside the right hand post.
It meant the Blues went in at the break with a mountain to climb.
But Harsley’s troops possess grit as well as guile and they staged an amazing comeback which stunned the hosts and spoke volumes as to our character.
With Poveda switched to the right flank and Bolton upfront City looked much more threatening.
And as Dale keeper Norman attempted to play the ball out he was closed down by Bolton.
The City skipper robbed the ball and then had the presence of mind to score from a tight angle.
It was just the boost Harsley’s charges needed and they took the game to the hosts creating a plethora of chances.
And the introduction of substitute Braaf on 55 minutes proved decisive.
The 16-year-old left winger gave a hint of what was to come with a shot that whistled wide before on 70 minutes he levelled affairs.
Pozo and Bolton both threatened after City had cut open the Dale defence and Braaf bundled in form close range.
The Blues kept our foot on the throttle and on 82 minutes Braaf struck again.,
The winger went on a mazy run, cutting a swathe through the Dale defence before applying a superb finish.
And City rounded off a remarkable comeback two minutes later when Taylor Richards atoned for his penalty miss by heading home Poveda’s cross to the delight of the 500 travelling fans.
How we lined up
Daniel Grimshaw was in goal with Nathanael Ogbeta, Taylor Harwood-Bellis, Philippe Sandler and Jeremie Frimpong forming the back four.
Iker Pozo, Taylor Richards and Felix Nmecha were deployed in midfield with Benjamin Garre and skipper Luke Bolton on either flank. Ian Poveda was the spearhead up-front.
What it means
City now await Thursday's last eight draw of the Checkatrade Trophy, with Wembley now just two games away.
Our EDS side return to action next Monday evening when we travel to take on Arsenal in a Premier League 2 clash.