City had to dig deep after falling behind to a Nicolas Otamendi own goal on the stroke of half-time.
However, City maintained their composure and were rewarded after the break with a Sergio Aguero penalty drawing them level before Raheem Sterling's dramatic 84th minute goal secured another priceless win.
In a record-breaking season, it was the club's 18th win in all competitions and was City's 11th successive away win, equalling a club record.
And according to Kompany, the win at the John Smith's Stadium was one of the most important of the campaign to date.
"Today was what epitomizes the Premier League," said the City skipper.
"Winter is there, it’s a bit colder and a bit harder to move the ball. They defended well, they were strong up-front and they had a game plan with three defensive midfielders but the team deserved the win because we kept at it.
"Even at 1-0 down we still believed we could do it.
"You don't ever go into a game not thinking it's a big three points, regardless of whether other contenders have dropped points.
"But this one is even bigger because we were 1-0 down and we ran out of options at points. We were creating chances, we were dangerous and then they went 1-0 up and all of a sudden it becomes a different game.
"These are the big wins because of how the game went - especially when you are losing but like it happened against Bournemouth, we scored in the last minute."