The Canaries belied their lowly position by racing into a two-goal first half lead through Kenny McLean and Todd Cantwell before Sergio Aguero narrowed the deficit just before the break.
On what was a tough evening at Carrow Road, the hosts then further stretched their lead through Teemu Pukki early after the restart.
After continuous City pressure, Rodrigo fired home a wonderful low shot to record his first goal for the Club with two minutes left but despite frantic late pressure, Guardiola's side couldn't force an equaliser.
The onus was on the champions ahead of kick-off given that leaders Liverpool had recorded a 3-1 win over Newcastle in the early kick-off to open up a five-point lead at the top of the table.
But that task soon took on challenging proportions as Norwich stunned the visitors with an early two goal salvo.
Despite missing 11 first team players through injury, the Canaries proved formidable opponents, showing a willingness to play on the front foot.
With space at a premium as Norwich threw a collective green and yellow clamp over the final third of the pitch, City also struggled to find any rhythm and, after a spirited start, Norwich took an 18th minute lead from their first meaningful attack.
Todd Cantwell whipped in a vicious, curling corner and Kenny McLean rose first at the near post to bullet home a powerful header past Ederson.
However, our task then got even harder on 32 minutes as Norwich struck again in clinical fashion.
A slide-rule pass from Marco Stiepermann split the City defence and with a two-on-one advantage, Teemu Pukki perfectly timed his pass to Cantwell who guided the ball into the empty net.
The champions almost immediately narrowed the deficit as Sterling powered in a close-range header only to see his effort rebound back off the post.
However, after surviving a VAR scare when the ball accidentally struck John Stones’ arm, City finally did summon up all-important strike right on the stroke of half-time.
City came out seeking to step up the intensity at the start of the second half… instead Norwich struck for a third time on 49 minutes.
Pep Guardiola’s response was to send on Kevin De Bruyne and Gabriel Jesus for Silva and Gundogan and the impact was almost immediate with Aguero’s goal bound shot deflected and Otamendi then heading a De Bruyne corner straight at Tim Norwich keeper Kruul.
With time ebbing away, Rodrigo set up a grandstand finish by firing home a brilliant low 88th minute drive for his first goal for the Club since his summer move from Atletico Madrid.
As City laid siege to the Norwich goal, Aguero almost curled home an equalizer before Kruul smothered Jesus's drive as Norwich held on.
Sergio Aguero’s goal saw him become only the third player in Premier League history to score in a club’s first five league games of a season.
Jose Antonio Reyes for Arsenal in 2004/05 and Wayne Rooney for Manchester United in 2011/12 have previously achieved the feat.
It also meant that from his first seven efforts on target, Aguero had scored seven times this term, sending him back to the top of the Premier League scoring charts.
Pep Guardiola's reaction
"Congratulations to Norwich. The first goal was from a set-piece and the second on the counter so credit to them. It is what it is. We have to learn from this and carry on.
"Our passing was not bad. We created chances but we could not score them today. They are a really good team with good players with quality, we saw that in the Championship last year, they were clinical today.
"We did not have urgency in the final third that we normally have. In football you can't always avoid mistakes. I don't know how many shots we had or how many they had, but football is about goals and about what you do in the boxes."
What it means
The result, coupled with Liverpool’s earlier win, means the Merseysiders are now five points clear at the top of the table.
They have 15 points from five games, with City in second place on 10 points from five.
City open our UEFA Champions’ League group campaign with a trip to the Ukraine where we will take on Shakhtar Donetsk next Wednesday night. Kick-off at the Metalist Stadium in Kharkiv is 8pm BST.