Gloucester scored twice in the last six minutes of the game to snatch a point at the Martin and Co Arena.
Trinity looked to have sealed all three points when Craig Reid calmly slotted the Blues into a 3-1 lead before Tigers substitute Luke Hopper deflected a shot into the back of the net to grab a point.
It was the second time in five days that Gainsborough had let in a stoppage time equaliser after Alfreton on Monday.
Gainsborough boss Dom Roma made one change to the side which drew with Alfreton with left back Stephen Brogan passing a fitness test to return to the side in place of his manager.
Trinity started brightly on a sunny afternoon at the Martin and Co Arena and maybe should have been ahead inside five minutes.
Jamie Yates when a free kick down the right hand side and Brogan found Jake Picton at the far post, with his header going all the way across goal with nobody getting a touch.
Gloucester then responded with some shots from range but they did little to worry Jan Budtz in net for the Blues.
A Budtz clearance then worried City centre half Sam Avery. Avery attempted to take the ball down under pressure, instead controlling the ball into the path of Nathan Jarman who pounced on the error and placed the ball into the back of the net to give Trinity the lead.
Gloucester responded well to being a goal down and an air of nervousness spread throughout Gainsborough, as the visitors began to keep Trinity in their own half.
Joe Hanks almost got the Tigers level with a shot from the edge of the box but Budtz just scrambled across in time to turn it away for a corner.
The Blues did look dangerous on the counter and should have extended their lead a minute before the break when Brogan advanced down the left hand side, played a quick one-two with Jarman. The former Rotherham man put in a low cross for Marc Newsham but the former Boston man could not get on the end of it.
The Tigers then punished Trinity in injury time. Ed Williams turned his man and found Louis Robles in the box to head home.
City started the second period on the front foot, twice coming close with long range strikes.
Against the run of play though, Trinity got their noses back in front.
Jamie Yates burst away from his marker down the right hand side and curled in a cross that was a little behind Marc Newsham. The former Boston striker quickly reshuffled his feet and acrobatically found the back of the net.
Gloucester should have equalised ten minutes later when Hanks got a shot away from eighty yards after a corner was not cleared but he could not find the target, blazing wide.
The Blues took advantage of the mishap, with Jarman proving a menace once more. The former Barnsley striker came away with the ball after an aerial duel, played a perfectly weighted through ball to Craig Reid who rolled the ball passed the visiting keeper.
The game looked all but over at this point until City grabbed a goal back with six minutes to play. Hopper scrambled the ball over the line following a Robles cut back.
The inevitable then happened as with 92 minutes on the clock, Hopper diverted a long range Hanks strike passed Budtz to the delight of the travelling contingent.
Gainsborough Trinity: Budtz, Lacey, Brogan, D’Laryea, Quinn, Picton, Davis, Russell, Newsham (Reid 64), Jarman, Yates. Subs (not used): Roma, Binns, Bemrose, Hedge.
Gloucester City: Singh, Green, Jones, Cundy (Sterling 83) , Avery, Dinsley, Webb, Hanks, Bishop (Hall 78), Robles, E Williams (Hopper 64). Subs (not used): Hamilton, M Williams.
Gainsborough Goalscorers: Jarman 14, Newsham 56, Reid 80.
Gloucester Goalscorers: Robles 45, Hopper 84 90.
Bookings: Jarman, Quinn; Jones, Robles.
Referee: Mr A Tankard. Assistants: Mr M Sowerby, Mr A Bannister.