Despite taking an early lead through Connor Goldson and finding an unlikely equaliser in the 88th minute thanks to Steven Davis, the Light Blues were finally undone by a Conor McCarthy goal after Jamie McGrath had scored twice to turn the match on its head.
At this point, it looks like it will still be a successful season for the Ibrox side, who hold a 13-point lead at the top of the Scottish Premiership table and are in the knockout stages of the Europa League.
However, last night was a familiar disappointment to fans who are still waiting on their favourites to win a major trophy since returning to the top flight in 2016.
Here are all nine exits that Rangers have suffered in cup competitions since the 2016 Scottish Cup final defeat to Hibs.
5. 2018 Betfred Cup semi-final
New manager in Steven Gerrard but another semi-final defeat. Lewis Ferguson's late header was enough to separate the sides at Hampden. Photo: SNS Group Paul Devlin
6. 2018-19 Scottish Cup quarter-final
Aberdeen dumped Rangers out of the cups twice in one season as goals from Niall McGinn and Connor McLennan won this replay at Ibrox. Photo: SNS Group Craig Williamson
7. 2019 Betfred Cup final
Despite dominating the game, playing against ten men for 30 minutes and being awarded a penalty, Rangers couldn't find a way past Fraser Forster as Gerrard lost his first final as manager. Photo: Craig Williamson - SNS Group
8. 2019-20 Scottish Cup quarter-final
Gerrard's side lost to Daniel Stendel's Hearts twice in the space of six weeks after this exit at Tynecastle. Oliver Bozanic scored the game's only goal. Photo: Craig Williamson - SNS Group