The Lions led early on thanks to a deft lob from their forward Joel Nouble and the same player had another chance, as did Ayo Obileye, to add to their tally.
Rangers found the going tough against the West Lothian outfit and took time to settle, not testing Livingston’s debutant goalkeeper Shamal George often enough.
But Rangers substitute Scott Arfield levelled with a glancing header in the 72nd minute before skipper Tavernier quickly curled in a free-kick to the relief of the Light Blues support whose nerves were frayed.
Rangers will travel to Belgium for the first leg of their Champions League third qualifying round clash with Union Saint Gilloise on Tuesday with more optimism than they might have had they not grabbed those two crucial goals in two minutes.
Andrew Smith took in the match and gives the Rangers players a mark out of ten – with one new boy suffering a torrid afternoon.
9. Scott Wright
Took some heavy treatment from Cancar as he proved one of the more involved Rangers players in the visitors’ attempts to produce a spark in the first half, only to fail to re-appear after the break. 6 Photo: Craig Foy - SNS Group
10. Ryan Kent
The winger remains a hugely frustrating player. Struggled to beat his man for the first hour but then when he did so, and stepped in to cross with right foot rather than left, the equaliser followed. 6 Photo: Craig Foy - SNS Group
11. Antonio Colak
Might have been unfortunate with an offside call when he had headed into the net early in the second period but was sluggish too often across the afternoon. Photo: Alan Harvey - SNS Group
12. Substitutes
Scott Arfield (7, above), was only on the pitch for 25 minutes but bagged the crucial goal. Malik Tillman (7) was impressive in midfield, while James Sands (6) shored up the defence. Ryan Jack made a late appearance. Photo: Craig Williamson - SNS Group