The Lions led early on thanks to a deft lob from their forward Joel Nouble and the same player had another chance, as did Ayo Obileye, to add to their tally.
Rangers found the going tough against the West Lothian outfit and took time to settle, not testing Livingston’s debutant goalkeeper Shamal George often enough.
But Rangers substitute Scott Arfield levelled with a glancing header in the 72nd minute before skipper Tavernier quickly curled in a free-kick to the relief of the Light Blues support whose nerves were frayed.
Rangers will travel to Belgium for the first leg of their Champions League third qualifying round clash with Union Saint Gilloise on Tuesday with more optimism than they might have had they not grabbed those two crucial goals in two minutes.
Andrew Smith took in the match and gives the Rangers players a mark out of ten – with one new boy suffering a torrid afternoon.
5. Borna Barisic
Initially tentative coming forward and troubled by the physicality of the West Lothian team, the Croatian did settle as Rangers took a firm grip after the break. 6 Photo: Rob Casey - SNS Group
6. Glen Kamara
Picked up a booking and found the going tough in midfield. 5 Photo: Rob Casey - SNS Group
7. John Lundstram
Oddly for a player not shy to mix it, the midfielder seemed affronted by the scrapping from his opponents, and allowed his focus perhaps to waiver. 5 Photo: Rob Casey - SNS Group
8. Tom Lawrence
A game was being played in the near vicinity of the former Derby County captain, but in the first half he couldn’t engage in it. Slightly better when moved out to the right at half time. 6 Photo: Rob Casey - SNS Group