Small carving gouge, can't remember the number, is how I do minor reworking when one side of the hole is out of round.

I got my 5th string tapered reamer from StuMac years ago.....

Years ago, I imported 12 entry level banjos from US based Saga Music. These came with a poor quality 5th string friction tuner peg. Saga included 12 geared 5th pegs for me to install as they recognised that the other ones were poor.
As I had the original Stewmac tapered reamer I found that it would not fit the hole. I didn't want to remove the neck to re-drill the hole larger. I locally purchased a smaller tapered reamer, measured the hole, then cut the reamer with my Dremel tool so that it would just fit into the hole. This allowed me to enlarge the hole for the larger reamer. It worked a treat.

Dremel

Thanks for all your replies! While I am a full time repairman, I rarely work on banjos to this degree, so I do not have the special reamer. Looks like small gouges and maybe a Dremel will work for me.

Without using a reamer I have successfully retrofitted several 5th string pegs that are larger in diameter than the OME peg. I used dowel that I sanded to match the taper of the new (larger Gotoh) 5th peg. I then wrapped and glued 100-120 grit aluminum oxide sandpaper to the taped dowel. This method is a bit more tedious but it does allow small increments of wood to be removed between test fitting the peg. This results in the perfect fit. A dab of white Elmer’s glue and the installation is complete.

The StewMac reamer is an older version of the standard cheapo (Harbor Freight, Amazon etc.) cut to a certain depth. It does not actually match the taper of any 5th tuner on the market. The secret is that it doesn't have to—close really is good enough.

Harbor Freight T-Handle Reamer

I think this was one of the many tools that the late Frank Ford developed for his own use and let StewMac copy. I never asked Frank about his business relationship with them—and he never volunteered that info.

I don’t believe I could start a 5th string reamer into that hole. Looks like the non concentric hole is smaller in diameter than flat bottom tapered 5th string reamer. You might get fret rise or a hump starting right there after you grt fifth fitted and have to level fret. Fifth fret bad to rise up from pushing or tapping fifth in.