A significant contributor to the low fuel economy of buses is frequent stop-start. This can be addressed through less frequent stops (on some lines, there are equity issues for people with impaired mobility) and signal prioritization. Top of my dome, city buses get 3-5 MPG on average on city roads but could be cheesed as high as 8 MPG in ideal circumstances!

Given average fuel economy in the US is somewhere between 25-33 MPG, and the average vehicle occupancy is 1.5 MPG, the breakeven point goes down to 4.7-6.2 passengers! This is only based on gallons of fuel consumed though. It does not take into account embedded energy in the vehicle itself, or the fact that most passenger vehicles use gas whereas busses mainly use diesel which has 13% more energy per gallon.