Just tell the bees the rent is going up, that should get them to leave. Free honey sounds nice until it starts dripping through your ceiling. Bet the beekeeper was happier about the find than you were. Always something with old houses.
Wow, my friend had something just like that happen in her old house, but it was inside the wall. She kept hearing this weird humming for weeks. Turned out a whole colony had moved into the space between her bedroom and the outside wall. The beekeeper had to cut a small hole in the drywall to get them all out safely. It was a huge mess, with honey slowly soaking into the wall. She had to replace a big section of drywall and the insulation after.