Unless you enjoy their company, put about a 1/2 inch of cider vinegar in an open container then add a few drops of washing up liquid. Soon gets rid of them. Of course you will pay for it in Hell, by spending eternity being repeatedly drowned by a giant fruit fly demon. But your bananas will be safe.

If you don't have vinegar, a banana peel or apple core in a jam jar with a splash of water and clingfilm over the top that you stick a fork through

I swear by this method. I keep one out in the summer months