You could manually turn the bluetooth off on the phone.
The BT module in the Jeep will turn off if you're not in a call within a certain time of shutting the engine off, or when you open the door. It's very possible that your phone simply doesn't realize it needs to disconnect due to very old software.
I'd try turning the BT off before you shut the car off.
...and getting a new phone.