When you want to move your pet from the shared bank into your bank/backpack (of the other wizard) it takes all your energy. So if you just tended to your plants, you may not have enough left. I don't think there is a setting about your shared bank...I could be wrong, but I never saw one.
I have no problem switching pets. My best character is Fire/life and has quite a few pets.
I can put them in the shared bank and let another character take them out. For example 3 Death pets can go to my death character. Sure I lose Energy for taking out the first one, and it still says you will lose all energy for every pet after that, but I already have none from taking the first pet.
Your bank might be full, in which case you would have to remove something before adding a pet to it. Other than that you should be able to share it through the bank. The pet energy level doesn't matter; it takes away your pet energy so you can't train the pet multiple times in a row with different wizards.