Batteries taste sour because the hydroxide ions (formed as a result of electrolysis) activate the sour receptors. When you touch the lead of a battery, your saliva causes the electrons from one end of the lead to start moving to the other, creating an electric current. Also the electrons, on the way, interact with water molecules in a process called electrolysis, and gives out hydrogen and oxygen gases, along with hydroxide ions.

THIS IS COOL. I WANT TO LEARN SOMETHING ELSE, TOO!

Video via – SciShow
Further Readings & References @ Sparkonit, ACM Digital Library, and ResearchGate