Cannon balls were originally kept in a piece of metal called a monkey, normally made of brass. When metals get cold, they shrink so when the brass monkey was cold enough, the cannon balls would fall out because threy wouldn't fit
At least, I think that's where it came from