Straight dope.com explains that ancient Saxons called the clay used to make household items, including jars used for storing money, “pygg,” which was pronounced “pig.” By the 18th Century it became common for the “pygg jars” to be cast in the form of a pig. The bank part of the term is not as simple as it seems. Bank originally meant “bench,” and money lenders worked at benches.