I was uncertain if the “weather” in this phrase referred to storms or to the wise old goat that sheep cluster around.

It refers to storms – the origin is nautical, but the definition seems to be changing.

A wiktionary discussion says the phrase is “unanimously” used to mean to watch something intently since it may cause trouble. But the commenter claims the original nautical meaning is to keep half an eye on something while engaged in other tasks.

A number of definition sources are cited that are ambiguous – simply saying to be on guard or alert.

The commenter tries to prove his case by logic – hardly conclusive when it comes to phrases!

dictionary.com includes one dated citation, though it may not be the earliest:

“Only I feel it my duty to say this to you—keep your weather-eye open.”

Blackwood’s Edinburgh Magazine, Volume 67, No. 411, January 1850

At least I know it’s not a goat.