Why gold just can't win

If no one expected the Swiss vote forcing the central bank to hold more gold to succeed, why is gold tanking?

Over the weekend, Switzerland's voters overwhelmingly rejected a proposal to require the country's central bank, Swiss National Bank (SNB), to raise its gold holdings to 20 percent of its assets from around 7 percent currently as well as preventing it from selling any more of its gold holdings.

Read More Swiss voters resoundingly reject gold, immigration proposals

But while the outcome was well expected, gold (CEC:Commodities Exchange Centre: @GC15J) still swooned Monday, dropping as low as around $1,141 in Asian trade Monday for an around 3 percent decline, a far cry from the more than $1,900 an ounce peak set in September 2011.

"Even though the Swiss announcement was well known and the Swiss government isn't a huge buyer, the market is drawing comparisons with other central banks going down the same road of not increasing [gold holdings] as much as thought," said David Lennox, a resources analyst at Fat Prophets, noting that central banks have been buying quite a bit lately.

Bear market

Gold's decline Monday may not represent any developments.

"Gold's been in a bear market since the high in September 2011," Barry Dawes, head of resources at Paradigm Securities, told CNBC.

Read More Swiss set for gold vote amid '6,000-year bubble' warning

Indeed, around $4.41 billion has flowed out of gold mutual funds and exchange traded funds so far this year, with another $3.78 billion coming out of precious metal funds, according to data from Jefferies.

Oil price drag

Another factor that may be pressuring gold: oil prices have dropped sharply over the past week.

"Oil is taking the entire commodity spectrum down with it, as energy is the major cost of production in most commodities (extraction and transportation)," Julius Baer (Swiss Exchange: BAER-CH) said in a note Monday.

Read More Central banks: The new gold bugs?

"Low oil also means low inflation," the bank added. That's something that short-circuits demand for gold as an inflation hedge.

No fillip on India's move

The shiny metal even failed to get a fillip news Friday that India's central bank took the surprise move of ending rules that required gold importers to re-export 20 percent of their shipments.

It isn't clear how much of a buying boost the change will bring.

"[The restriction] just drove buying underground," Lennox said, although he expects buying will become more visible. "Those persons who didn't want to participate through underground buying and didn't want to pay the tax should enter the market."

Price forecasts

Gold's decline (CEC:Commodities Exchange Centre: @GC.2) has prompted analysts to once again trot out the oft-repeated predictions that the shiny metal could fall all the way to $1,000 an ounce.

Read More Chinese unmoved by gold price drop, see it cheaper still

"I wouldn't suggest $1,000, but it may not be out of the question," Lennox said, citing rampant U.S. dollar strength as weighing on the metal.

"We need to see something in other currencies to counter the U.S. dollar (Exchange: .DXY) and we can't see that coming out of Japan and Europe next year and of course, China pegs [it's currency], which doesn't leave a lot of competitive currencies against the U.S. dollar," Lennox said.

Read More Charts suggest sub-$1,000 gold is likely

To be sure, Paradigm's Dawes thinks physical gold might finally be finding a floor.

"The demand for gold is really out of India and China," Dawes said, noting the importance of European and North American demand has waned. But he expects improved demand from the middle-class in India and China, as standards of living improve there, and as fears of currency debasement persist.

-By CNBC.Com's Leslie Shaffer; Follow her on Twitter @LeslieShaffer1

Clarification: This story has been updated to reflect that the current gold holdings of the Swiss National Bank is about 7 percent of its total assets.