Feed Your (Machine) Brain

Few can tell you what goes into a chicken nugget, but most will agree that it’s good for your brain. If you’re a little sluggish and can’t focus, what do you normally do? That’s right, you pop a couple chicken nuggets. And similar to our brains, our algorithms need some food too, to think properly. It’s not a simple matter for them though. They can’t drive to a fast-food establishment, so they rely on us to make the nuggets for them. With R and a little awk, it’s actually quite easy to do this at home. Also, when you make brain food from scratch, you’re one of those few people who actually know what goes into it.

For our purposes, we are going to prepare some information nuggets for the C/C++ SVM library of algorithms known as libsvm. This is open-source and award-winning, so it’s a good brain for our purposes. It aims to predict an outcome given a specific set of conditions. Before it can start on the business of prediction, it needs to learn from “quote” — examples. It needs to eat some chicken nuggets. And it needs those nuggets presented in a specific way. The libsvm format for data is:

label   index1:value1   index2:value2   ... 

The label is the answer. This of course depends on what you want to predict. Let’s say we are interested in a prediction if the market for a particular stock is going to be up or down from yesterday. In this case, label will be either 1 or -1. This is what it looks like when the answer is an up day.

1   index1:value1   index2:value2   ... 

We could say TRUE or FALSE, but libsvm needs a numerical representation. The next series of lines are what you think the brain needs to get the answer correct. It follows in the format of:

1 1: "value of 50-day SMA" 2: "value of RSI" ...

We need to replace the quoted string with an actual number and that’s where R comes in. I chose a Dow 30 stock as my example. The following R code gets us most of the way there.



MCD$Cl.sma_10       <- Lag(SMA(Cl(MCD), n=10))    #yesterday's value
MCD$Cl.sma_30       <- Lag(SMA(Cl(MCD), n=30))    #yesterday's value
MCD$Vo.sma_10       <- Lag(SMA(Vo(MCD), n=10))    #yesterday's value
MCD$Vo.sma_30       <- Lag(SMA(Vo(MCD), n=30))    #yesterday's value
MCD$Cl.rsi          <- Lag(RSI(Cl(MCD)))          #yesterday's value
MCD$Cl.return.daily <- Lag(Delt(Cl(MCD)))         #yesterday's value
MCD$Cl.return.10    <- Lag(Delt(Cl(MCD), k=10))   #yesterday's value
MCD$Cl.return.30    <- Lag(Delt(Cl(MCD), k=30))   #yesterday's value
MCD$pre_answer      <- Delt(Cl(MCD))              #today's pre_answer

squish <- function(x){

 else(x< 0)
MCD <- na.locf(MCD, na.rm=TRUE)  

MCD$answer          <- cbind(MCD, apply(MCD,1, function(x)squish(x[15])))

write.table(MCD, "~/Desktop/goo", row.names=FALSE, col.names=FALSE)

Try this yourself. You should get a text file on your desktop called goo. Here is what the first row looks like, but a warning first. It’s not pretty. Remember what were making here.

45 45.38 44.86 45.32 
6806600 39.83 44.71 44.2873333333333 4683760 
6206100 60.5405839422285 -0.000888494002665663 
0.0112410071942446 0.0253020287212218 0.00755891507336592 
45 45.38 44.86 45.32 6806600 39.83 44.71 44.2873333333333 
4683760 6206100 60.5405839422285 -0.000888494002665663 
0.0112410071942446 0.0253020287212218 0.00755891507336592 1

I would have preferred doing a little more prep in the R code, but some mysterious going-ons created new columns when I tried indexing out data I wasn’t interested in. I suppose it has to do with not being able to delete a column whose value creates a column you want to keep. Not sure about this one. I’ve turned to a little awk wizardry to get the values I truly want, and to get the format just so. Here, we convert the goo file to a paste file. This is all on one line from command line in the directory where your goo file is located.

$  awk  '{print $31 " 1:" $7 " 2:" $8 
" 3:" $9 " 4:" $10 " 5:" $11 " 6:" $12  " 7:" $13  " 8:" $14}' goo > paste

This is the first line of the paste file. Still not appropriate for visually sensitive people, so be careful.

1 1:44.71 2:44.2873333333333 
3:4683760 4:6206100 5:60.5405839422285 6:-0.000888494002665663 
7:0.0112410071942446 8:0.0253020287212218

I didn’t mention python earlier because I didn’t want to make this sound too complicated right off the bat. But there is actually a little python script that checks to see if the format is satisfactory. The program comes with libsvm.

$ python checkdata.py paste
No error.

There is a little work to do with scaling the data, so be careful to feed this raw paste to your beloved algorithm. You still need to change the bubblegum color and pasteurize it. The README that comes with libsvm explains it well.

The elegance of this approach to feeding your brain is that not only do you control the ingredients, but you can experiment with those ingredients to find the best chicken nugget recipe of all time. Ever.


A suggestion by a reader below replaces lines 13-25 of the example above with the following:

MCD$answer <- sign(Delt(Cl(MCD)))
MCD <- na.locf(MCD, na.rm=TRUE)
MCD <- cbind(MCD$answer, MCD[,-ncol(MCD)])

I timed my verbose version that uses an explicit loop in R and got the following speed data:

  proc.time() – ptm  user  system elapsed  0.394 0.004  0.415 

Then running the more compact version suggeted by @jro below:

    proc.time() – ptm
  user  system elapsed
  0.181 0.002  0.187

Much faster, compact and the proper way to write code in R.

Volatility Violins

Unlike many humans, markets love change. In fact, they look forward to it with great anticipation. Regular people like stability, for the most part. Unless you’re a career gypsy, you like to stay in one place for some time. Making a home. Settling in, as it were. Unlike markets, where volatility is the raison d’etre.

I’ve plotted the values for VIX, which is the CBOE’s volatility index calculated by some magic formula that considers 30-day expectation of price movement, in the chart below. Next to it is a plot of 30-day returns for the SPX, or a graph that plots all the percentage changes for each 30-day window. Quite frankly, I’m not sure what conclusions to draw from this, except that expectations in markets usually fall short.


The R code:


VIX <- getSymbols(“^VIX”, auto.assign=FALSE)[,4]
SPX <- getSymbols(“^GSPC”, auto.assign=FALSE)[,4]

VIX <- VIX/100
SPX <- Delt(SPX, k=30)
SPX <- na.locf(SPX, na.rm=TRUE)
SPX <- abs(SPX)

vioplot(VIX, SPX, names=c(“VIX”, “SPX”), col=”red3″)

title(“Violins of Volatility”)

editorial note: the graph annualizing 30-day returns has been removed.

Silver Is A Weighted Coin

editorial note: there is an error in the code explained below the code. 

When you flip a quarter, you normally assume the coin is fair and that there is a 50% chance of getting either heads or tails. Option pricing assumes the world of trading is filled with fair coins. In other words, there is basically an equal chance that the option’s underlying will close 10% higher in 24 days or 10% lower in that time frame. This assumption has always puzzled me. So I constructed a simple algorithm in R to check what the history of coin-flipping in the silver ETF SLV shows.

If you slice up an options probability graph with a +/- 10% threshold, you’ll get the implied probability of such an event happening within a prescribed time frame. Today, SLV options have priced in a 16.5% chance that in 24 days, SLV will be 10% higher. And, unsurprisingly, options have priced in a 16.5% chance that in 24 days that SLV will be 10% lower. Higher, lower, what’s the difference? We’re playing with fair coins here.

See more: Feed Your (Machine) Brain(Opens in a new browser tab)

If you look back 1,040 trading days, you get a different distribution. How many times has the event happened that SLV closed 10% higher than it did 24 days ago? That would be 81 days. How many times did it close 10% lower than it did 24 days ago? Uh, that’s only 39 days. That doesn’t seem fair to me.

The R code, in 36 double-spaced lines:

lemon_lime <- function(sym="SLV", expiry=24, pct=.1){
ticker <- getSymbols(sym, auto.assign=FALSE)                             
ticker <- ticker[,6]       

win  <- 0   
lose <- 0                                      
for(i in expiry:NROW(ticker))                                                                                
if(as.numeric(ticker[i]) > (as.numeric(ticker[i-(expiry-1)])*pct + as.numeric(ticker[i-(expiry-1)])) )                                            
win <- win + 1                                                              
winners <- win/(NROW(ticker)-expiry)                                                                                                      
for(i in expiry:NROW(ticker))                                                                                
if(as.numeric(ticker[i]) < (as.numeric(ticker[i-(expiry-1)]) - as.numeric(ticker[i-(expiry-1)])*pct) )                                            
lose <- lose + 1                                                              
losers <- lose/(NROW(ticker)-expiry)  

weighted <- win/(win+lose)                
cat(   " Trading Days:               ", NROW(ticker)-expiry, "\n",
       "Winners:                    ", win, "\n",
       "Percentage of winning days: ", round(winners*100, digits=2), "\n",
       "Losers:                     ", lose, "\n",
       "Percentage of losing days:  ", round(losers*100, digits=2), "\n",
       "Weighted Coin Towards Winners? ", round(weighted, digits=2), "\n")                                   

Ah, the perils of quant analysis. I have changed one digit in the code above to account for the obnoxious stock split in 2008. My change was to line 7:

ticker <- ticker[,6]  # previously ticker <- ticker[,4]

It didn’t completely change the notion I was exploring so I decided not to change the entire blog text. The new results though show a ratio of 261:97 in favor of the upside excursions of the 10% limit, versus 81:39.

Reversion to the Irrational Standard Deviation

The current market is a fine illustration of how markets don’t revert to a mean, and they don’t behave rationally. Pretty much, we can throw those market theories into the toy chest in the attic, along with other diversionary entrapments from our younger days. The theory that markets trend explains the largest set of current phenomena, more than anything else.

To be fair to mean-reversion traders, they don’t presume market returns to be a stationary time series. More often they trade off pairs that preserve statistical canon while individual underlyings behave weirdly on their own. And rational market theorists always, always have an out. It’s almost impossible to refute a rational market theorist because they know all about rationalizing.

It’s still important to understand underlying macro conditions, and in my view it’s important to have a work-in-progress thesis of what’s going on. But don’t let integrity to your thesis prevent you from acting like a drunken bull. Just make sure you get less drunk then everyone else. That may be the only edge you need.

The Art of !Trading

Taking a break from trading is not a simple matter. Unless you’re simply quitting, in which case it’s a very simple matter. But if you find yourself paying less attention to the market as your life becomes consumed by something else (a medical condition, the birth of a new child, a new job, relocation, golf lessons) then you need to identify it’s time to participate in the art of not trading.

It’s not the same thing as “quote” — putting your trading on the shelf for now. No, that’s pretty much quitting. The art of not trading is something different. It’s more like hibernating. Or like the deep sleep you would experience in inter-galactic space travel. When you wake up, the market will be different, but probably much the same. You just need to remember how to stand up, walk, articulate verbally about how glad you are to be back and curse in a manner consistent with your personality.

The art has to do with being able to cycle between the state of trading, and the state of not trading. It’s an art so you are left to your own imagination as to how it plays out.

Tip-Toe Through a Minefield of Tulips

If the Fed does quantitative easing as we all expect, then it may be time to start buying flowers. Not as an investment, of course, but with full intention of selling at higher prices what we buy today on the cheap.  But let’s not all rush into this trade at once. Because the Fed has not actually QE’d anything lately, though they are signaling via wink and nod. Perhaps they may temper their plans as the list of potential unintended consequences mount.

Corn has limited up two days in a row. Wheat and Beans are rallying in sympathy. Precious metals are persistently trending upwards. What could be wrong with this you ask? Hmmm. Well, if you’re a Midwest farmer with a pile of gold coins buried in your corn field, nothing could be finer. But if you’re retired and chasing lower yields on risk-free assets so you can buy corn flakes for breakfast, well then it may be a different story.

In equities, it’s becoming a little heady as well with the new vogue perception that you just can’t lose with the Fed at your back, ready to continue debasing the dollar for your benefit. Either stocks go up and you win, or stocks go down and then back up, and you win. I wonder what could go wrong here. I don’t know, but if I were a swan I’d get my black Sharpie out right about now.

Be careful not to trip over dollars to pick up flowers as you head to your local florist. Someone is laying tripwire out there, and it may not end well.

Learning From Losers

Traders will typically approach a large loss in one of two ways. First is the dumb way, and that is to become a petulant whiner and throw a fit. Next is the more-constructive way, and that is to use the loss as a means of developing as a trader and to “quote” — learn from your mistakes. But there is a third way. And that is to view the loss as the cost of information.

I don’t mean the cost of doing business per se. This is not typically associated with large losses. Small losses, yes. Because to make money you have to lose some along the way, as casinos do every day.  And not the cost of tuition where the market charges a fee to school us. No, I mean information.

Instead of asking yourself about where you placed your stops and getting all personal about the whole thing, ask yourself what happened. Why did the market move the way it did? If you haven’t suffered a capital depletion, you are not likely to demand an answer and more likely to throw off the question with a wave of the hand and a shrug. “Who knows, who cares. I only play odds.”

Markets are a beast and if you want to play with them, you’ll have to be careful. Wear protective goggles and gloves. If you want to tame them though, you’ll need to wrestle with them. And sometimes you lose some body parts along the way. 

The Compleat Trader

Traders are always fishing for new ideas and that can lead to trouble, but it can also lead to the discovery of a new lifetime pursuit, the pursuit of comprehensive knowledge. There is a general fear and loathing in trading when it comes to “quote”  — thinking too much. We are admonished to keeps things simple and to not muddle our minds with too much, lest we lose our way.

Instead of saying “I don’t trade options because they’re too risky and I don’t understand them” why not say: “let me explain how an Iron Condor is basically a long butterfly with an embedded short box.” Over a nice cup of tea. Yes, that would be better. And let’s not run from technology and the underlying mechanics of how it works.

Yes, I’m talking about programming, every traders love-to-hate topic. There is no glory in being a Luddite during this extraordinary time of technological advancement.  The aw-shucks I’m just a stupid trader who makes millions of dollars playing dumb is so passe. That shtick has lost its luster, shall we say.

Don’t cringe when someone says “contango” but teach the definition and the information it contains to your children. Embrace statistics. Learn how bonds are valued. Investigate how central banks influence their currencies and why. Explore how behavioral finance has turned Efficient Market Hypothesis on its head.

It doesn’t hurt to be comprehensive in your knowledge and targeted in your approach. Who knows, you may get hooked on the idea if you try it. 

A Complicated Trade

If you like to keep things simple, then you’ll despise the following trade. But if you’re not scared about a little complexity and are interested in some ideas, then read on. The basic premise that I’d like to express in a trade is that US Treasuries are overbought and due for a pullback. You could sell the ZB future or short TLT, but these scenarios expose you to unlimited risk (more or less). You can place a “quote” – stop – on your trade and thereby console yourself that you have defined risk, when really you’ve just described how much you are going to lose.

No, we are going to go all-in on this trade and sleep (comfortably)  in the bed we make. The risk (amount we are willing to lose if we’re wrong and US Treasuries skyrocket to the moon) is the amount we are going to pay for an in-the-money (ITM) call. We need to choose an underlying that goes up when we are correct, so we need an inverse ETF. For our purposes, we will use TBT. Buying the SEP 25 call for $5.20. We are exposed to a little time-decay, but not stupid like if we were to buy out-the-money (OTM) calls. Three weeks until a decision is made for us, if we chose not to take it off before then.

Note: though I like to blog entries, I’m not likely to blog the exit so please follow me on twitter if you are interested in my exit.

Options For Sale

I sold the Conoco Phillips (COP) August 50 put for $1.32. It is an expansion of a strategy I’ve been deploying all year with TLT. I call it Put Me First, Call Me Last for short. The basic idea: to start this game, you sell a put below the market. Then when the price sells off, you get assigned and become proud owner of 100 shares per put sold. Then you sell a call above the market. After the market rallies, you get your stock taken. It’s a premium collection strategy at its heart. Because you’re the seller, time decay is on your side. This means that if nothing happens, you’re good. Ho hum, big deal, I’m happy.

The key to this strategy is to pick a stock that you don’t mind owning. I started with TLT back when it was selling in the 90s. It pays a dividend and it tends to go up in most “end of the world” scenarios. My method for stocks is a simple 2-step process. Step one involves sorting based on the following criteria:

  1. $1B market cap — not likely to evaporate overnight
  2. 1 million daily shares traded — liquidity
  3. Yield of 4% or more — pays dividend
  4. Positive EPS –makes money
  5. Price between $15 and $100 — various random reasons

Step two is to find good liquidity in the options chain, pick one that is in a relatively strong industry group and make sure its technicals and/or fundamentals are not screaming to stay away.