System Trading is Quicksand without the Quick

Perhaps the best way to describe the path to system trading is slogging. It’s like one of those dreams where you’re trying to run in the sand, and it keeps sinking. And then it starts to rain and the wind starts blowing you backwards. Like many things related to trading, system trading sounds simple on its face, but becomes increasingly elusive as one pursues it.

System trading distinguishes itself from other forms of trading in that it has a high threshold for capital allocations. Feelings, notions, guru tips and gut instinct are not sufficient for the system trader. There needs to be a formalized approach to the trade that can be back-tested on historical data. This first step is not all that involved, really. There are many off-the-shelf programs that can perform a back-test of a trading system. But once you get the results, what do you do with it?

You ponder, perhaps, and cast a skeptical eye upon promising results. You start wondering (and rightfully so) whether past performance has any correlation with future returns. Well, how would you know? At this point you can either sell the program to others or try it out yourself.

A way to test the back-test is to perform an optimization. This is running the back-test with different parameters (ie, moving averages with varying length for an MA crossover system). If it performs well under this process, you can gain some more confidence in your system. But how would one define ‘performs well’? There are many metrics to look at including net profit, drawdown and ratios of those two. How many permutations were profitable as a percentage of all the permutations that were run? What percentage of permutations had a net profit that exceeded maximum drawdown? Is that a good number? What is the best parameter set? What was the measuring stick used to determine the winning set? Now we see where the quagmire begins.

You are now the proud owner of stacks of statistics that you need to sort out. It’s like going through your coupon drawer. You know, the one that has mostly expired coupons, and some for junk you wouldn’t buy if it were free. The optimization results are not a self-evident validation or rejection of a trade system. It takes a thinking person to evaluate what they’ve got.

And after this is sorted out, there is still another step of testing the system on out-of-sample data. Why? Well, because your optimization process found good parameters, but it fit the data to do it. The process is called curve-fitting in statistics and involves making things fit. Not necessarily bad, but it can be overdone. That’s why we walk-forward, or test on out-of-sample data. It’s the same thing as trading the system in real-time. After this process is complete, you find yourself with another truckload of disorganized coupons.

Besides the programming skills needed to do the basics, you need a decent grasp of statistics, good organization skills (where did I put that file on the Coffee market between 1998 and 1990?), and the basic understanding of how markets work. Understanding markets is crucial, as many nerdy statisticians with excellent programming skills have proven by losing large sums of money. Don’t be afraid of having to learn a lot of stuff, though. Trading is not for the lazy.

Leave a Reply

Your email address will not be published. Required fields are marked *