By the way: Aronson says with I said: just test and compare enough stocks and different periodes with your system and compare it with the standard to beat: buy and hold or "riding the waves"
Dare to make a comparison as I did in the spreadsheet: testing with 50 stocks for 25 or more periods should provide consistently better results or it;s worthless. I am curious to the Monte Carlo routine but don't know where tot look for it.
This review I found covering must remarks in this discussion thread (even the title of this tread "Alchemy" and the relation with gold at the end of this review fits)
This review is from: Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals (Hardcover)
My experience with "technical" trading books has been that they are
either pseudo science of the quasi-astrological variety (eg, wave
theory or fibonacci hocus-pocus) or entertaining and vague
reminiscences of some successful trader(s). The "serious" literature,
instead, is written in statistical/mathematical formalisms which you
may be able to understand given an adequate background, but even then
serious effort and time is required and the sheer distance from the
formalisms to the pits is fraught.
Professor Aronson has done an admirable and unique job of dispensing
with the hocus-pocus and applying the requisite rigor to the many
difficulties associated with analyzing the results of historical
backtesting and datamining. His bibliography seemingly covers all
serious (and some not so) references of interest to the algorithmic
trader. Months after having consumed the book, I still refer to the
bibliography to glean interesting sources.
As an earlier reviewer noted, the early portion of the book in which
he debunks TA and establishes basic statistical literacy for later
chapters may or may not be of interest to many readers, but his
chapters five and six are genuine contributions to the field and
should be read, studied and implemented by anyone looking to derive
tradable strategies based on historical testing.
Anyone who has spent time attempting to optimize parameters or
otherwise evince trading rules from historical data will have quickly
learned that if you look at enough cases, you will always find
something that looks good but (alas!) really isn't. Fool's gold,
indeed! This is the first trading book which directly addresses this
issue in a rigorous yet accessible fashion.
This is not the book for someone who is looking for the magical key to
a kingdom of riches (I recommend Voltaire's "Candide"). But if you
are an algorithmic trader looking to get a statistical grounding and
some concrete methods for assessing your backtested results, there's
simply no better book available.