My professional background is in systems development and software engineering. Over the years, I’ve gained experience writing computer simulation models, developing large-scale systems, and analyzing large data sets.
When I first started trading several years ago, this engineering background gave me transferable skills to backtest and validate trading strategies. For example, it was a shallow learning curve to program and backtest strategies using tools like NinjaTrader and Amibroker.
However, it has also taken years to learn about and find a balance with important higher-level concerns, such as market dynamics, supply and demand, and trading psychology. Dedication to ongoing research has been needed to discover new trading ideas or refine existing strategies.
In general and for purposes of discussion, I also very much consider a trader to be an integral part of an overall system. Even if a trader adopts an existing system, there will at least be development of a custom workflow and process around the system. In this regard, there is always some further development or adaptation of a system to suit an individual trader. Moreover, an otherwise profitable system can fail if a trader doesn’t have the right skill, discipline or psychology to effectively trade a system; in other words, a trader can be the weakest link in the system.
Don’t Overly Focus on Algorithm Design and Technical Analysis
Coming from my own experience and background, there was a tendency to initially focus too much on the lower-level engineering of a system, but not have a more holistic, top-down approach. There are several pitfalls to overly-technical trading systems development:
- Hammer looking for nails (aka “Law of the instrument”): Especially for engineers and programmers, there is a strong tendency to view trading primarily as an engineering, programming or math problem. This was definitely the case when I first started trading, and I see it all the time when programmers, engineers or data scientists focus almost exclusively on backtesting, machine learning, artificial intelligence, trading bots and the like.
- Overuse or Misuse of Computerized Backtesting: Backtesting is a very powerful tool, but has its limitations. For example, backtesting tools orient around information which can be easily represented in a computer algorithm, such as moving averages and other technical indicators. Without advanced programming, it is challenging to incorporate fundamental analysis or other “non-backtest-friendly” information. In effect, backtesting only readily supports a subset of possible strategies. Moreover, to the extent automated strategies are built upon the same type of tools as backtesting, and many traders use these same tools, trading back-tested strategies in live markets can result in over-crowded trades.
- Over-reliance on Financial Charts and Technical indicators: As a retail trader, what’s the first thing you see when you log into a trading website? There’s typically some type of news feed, summary information on your open positions, and some basic charts with technical indicators. I’m thankful my broker provides me with these tools, but I believe it introduces an unintended bias. A broker is basically constrained to support “least common denominator” type tools which can be used by all their clients. I believe retail traders will tend to use the tools which are readily available, whether or not they are truly effective or a complete set of tools needed to effectively trade a specialized system. Full-blown, specialized trading systems encompass more advanced, higher-level concepts such as confluence, trading catalysts, multiple timeframes, or relative strength; these higher-level concepts can play a far more important role than chart analysis and technical indicators.
Over time, I’ve evolved to approach trading from a more creative and observational perspective. While I appreciate the technical and analytical aspects of trading, I now view trading as both an art and science. With respect to the limitations of backtesting, algorithmic trading and technical analysis, a technical paper written by Michael Harris also reinforced my view of trading as both a creative and scientific endeavor.
Put Enough Focus on Fundamentals, Market Dynamics, and Other Trading Disciplines
As an engineer, I was initially comfortable focusing on backtesting, technical indicators and analysis of trading results. However, it has taken perseverance, time, and experience to balance out my trading with other important knowledge and skills:
- Continuous Learning and Research: Have you ever walked into a professor’s office and noticed a stack of journal articles or bookshelves filled with books? This is because professors cast a wide net to continuously learn and generate new ideas for their research. I’ve come to appreciate this same type of ongoing research and learning is definitely applicable to traders. For example, traders can support their research and learning by reading books, listening to podcasts, watching instructional YouTube videos, participating in forums, or talking and collaborating with others. Some of the best ideas for my own style of trading have actually come from traders who trade different assets or timeframes.
- Learn about Market Fundamentals and Dynamics: Trader’s don’t necessarily need an economics degree, but it is helpful to have a basic theoretical understanding of market dynamics, supply and demand, catalysts which can trigger price movements, etc. My own background includes some graduate research in system dynamics and systems thinking, which pertains to feedback loops, cause-and-effect, and nonlinear systems behaviors. Over the past several years, it’s been very interesting to learn how system dynamics play out in the financial markets. For example, one dynamic in the markets is reflexivity, which means demand and resulting price movements will sometimes form a self-reinforcing feedback loop.
- Fundamental Analysis: Even if a strategy is primarily technical, fundamental analysis can be used to further confirm a trade setup or to help choose between two otherwise similar trades.
- Trade Journaling: Especially for more discretionary trading, trade journaling is very important. This type of journaling can capture detailed information about individual entries and exits, such as charts at the point of entry, reasons for entry, or earnings dates. Over time, some new patterns might emerge from these journal entries; these patterns can help to further refine a strategy, or serve as the basis for new trading ideas.
- General Journaling: Separate from a journal of individual trades, I keep a journal with general observations about the market, individual stocks, patterns I’m seeing, trade ideas, etc. To date, I have almost 2000 journal entries. This is a very unstructured, “anything goes” type of activity, but has proven to hone my observation skills. Similar to journaling individual trades, some patterns can emerge from journal entries. Or, an idea captured in this journal may prove to be worthy of further research.
- Manual Backtesting: Computerized backtesting is very powerful, but has its limitations. By manual backtesting, I’m referring to stepping through historical price and other trade related information, and visually inspecting this information for trade signals. The results are then recorded in a spreadsheet with one row per trade. Manual backtesting is a laborious process, but can be useful to refine existing strategies or to backtest strategies which are difficult to represent in a computer algorithm. This type of backtesting can also be performed by traders who aren’t programmers.
- Screen Time and Chart Analysis: Along with trade journaling and general journaling, I’d categorize screen time as an important tool to hone one’s observation skills. By screen time, I’m referring to time watching live markets to observe patterns, possible trade signals, etc. A major component of screen time is chart analysis; i.e. visually inspecting the price action and technical indicators to analyze supply and demand.
A common denominator for many of the disciplines described above is the lack of automation, and the need for an ongoing time commitment to observe the markets, gain new knowledge, and adapt to new market conditions. It requires dedication and perseverance to keep up with these activities.
Some of these activities may seem tedious or boring, much more so than the thrill and excitement of actively trading a live market, or the intellectual stimulation from developing new trading algorithms. Nonetheless, it is important not to shy away from these important disciplines.
Finding a Balance
Based upon my own experiences described above, a general lesson I’ve learned is too little or too much focus on any one element of trading can lead to failure. For example, a trader coming from an engineering background may focus too much on technical trading, while ignoring important market fundamentals or psychology. Similarly, a trader coming from a financial background may not have the skills to backtest and validate their trading strategies.
Human Factors and Trading Systems
A trader must have just the right mix of overlapping skills, disciplines, psychology and interests to successfully develop and/or trade a given system. The following venn diagram visualizes this important point:
To some extent, each element shown in the center of the venn diagram really does require an overlap and consideration of the three surrounding concerns. For example, trading algorithm development certainly requires programming skills. However, even if the resulting strategy looks good on paper, it may result in a system which would cause most traders to capitulate. Moreover, to the extent an algorithm is only as good as the person designing it, the algorithm design process requires creative input from someone with a higher-level understanding of the markets.
In my estimation, psychology and interest-level (or lack thereof) are more important than a trader’s technical skill. Without sustained interest, a trader will not have the self-motivation to develop the skills and disciplines needed to trade a system. Without the right psychology, a trader may be drawn to systems which are ill-suited to his or her temperament or skill level.
For example, a new trader may try and copy the strategies of more experienced, professional traders, but fail for lack of patience or discipline. Similarly, many new traders are likely drawn to the thrill and excitement of cryptocurrency trading or day-trading. In reality, this type of trading may not be a good fit for new traders’ experience levels, psychology or risk tolerance.
Matching Individuals with Trading Systems
There are several ways to match an individual trader with a trading system, notably:
- Adapt the trader to a system: If an existing system requires certain skills and disciplines a trader is initially lacking, a trader can potentially obtain the knowledge and training to successfully trade a given system. For example, aspiring day traders will often join trading rooms as a way to shore up the necessary skills.
- Adapt an existing system to the trader: Another approach is to find an existing system which is already a close match to a trader’s skill level, psychology and interests. Often times, an existing system is flexible enough to be modified for an individual trader. For example, the rules for exiting positions can be adapted to an individual trader’s patience level. However, there’s a couple of problems with this approach. First, one could find trading systems in books or on websites, such as those based upon moving average crossovers; these are useful for learning purposes, but without extensive modifications, it is unlikely these “textbook” systems will provide any signficant edge. For a new trader, it’s also hard to know which type of systems are suited to them; there may be some trial and error and system hopping to find a suitable system.
- Design new systems: A final approach is to design new systems, taking into consideration a trader’s unique interests, skills and psychology. This is the approach I’ve ultimately taken. While I’ve adopted some building blocks from other systems, this has proven to be the best approach to build systems around my core trading methodology, specialized interests in the markets, and unique skill set. This is arguably the most difficult approach, because one needs a breadth of skills and knowledge to know which building blocks to include in a system.
Unfortunately, there is no clear-cut path or fixed timeline to develop the necessary trading skills, build up a core methodology and match yourself with suitable trading systems. This challenging and frustrating development process can take years, and include much trial and error. It takes sustained interest in the markets and perseverance to get through this development process. Many traders will simply give up before getting some positive traction.
What’s Worked for Me
Speaking for myself, it took more than five years to fill in the knowledge gaps and develop some trading systems suited to me. Granted I’m a part-time trader, but this begs the question whether or not there are ways to shorten this learning curve. For me, it was a matter of continuous learning, experimentation, trial and error with different systems, and perseverance to keep trading no matter what.
Developing myself as a trader has been frustrating, but there has been a positive side effect to all this trial and error and systems hopping. Over time, I’ve accumulated more and more knowledge and practical skills which can be used for any style of trading. This same accumulation of knowledge gives me a knoweldge base to pull from for new systems development or to refine existing systems.
To formulate strategies, I pick and choose from many sources. I have somewhat of a core methodology which influences the types of specialized strategies I’ll trade, including those based around momentum and volatility. Some of the building blocks for my strategies are from my own observations. However, I’m not opposed to incorporating techniques learned from other traders, no matter what type of assets or time-frames they trade. This results is strategies which are a little bit unorthodox and a unique combination of different building blocks; I believe being a little bit unorthodox helps give me an edge.
For now, I’m perfectly happy being a part-time, end-of-day (EOD), retail trader. I primarily trade U.S. equities (stocks), including individual companies and ETFs. One reason I’m interested in stocks is because there are literally thousands of names to choose from. There’s never a shortage of good setups. This fits well with my temperament and other time commitments outside trading.
If you pinned me down, I’d call myself a rule-based discretionary trader. While I try to be as systematic as possible, I’m open-minded to include some discretion. There is a relatively disciplined process and workflow surrounding my trading strategies. However, my process isn’t fully automated, and includes manual checklists and a somewhat disconnected set of tracking spreadsheets, scripts, screening tools and charting packages.
I believe in the “power of observation” to look for new and unique market patterns, correlations, catalysts, etc. Journaling and screen time play an important role here. Including some discretion also helps me observe new patterns, find new trading ideas, and think of ways to refine my existing strategies.
Quantitative analysis is a part of my trading systems development process, including backtesting (both automated and manual) and algorithm development. My systems incorporate some technical analysis; I’ve even developed a few technical indicators myself, including volatility-based momentum and oscillator indicators. However, I try not to get too carried away. Ultimately, I believe price itself is the ultimate technical indicator and the most direct representation of supply and demand. In this regard, I believe it’s a mistake to use technical indicators or trading algorithms which are too far removed from price itself.
I don’t spend a lot of time with fundamental analysis, but do keep an eye on companies' financial statements for things like earnings growth or potential red flags. With respect to supply and demand, what matters more to me is someone actually purchasing or selling an asset; i.e., “putting their money where their mouth is.''
One area I’ve struggled with is psychology, and in particular impatience. I’ve become more patient using a number of tactics, but this is a work in progress. When I first started trading, I didn’t recognize pscyhology as an important factor. I now see psychology as integral to the success or failure of an overall trading system.
Over the years, I’ve maintained a strong interest in the markets. This has given me the self-motivation to continuously learn and discover new ideas. This has also given me some perseverance, even when the trading itself wasn’t gaining much traction. I believe my skills and discipline suffice for my current style of trading, but this is also a work in progress.
Think of Trading Systems as Recipes
To tie these concepts together, one could think of trading systems as recipes. Recipes have ingredients, quantities or portion sizes for each ingredient, and a step-by-step procedure to prepare the recipe. To take this analogy one step further, traders are not unlike chefs. Almost anybody can learn how to follow a recipe, but a master chef knows how to balance the ingredients of a recipe, and to mix and match the right ingredients to make new recipes.
Likewise, trading systems have any number elements or ingredients. The list of elements and their relative importance will vary from one trading system to the next. Compared with food recipes, the margin for error with trading systems is slim. If a trading system is missing a single element, or if the individual elements are out of balance, the whole system can fall apart.
A new and inexperienced trader may be able to follow a trading plan and strategy, but lack the knowledge and skills to develop new systems or refine existing ones. However, experienced traders can draw from their cumulative experience and knowledge to build and refine trading systems.
Chefs also specialize around different types of food, such as Italian cuisine or pastries. Similarly, based upon their skills and interests, traders will inevitably specialize around certain types of assets, timeframes and mechanical versus discretionary trading systems.
Summary
The development of trading systems is a very challenging endeavor, one that is fraught with pitfalls and complexities. An overall trading system needs a multitude of elements, including but not limited to a trading plan, rules for risk management, a backtested strategy, and checklists for executing the strategy. While it is imperative that a strategy have an edge and positive expectancy, the overall system can fail if anything is missing or out of balance.
Adding to the complexity, a given trading system may or may not be suited to an individual trader. To successfully trade a given system, it takes just the right mix of human factors, including psychology, technical skills, discipline and sustained interest. Some adaptation of a system may also be needed to suit the trader, or vice-versa. The development of new trading systems must also take into consideration these human factors. In this regard, an individual trader is very much an integral part of the overall system.
Every trading system is unique, but fortunately there is a relatively common set of tools, building blocks and techniques needed to develop and trade these systems. Even though it can take years for a trader to get some positive traction, an accumulation of knowledge takes place no matter which system is being traded. This cumulative knowledge is helpful to formulate new systems and strategies, or to adapt existing ones. Moreover, even if a trader is not initially profitable, trading experience can help to improve psychology and discipline. Ultimately, a trader can find a niche with a particular system, but there needs to be a strategy with a quantifiable edge, a complete and balanced overall trading system, and a trader who has the right skills, interests and psychology.
About the Author: Steve Roehling is a professional systems developer and software engineer. Trading systems development is one of his research and development interests, encompassing both the technical aspects of systems development and human factors such as psychology. To learn more about or to contact Steve, please visit his website.