What is Financial Data Mining?
Financial Data Mining (FDM) consists of analytics for trading and market surveillance, which are the applications of data mining and the related technology including machine learning on financial data, to mine for interesting trading patterns and exceptions in financial markets.
Aims of Financial Data Mining
FDM aims to
- invent fundamental methodologies, tools and systems for trading analytics and market surveillance analytics
- develop benchmarks for trading and surveillance oriented analytics
- model behavior, behavior network, group-based behavior, and behavior impact
- design algorithms for various pattern and exception detection and mining
- analyze market impact, risk, cost, benefit, reputation and trust
- mine for patterns and exceptions in either daily and intraday data
- conduct real-life experiments and case studies
Why Focus on Analytical and Learning Techniques?
With the dramatic increase of frequency, quantity, distribution, and rate of market transactions and participants, as well as the evolution of market behavior, it is very challenging to undertake immediate capture and intervention on abnormal behavior. On the other hand, the identification of normality is equally difficult. Advanced Analytical and Learning Techniques such as Knowledge Discovery and Machine Learning, integrated with domain knowledge and human supervision, can demonstrate irreplaceable roles in handling such challenges.
In general, Analytical and Learning Techniques can play significant roles including but not limited to:
- Identifying benchmarks
- Detecting abnormal behavior
- Evidence extraction
- Linkage analysis
- Case analysis
- Risk analysis
- Scenario analysis
What Is Market Surveillance Analytics?
Analytical Market Surveillance refers to the involvement of data mining and machine learning into the detection of and the infrastructure for deep understanding of abnormal market dynamics, trading behaviors, and market impact. Analytical market surveillance, compared to business rule-based method, has great potential to deeply understand the cause and effect of, and maintain and enhance the market integrity by instantly monitoring the market movements as well as detecting any abnormal behavior and activities in a market.
Why Is Market Surveillance Analytics Important?
Market integrity is essential to the success of a market operation. It ensures that the investors can trade in a fair, orderly and transparent market environment. It can assist in higher levels of investment because of the improvement of confidence of investors in the market. To this end, market surveillance plays an essential role. All the market designers, regulators and investors can benefit from high quality market surveillance from a long-term perspective.
Typical market surveillance systems rely on market surveillance rules, which are based on business logic and explicit market movement detection. In fact, many large stock exchanges are challenged by sophisticated market manipulators in terms of evolving manipulative behaviors, increasing transactions and traders, and complicated interactions across many factors and areas. The usual top-down detection runs at the risk of instant capturing of complex and evolving behaviors. The complementary bottom-up analysis, plus the involvement of organizational factors and social media analysis etc. is very promising in tackling increasingly challenging market exceptions.
What is Trading Analytics?
While it is arguable about pattern-based trading, many experienced market analysts rely on chart analysis. With broad-based aims and possibilities, analytical techniques can assist in capturing trading behavior patterns, market trends, correlations, pairwise relationships, causal relationships, etc. through deep analysis of large scale of data, and the involvement of organizational factors and business analysts. Trading strategies can then be designed on top of the identified relationships. Trading Analytics seeks to invent methodologies, algorithms, models, tools and systems for identifying trading patterns, charts and trends in the market.
The Opportunities with Trading Agents
Trading agents, if powered with patterns identified in markets, have great potential in automated trading. The opportunities lie on many emerging tools and facilities, for instance, collaborations, negotiations, risk management, behavior analysis, learning and simulation of other agent behaviors, and adaption to market dynamics.
