The importance of qualitative tools in Finance
We can define qualitative tools as tools which process qualitative data and produce qualitative information. While quantitative data is easily identifiable in prices of stocks, historical time-series of share, bonds, inflation, interest rate and all sort of relevant numerical data Qualitative data is more difficult to define. Quantitative data can be easily used in mathematical or statistical equations, which does not normally apply to qualitative data.
Qualitative data, instead, is data that is difficult, if not impossible, to express in a numeric format. For example, data regarding rumors, fears, broker's recommendations, takeovers etc. are all qualitative data. A sentence such as: there are rumors of a possible takeover of Apple, the troubled computer manufacturer represents an information which is extremely relevant to the financial operators, since it will probably immediately cause a marked movement in the quotation of Apple's shares as well as those of the possible buyers. However, taking it into account in a mathematical/statistical equation would be extremely difficult, if not impossible. These kinds of news are extremely important because they affect the expectations of the operators regarding a particular share. The way in which the operators are influenced depends on how an operator perceives the information. Even if, in theory, it could be possible to produce a complete econometric model which takes into account all possible variables and expected behaviors of the players, the complexity of the financial world makes it impossible to produce such a model which would anyway be extremely expensive in terms of the computing-power necessary.
A similar situation can be found in macro-economics. The advanced econometric macroeconomics models employed by the central banks often fail to predict the development of the economical cycles, crisis and expansions. Only very few macroeconomic relations (e.g. interest-rate/investments) are actually widely used and effects of a change in one of the variable is easily predictable.
Qualitative data is much more difficult to process than quantitative. Therefore, while all sorts of quantitative financial tools are nowadays available on the market, very little progress has been done in the processing of qualitative information, which is usually left to the financial operators.
In the financial community, news, rumors and facts are among the most important factors that determine the operators behavior. Operators, in fact, are much more influenced by news than by analysts forecasts or historical price analysis of shares. When a news such as "the inflation rate is expected to increase next month" arrives, the consequences are immediately visible and operators base their decisions on their personal experience and on other's people behavior, rather than on expensive and complex forecasts produced by complicated neural-networks financial forecasting systems. Quantitative methods work fine and help the operators' decision-making process by suggesting a "normal" path of the prices but, at the end, what is important is the news and people's reaction to it.
The financial operators and information providers understood the importance of qualitative data as the key-point in the trading decision-making process long time ago. Therefore, the emphasis has been on providing as much relevant qualitative information as possible. Financial operators receive in real-time news regarding companies (e.g. announcements, rumors, profit forecasts etc.), macroeconomics (e.g. movements of inflation rate, unemployment etc.), politics (e.g. changes in general macro economic policy of the government, tax policies etc.). They also have access to huge quantities of past information.
The ideal situation would be a system which is able to process the qualitative information, take into account any possible factor and produce a response such as "buy/sell". Unfortunately, excluding traditional mathematical and statistical techniques which are likely to be impossible to be used for such analysis, current artificial intelligence techniques are not sophisticated enough to produce such an output and, therefore, the decisions are still mainly taken by the operators
Current development of qualitative tools is therefore concerned with "reducing", summarizing or partitioning the news according to specific criteria, rather than inferring decisions from them. This is equivalent to performing simple analysis on quantitative data (e.g. the Moving-Average index), rather than producing a clear operative decision. Explanatory or forecasting qualitative tools have yet to be developed. Natural Language Processing can be potentially used for such analysis (see next sections).
Natural Language Processing as general support tool
The goal of Natural Language Processing as general support tool is to summarize, reduce or categorize the input qualitative data, rather than produce any analysis directly related to the actions that will be taken by the operators as a result of their decision-making process. Therefore, the emphasis is to provide the financial operator with information which is a summary of the most important data, rather than actually trying to suggest the action to be taken. This is similar, in a way, to the output produced by simple statistical techniques - ie moving average. The trend is captured and identified, but the interpretation of the final information and the decision is left to the financial operator. Similarly, the Natural Language Processing tool can tell the operator that the main underlying event of a particular group of news is, for example, about a probable increase in the inflation rate, but the interpretation of the results is left to the operator.
The main task of general support tools based on Natural Language Processing is therefore to help financial operators to overcome the actual qualitative data overload simplifying and reducing the amount of qualitative information that are needed to support their decision-making process.
Information extraction is the NLP technique used to identify and extract the relevant information according to specific structures (templates) from a source text. Therefore, most of the financial tools based on NLP performs information extraction tasks.
So far, very few examples of tools based on NLP have been successfully introduced in the financial community. Moreover, most of them have been designed to solve very specific tasks in extremely limited domains. They are usually based on techniques nearer to Information Retrieval and word-matching than to NLP. No information-extraction systems are able to process a large amount of qualitative data and produce sensible results for a relatively large financial domain
NLP systems and information extraction are mainly employed and developed for information providers. They are extremely keen on trying to further classify, reduce and summarize huge amounts of financial information that they provide to their customers. For example the Dow-vision news service a Internet news service provided by Dow Jones automatically provides some additional information linked to the articles available in real time. However, such information is quite naive and simple, such as the market sector and the category to which the company belongs. This information is mainly obtained using pattern-matching techniques, rather than NLP techniques or even by hand by the information provider's operators.
ATRANS, Automatic Processing of Money Transfer Messages, has been successfully used in real tasks. However, the system was concerned with a specific kind of messages between banks which had an extremely limited domain not directly related to the trading of securities on the stock exchange market.
Natural Language Processing as explanatory tool
It is our belief that Natural Language Processing can be potentially employed as explanatory and prediction} tools. At present, no explanatory / prediction natural tool based on natural language has been implemented.
In our view, a possible NLP explanatory financial tool can be based on the following points:
a NLP information extraction system can be able to infer knowledge from the source data (articles) which is not directly available in it. So, for example, a text can contain information regarding a takeover even if the word "takeover" or "acquisition" has never been cited. The NLP information extraction system is able to infer the underlying concept and extract the relevant (explanatory) information attached. This analysis can be used by the financial operator for explanatory tasks such as analysis of price behavior related to qualitative information, which has normally to be performed by hand.
adding meta-analysis to the original summary / template produced by the NLP information extraction tool. Securities prices are, in reality, affected by people's perception / view of a particular event, rather than by logical or mathematical equations. Therefore, the financial operator can be interested in the "way" a particular information was cited in the source article. In fact, the "way" in which an event is cited incorporates directly the writer's perception of the event and will normally have some influence on the reader's opinion / view of the facts. The meta-analysis will consist of information added to the summary / template extracted from the source texts. Two ways for the identification of the meta-analysis could be potentially used:
how many times a particular news (topic) has been cited in the source article. If a particular news or information is repeated numerous times in a source article, it is likely to be that the number of times the news has been cited in the source article and its importance are directly related. The NLP system should therefore identify the number and relevance of each time a particular news has been cited and report this information to the user (using a semantic, rather than shallow, comparison).
how the information has been said. In this case the system could identify the "way" in which the news has been said and present the information to the user.
Natural Language Processing as forecasting tool
Natural Language Processing, finally, can potentially be used as prediction / forecasting tool, suggesting to the financial operator final decisions (buy/sell decisions). The underlying idea is that operators often base their decisions on (real-time) news, rather than on quantitative predictions. In this view, the decision-making process of the financial operators can be described in the following steps:
1. the operator reads the news,
2. the operator gives his/her own interpretation of the new information,
3. the operator compares and analyses the new information with the knowledge he/she already owns,
4. a final decision (buy/sell) is taken.
The aim of Natural Language Processing as prediction tool is to automatize such processes. The system would therefore process the new data, identify the relevant information, process it according to specific domain-knowledge for that situation and present it to the operator, together with the summary / template of the original article, the suggested decision to be taken. A possible alternative would be to make use of existing financial prediction tools, such as Expert Systems and Neural Networks. The NLP System would process and identify the relevant information which would then be supplied to the expert system / neural network in the appropriate specific form.