Unformatted text preview: Preface We describe a realtime, internetbased S&P futures trading system, including a description of general aspects of internetmediated interactions with
electronic exchanges. Innershell stochastic nonlinear dynamic models are developed, and Canonical Momenta Indicators (CMI) are derived from a ﬁtted
Lagrangian used by outershell trading models dependent on these indicators.
Recursive and adaptive optimization using Adaptive Simulated Annealing
(ASA) is used for ﬁtting parameters shared across these shells of dynamic
and trading models.
Chicago, August 2001 Ingber, Lester and Mondescu, Radu Paul Table of Contents 1. Automated Internet Trading based on
Optimized Physics Models of Markets
Lester Ingber and Radu Paul Mondescu . . . . . . . . . . . . . . . . . . . . . . . .
1.1 1.2 1.3 1.4 1.5 1.6 1.7 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.1 Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.2 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.3 Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
CONNECTION TO ELECTRONIC EXCHANGES . . . . . . . . .
1.2.1 Internet connectivity: overview . . . . . . . . . . . . . . . . . . . . .
1.2.2 Internet connectivity: hardware requirements . . . . . . . .
1.2.3 Internet connectivity: software requirements . . . . . . . . .
1.2.4 API order execution module: components and functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MODELS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.1 Langevin Equations for Random Walks . . . . . . . . . . . .
1.3.2 Adaptive Optimization of F x Models . . . . . . . . . . . . . .
STATISTICAL MECHANICS OF FINANCIAL MARKETS
(SMFM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.1 Statistical Mechanics of Large Systems . . . . . . . . . . . .
1.4.2 Algebraic Complexity Yields Simple Intuitive Results
1.4.3 Correlations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.4 ASA Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TRADING SYSTEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5.1 Use of CMI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5.2 Data Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5.3 InnerShell Optimization . . . . . . . . . . . . . . . . . . . . . . . . .
1.5.4 Trading Rules (OuterShell) Recursive Optimization .
RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6.1 Alternative Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6.2 Trading System Design . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6.3 Some Explicit Results . . . . . . . . . . . . . . . . . . . . . . . . . . .
CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.7.1 Main Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.7.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1
2
3
4
4
6
6
7
7
9
9
9
10
10
13
14
15
16
16
17
19
20
22
22
23
24
27
27
28 VIII Table of Contents 1.7.3 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.7.4 Standard Disclaimer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 1. Automated Internet Trading based on
Optimized Physics Models of Markets
Lester Ingber1 and Radu Paul Mondescu2
1 2 Lester Ingber Research
POB 06440 Sears Tower, Chicago, IL 60606 and
DRW Investments LLC
311 S Wacker Dr, Ste 900, Chicago, IL 60606
email: [email protected], [email protected]
DRW Investments LLC
311 S Wacker Dr, Ste 900, Chicago, IL 60606
email: [email protected] 1.1 INTRODUCTION
Launching and exploiting a successful automated trading system implies accomplishing two major tasks, of almost equal signiﬁcance:
– designing and developing a robust trading model of markets of interest,
– connecting the system to markets, addressing two problems
– the communications hardware infrastructure,
– the software interface.
To develop a robust and consistent model of markets, we should remark that
realworld problems are rarely solved in closed algebraic form, yet methods
must be devised to deal with this complexity to extract practical informations in ﬁnite time. This is indeed true in the ﬁeld of ﬁnancial engineering,
where time series of various ﬁnancial instruments reﬂect nonequilibrium,
highly nonlinear, possibly even chaotic [1.1] underlying processes. A further
diﬃculty is the huge amount of data necessary to be processed. Under these
circumstances, to develop models and schemes for automated, proﬁtable trading is a nontrivial task.
Apparently, the connectivity task involves mostly a programming eﬀort,
where a host of technical tools may considerably simplify the task. In practice an equal amount of work must be devoted to a proper design of various
software components and solving multiple hardware problems, given the following constraints:
– necessity of accessing multiple markets.
– lack of a standard API (Application Programming Interface) for accessing
diﬀerent exchanges.
– lack of an universal language of communication between ﬁnancial institutions.
– stringent reliability requirements posed on the communication infrastructure. 2 Lester Ingber et Radu P Mondescu Currently, there are sustained eﬀorts toward an uniﬁed, nonproprietary ﬁnancial “electronic” language (FIX — Financial Information Exchange —
open protocol [1.2]). FIX approach is to deﬁne and promote a common set of
types of messages, their format and the sessionlevel interaction, for communicating securities transactions between two parties, in a realtime electronic
trading environment.
1.1.1 Approaches
Note that more detailed discussions pertinent to the theoretical model underlying the trading system and computational aspects were published previously as Ref. [1.3].
Regarding the ﬁnancial modeling aspect, in the context of this paper,
it is important to stress that dealing with such complex systems invariably
requires modeling of dynamics, modeling of actions on these dynamics, and
algorithms to ﬁt parameters in these models to real data. We have elected to
use methods of mathematical physics for our models of the dynamics, artiﬁcial
intelligence (AI) heuristics for our models of trading rules acting on indicators
derived from our dynamics, and methods of sampling global optimization for
ﬁtting our parameters. Too often there is confusion about how these three
elements are being used for a complete system. For example, in the literature
there often is discussion of neural net trading systems or genetic algorithm
trading systems. However, neural net models (used for either or both models
discussed here) also require some method of ﬁtting their parameters, and
genetic algorithms must have some kind of cost function or process speciﬁed
to sample a parameter space, etc.
Some powerful methods have emerged during years, appearing from at
least two directions: One direction is based on inferring rules from past and
current behavior of market data leading to learningbased, inductive techniques, such as neural networks, or fuzzy logic. Another direction starts from
the bottomup, trying to build physical and mathematical models based on
diﬀerent economic prototypes. In many ways, these two directions are complementary and a proper understanding of their main strengths and weaknesses
should lead to synergetic eﬀects beneﬁcial to their common goals.
Among approaches in the ﬁrst direction, neural networks already have
won a prominent role in the ﬁnancial community, due to their ability to handle large quantities of data, and to uncover and model nonlinear functional
relationships between various combinations of fundamental indicators and
price data [1.4][1.5].
In the second direction we can include models based on nonequilibrium
statistical mechanics [1.6] fractal geometry [1.7], turbulence [1.8], spin glasses
and random matrix theory [1.9], renormalization group [1.10], and gauge
theory [1.11]. Although the very complex nonlinear multivariate character of
ﬁnancial markets is recognized [1.12], these approaches seem to have had a
lesser impact on current quantitative ﬁnance practice, although it is becoming 1. Automated Internet Trading 3 increasing clear that this direction can lead to practical trading strategies and
models.
To bridge the gap between theory and practice, as well as to aﬀord a
comparison with neural networks techniques, here we focus on presenting an
eﬀective trading system of S&P futures, anchored in the physical principles of
nonequilibrium statistical mechanics applied to ﬁnancial markets [1.13][1.6].
Starting with nonlinear, multivariate, nonlinear stochastic diﬀerential
equation descriptions of the price evolution of cash and futures indices, we
build an algebraic cost function in terms of a Lagrangian. Then, a maximum
likelihood ﬁt to the data is performed using a global optimization algorithm,
Adaptive Simulated Annealing (ASA) [1.14]. As ﬁrmly rooted in ﬁeld theoretical concepts, we derive market canonical momenta indicators, and we
use these as technical signals in a recursive ASA optimization that tunes the
outershell of trading rules. We do not employ metaphors for these physical
indicators, but rather derive them directly from models ﬁt to data.
The outline of the paper is as follows: Just below we brieﬂy discuss the
optimization method and momenta indicators.
In Section 1.2 we discuss some general, technical elements related to building an internetbased interface between the provider of ﬁnancial services (e.g.,
an exchange) and the client using an electronic trading system.
In the ensuing two sections we establish the theoretical framework supporting our model, and the statistical mechanics approach together with the
optimization method, respectively. In Section 1.5 we detail the trading system, and in Section 1.6 we describe our results. Our conclusions are presented
in Section 1.7.
1.1.2 Optimization
Largescale, nonlinear ﬁts of stochastic nonlinear forms to ﬁnancial data
require methods robust enough across data sets. (Just one day, tick data
for regular trading hours could reach 10,00030,000 data points.) Simple regression techniques exhibit deﬁciencies with respect to obtaining reasonable
ﬁts. They too often get trapped in local minima typically found in nonlinear
stochastic models of such data. ASA is a global optimization algorithm that
has the advantage — with respect to other global optimization methods as
genetic algorithms, combinatorial optimization, etc. — not only to be eﬃcient in its importancesampling search strategy, but to have the statistical
guarantee of ﬁnding the best optima [1.15][1.16]. This gives some conﬁdence
that a global minimum can be found, of course provided care is taken as
necessary to tune the algorithm [1.17].
It should be noted that such powerful sampling algorithms also are often required by other models of complex systems than those we use here
[1.18]. For example, neural network models have taken advantage of ASA
[1.19][1.20][1.21], as have other ﬁnancial and economic studies [1.22][1.23]. 4 Lester Ingber et Radu P Mondescu 1.1.3 Indicators
In general, neural network approaches attempt classiﬁcation and identiﬁcation of patterns, or try forecasting patterns and future evolution of ﬁnancial time series. Statistical mechanical methods attempt to ﬁnd dynamic
indicators derived from physical models based on general principles of nonequilibrium stochastic processes that reﬂect certain market factors. These
indicators are used subsequently to generate trading signals or to try forecasting upcoming data.
In this paper, the main indicators are called Canonical Momenta Indicators (CMI), as they faithfully mathematically carry the signiﬁcance of market
momentum, where the “mass” is inversely proportional to the price volatility
(the “masses” are just the elements of the metric tensor in this Lagrangian
formalism) and the “velocity” is the rate of price changes. 1.2 CONNECTION TO ELECTRONIC EXCHANGES
The growth of internet as a communication infrastructure and the exponential increase in computer power drastically altered the mechanics of securities
trading. Electronic matching of orders eliminates market makers and brokers
as intermediaries, allowing a vast increase in the number of market participants and better terms for ﬁnancial execution of trading orders.
Despite more or less visible obstructions by the traditional players, electronic exchanges appeared or traditional exchanges converted to electronic
ones (DTB—Germany, Matif—France, LIFFE—UK, Eurex—Germany and
Switzerland merged futures exchanges) and their volume exploded [1.24].
Intraday price feeds, realtime streaming quotes (even order books—
commonly referred to as Level II quotes— [1.25]) and integrated trade systems are available at almost no cost, and complicated models could be programmed and run by all market participants.
Sophisticated automated systems at large ﬁnancial institutions could
browse a wealth of data and ﬁltered it, based on various theoretical models, in the search of the arbitrage opportunity.
All these developments have made more prominent the role and the functionality of the interface connecting the trading system to the provider of
ﬁnancial services (which include both data sources and exchanges). By ﬁnancial services we refer throughout to services related to trading (submission
of orders, trading support or clearing services) provided by an exchange or
other ﬁnancial institutions to an end user client.
As a software application, a trading system has mainly two components:
the computational kernel and the connection API. We talk here about the
connection API at the client organization level. This API it is the software
layer allowing a trading tool to communicate with an exchange (or other
provider of ﬁnancial services). 1. Automated Internet Trading 5 Based on the data (prices, volume, time, various indicators) input and on
the theoretical model used, the computational kernel generates the trading
signals and sends them to the order execution module, a component of the
connection API.
The connection API must address two classes of problems:
1. Access to realtime price quotes.
2. Execution of the trade order.
We remark that above and in what follows we choose to use—for clarity
purposes—the term connection API as a rather broad grouping of functional
units that may not necessarily reﬂect a more constrained software engineering
point of view. For example, in most cases the data access component requires
a separate, independent development eﬀort from the order execution module.
A more complex, commercial version of a connection API should have
certain features, among which we list
– enables universal access to multiple exchanges with unique API,
– allows proprietary trading tools or other systems to connect to the order
execution system,
– provides compatibility with multiple ﬁnancial instruments (stocks, bonds,
futures, etc...),
– provides order routing service with realtime updates and various execution
types and order qualiﬁers,
– provides backoﬃce services (trade conﬁrmations, proﬁt/loss reports, execution reports, full order book update, settlement prices),
– provides market news services (market opens/closes announcements, market updates, instruments status/speciﬁcations changes),
– provides queries service: range of trades, range of prices, product speciﬁcation changes.
Collecting and processing realtime price data could be done using 3rd
party applications (two random examples: Reuters Triarch realtime services,
ESignal data services [1.26]), or by directly writing into the API provided by
the exchange, e.g., the Chicago Mercantile Exchange (CME) Market Data
API— MDAPI 1.0 — or the Eurex Values/Gate 3.0 API [1.28].
Usually, most vendors provide integrated solutions, essentially trading
applications that combine both the data and the execution systems. These
applications are usually blackbox systems that does not oﬀer a lower level
control of data, trading signals and trading orders, imperative requirements
for building a proprietary trading tool.
We focus next on describing the technological and design aspects common
to the connection API, with emphasis on the order routing component of the
API. We choose to do so because it is more complex than the data access
module and less details are available to a general audience. 6 Lester Ingber et Radu P Mondescu 1.2.1 Internet connectivity: overview
In general, connecting a trading system directly to one (or multiple) exchanges is a process requiring support and control from the dedicated technology and marketing departments of the exchange. It is reasonably understood that the trading system cannot be launched live without passing several
quality control checkpoints, imposed both by inhouse and exchange Quality
Assurance (QA) departments.
The evolution of the trading application from concept to production tool
could be subscribed to the following milestones:
– initial software development (concept, design, prototype),
– advanced development,
– technical certiﬁcation with substages
– functional testing,
– failover/recovery testing,
– stress testing,
– network certiﬁcation,
– preproduction testing
– connectivity testing,
– clearing cycle (endtoend) testing.
Associated with these development stages, various requirements (hardware and software) must be met within the automated trading environment.
We describe these requirements below.
1.2.2 Internet connectivity: hardware requirements
Reliable data feeds are critical components of a successful automated trading
system.
Internet access to exchanges through 3rd party applications/intermediaries
and standard communication infrastructure (modems, cable modems, DSL,
etc...) is possible, but due to reliability concerns and higher probability of
connection breakdowns, it is limited for trading systems operating at longer
time scales (daily, weekly trades) and lower trading volumes, or to personal
trading.
When trading time scale decreases to minutes or seconds and large transactions, direct access to exchanges, with dedicated lines is required.
For both data access and order routing, the development, initial testing
and certiﬁcation phases require at least an ISDN line. The production stage
necessitates frame relay (e.g., 256k AT&T) and ISDN connections as main
communication backbone, and backup lines, respectively.
Routers (Cisco 800, 2610) and possibly, a separate diagnostic line, are
also required, as well as some 3rd party software applications (e.g., Reuters
TIBCO). 1. Automated Internet Trading 7 All these equipment is usually installed by exchange personnel in collaboration hardware manufacturers technical support.
Costs and timelines for hardware deployment should be factor in when
evaluating capabilities of a trading model.
1.2.3 Internet connectivity: software requirements
Besides design aspects, important considerations are the choice of language
and development platform.
At this moment, preponderantly for trading engines requiring fast execution, Java still does not oﬀer the required speed and reliability. The languages
of choice remain C++ and C.
Although at the client level, the computational kernel could be developed
on any software platform, the need to interface with the API provided by
exchanges limits considerably the platform choices: currently, Windows NT
and Sun Solaris are the preferred operating systems, with some exchanges
supporting also IBM AIX.
Moreover, commercial development environments (as Microsoft Visual
Studio or Sun Workshop) and sometimes 3rd party libraries (e.g., Rogue
Wave [1.29]) are also necessary (at least when reaching certiﬁcation and production levels), as only these are usually supported by exchanges.
1.2.4 API order execution module: components and functionality
In terms of design, the connection API must insulate the computational kernel of various code changes operated by outside providers (e.g., exchanges) to
which the system is connected. Function of speciﬁc interests, various design
patterns (factory, template, bridge, fa¸ade, adapter [1.30]) could be applied.
c
The basic order of events necessary to be handled by the order routing
and execution component of the connection API is:
1. initialization (instantiate various object factories, register with the server
to receive responses, etc...),
2. connect to exchange API server (open session),
3. authenticate connection (login),
4. subscribe to a particular instrument (or multiple instruments), or to a
particular ﬁeld of a instrument (e.g., bid prices for a certain stock),
5. create and submit orders,
6. terminate communication with the exchange server and disconnect.
After opening the trading session, the connection API should insure (when
queried) that connection status and execution reports are available.
Various types of order (market order, stop order, limit order, stop limit
order, market if touched = the opposite of a stop order) and types of timeinforce (we list here only those suitable for automated trading) must be 8 Lester Ingber et Radu P Mondescu handled by the order routing module. The particular order type and timeinforce type applied in actual trading are chosen function of the characteristics
of the trading model:
– ﬁllorkill, a limit order, which is canceled if not ﬁlled immediately and
completely,
– ﬁllandkill, a limit order that, if not ﬁlled completely, all remaining quantity is cancelled,
– goodtillcancel, an order to be held until ﬁlled or until is cancelled,
Note that not all of these above qualiﬁers are necessarily supported by
the exchange of interest.
The main task of the order execution API is to create orders. An order
will contain several ﬁelds, among which we list the most important:
–
–
–
–
–
–
–
– order identiﬁcation number,
exchange identiﬁcation code,
instrument identiﬁer,
order type (market, limit, stop,...),
execution type (ﬁllandkill, etc...),
price (for stop, limit, stoplimit orders),
quantity,
time of entry. The order execution API component sends and receives (generally FIXcompliant) messages. We quote several of them below:
– single order (new order for a single instrument),
– cancel request (request to cancel an order),
– cancel/replace request (a request to cancel a previous order and replace it
with a new order),
– status request (a request for status of an order)
– heartbeat (a periodic signal send by exchange server to verify that connection is alive),
– reject (the order was rejected by the exchange server),
– cancel reject (the cancel request send by the client was rejected by the
exchange server).
– execution report.
Logic for taking appropriate action function of the message (or combination
of messages) received must be implemented at the API level, in connection
with signals produced by the computational engine.
Finally, from a development point of view, correct processing of previous
categories of messages is essential. In particular some points need attention:
– the cancel/replace logic, which may depend on the exchange (e.g., with the
CME FIX API the client needs to send a status request to check the state
of an order), 1. Automated Internet Trading 9 – the closing of a session (should be done gracefully, otherwise lost messages
or damaged session accounting could occur),
– error handling (all possible errors/exceptions should be dealt properly),
– connection management (a crucial component of a connection API. The
API should dynamically monitor and react to connectivity problems). 1.3 MODELS
1.3.1 Langevin Equations for Random Walks
The use of Brownian motion as a model for ﬁnancial systems is generally
attributed to Bachelier [1.31], though he incorrectly intuited that the noise
scaled linearly instead of as the square root relative to the random logprice
variable. Einstein is generally credited with using the correct mathematical description in a larger physical context of statistical systems. However,
several studies imply that changing prices of many markets do not follow a
random walk, that they may have longterm dependences in price correlations, and that they may not be eﬃcient in quickly arbitraging new information [1.32][1.33][1.34]. A random walk for returns, rate of change of prices
over prices, is described by a Langevin equation with simple additive noise
η , typically representing the continual random inﬂux of information into the
market.
˙
M = −f + gη,
dM
˙
M=
,
dt
< η (t) >η = 0, < η (t), η (t ) >η = δ (t − t ), (1.1) where f and g are constants, and M is the logarithm of (scaled) price, M (t) =
log P (t)/P (t − dt) . Price, although the most dramatic observable, may not
be the only appropriate dependent variable or order parameter for the system
of markets [1.35]. This possibility has also been called the “semistrong form
of the eﬃcient market hypothesis” [1.32].
The generalization of this approach to include multivariate nonlinear nonequilibrium markets led to a model of statistical mechanics of ﬁnancial markets (SMFM) [1.13].
1.3.2 Adaptive Optimization of F x Models
Our S&P model for the evolution of futures price F is
dF = µdt + σF x dz,
< dz > = 0,
< dz (t) dz (t ) > = dtδ (t − t ), (1.2) 10 Lester Ingber et Radu P Mondescu where the exponent x of F is one of the dynamical parameters to be ﬁt to
futures data together with µ and σ .
We have used this model in several ways to ﬁt the distribution’s volatility
deﬁned in terms of a scale and an exponent of the independent variable [1.6].
A major component of our trading system is the use of adaptive optimization, essentially constantly retuning the parameters of our dynamic model
each time new data is encountered in our training, testing and realtime applications. The parameters {µ, σ } are constantly tuned using a quasilocal
simplex code [1.36][1.37] included with the ASA (Adaptive Simulated Annealing) code [1.14].
We have tested several quasilocal codes for this kind of trading problem,
versus using robust ASA adaptive optimizations, and the faster quasilocal
codes seem to work quite well for adaptive updates after a zeroth order parameters set is found by ASA [1.38][1.39]. 1.4 STATISTICAL MECHANICS OF FINANCIAL
MARKETS (SMFM)
1.4.1 Statistical Mechanics of Large Systems
Aggregation problems in nonlinear nonequilibrium systems typically are
“solved” (accommodated) by having new entities/languages developed at
these disparate scales in order to eﬃciently pass information back and forth
between scales. This is quite diﬀerent from the nature of quasiequilibrium
quasilinear systems, where thermodynamic or cybernetic approaches are possible. These thermodynamic approaches typically fail for nonequilibrium nonlinear systems.
Many systems are aptly modeled in terms of multivariate diﬀerential rateequations, known as Langevin equations [1.40],
˙
M G = f G + gj η j , (G = 1, . . . , Λ)(j = 1, . . . , N ),
ˆG
dM G
˙
MG =
,
dt
< η j (t) >η = 0, < η j (t), η j (t ) >η = δ jj δ (t − t ),
G gj
ˆG (1.3) where f and
are generally nonlinear functions of mesoscopic order paG
rameters M , j is an index indicating the source of ﬂuctuations, and N ≥ Λ.
The Einstein convention of summing over repeated indices is used. Vertical
bars on an index, e.g., j , imply no sum is to be taken on repeated indices.
The “microscopic” index j relates to the typical physical nature of ﬂuctuations in such statistical mechanical systems, wherein the variables η are
considered to be aggregated from ﬁner scales relative to the “mesoscopic”
variables M . 1. Automated Internet Trading 11 Via a somewhat lengthy, albeit instructive calculation, outlined in several
other papers [1.13][1.41][1.42], involving an intermediate derivation of a corresponding FokkerPlanck or Schr¨dingertype equation for the conditional
o
probability distribution P [M (t)M (t0 )], the Langevin rate Eq. (1.3) is developed into the more useful probability distribution for M G at longtime macroscopic time event tu+1 = (u +1)θ + t0 , in terms of a Stratonovich pathintegral
over mesoscopic Gaussian conditional probabilities [1.43][1.44][1.45][1.46][1.47].
Here, macroscopic variables are deﬁned as the longtime limit of the evolving
mesoscopic system.
The corresponding Schr¨dingertype equation is [1.45][1.46]
o
∂P
1
= (g GG P ),GG − (g G P ),G + V,
∂t
2
ˆG ˆG
g GG = δ jk gj gk ,
1
g G = f G + δ jk gj gk,G ,
ˆG ˆG
2
∂ [. . .]
.
(1.4)
[. . .],G =
∂M G
This is properly referred to as a FokkerPlanck equation when V ≡ 0. Note
that although the partial diﬀerential Eq. (1.4) contains information regarding
M G as in the stochastic diﬀerential Eq. (1.3), all references to j have been
properly averaged over. I.e., gj in Eq. (1.3) is an entity with parameters
ˆG
in both microscopic and mesoscopic spaces, but M is a purely mesoscopic
variable, and this is more clearly reﬂected in Eq. (1.4). In the following, we
often drop superscripts on M for clarity, with the understanding that M
represents the vector {M G }.
The path integral representation can be written in terms of the prepoint
discretized Lagrangian L, further discussed below [1.45][1.47][1.48],
P [M, tM, t0 ]dM (t) = ... DM exp(−S ) ×δ [M (t0 )]δ [M (t)],
t S = min dt L,
t0 u+1 g 1/2 DM = lim u→∞ v =1 (2πθ)−1/2 dM G (tv) ,
G 1˙
˙
˙
L(M , M , t) = (M G − g G )gGG (M G − g G )
2
−V,
G G gGG = (g GG )−1 ,
g = det(gGG ). (1.5) 12 Lester Ingber et Radu P Mondescu Mesoscopic variables have been deﬁned as M G in the Langevin and FokkerPlanck representations, in terms of their development from the microscopic
system labeled by j . The entity gGG , is a bona ﬁde metric of this space [1.45].
Shorttime “forecast” of data points is realized using the most probable path
equation [1.49]
dM G
= g G − g 1/2 (g −1/2 g GG ),G .
(1.6)
dt
In the economics literature, there appears to be sentiment to deﬁne Eq.
(1.3) by the Itˆ, rather than the Stratonovich prescription. It is true that
o
Itˆ integrals have Martingale properties not possessed by Stratonovich ino
tegrals [1.50] which leads to riskneural theorems for markets [1.51][1.52],
but the nature of the proper mathematics — actually a simple transformation between these two discretizations — should eventually be determined by proper aggregation of relatively microscopic models of markets.
It should be noted that virtually all investigations of other physical systems, which are also continuous time models of discrete processes, conclude that the Stratonovich interpretation coincides with reality, when multiplicative noise with zero correlation time, modeled in terms of white noise
η j , is properly considered as the limit of real noise with ﬁnite correlation
time [1.53]. The path integral succinctly demonstrates the diﬀerence between the two: The Itˆ prescription corresponds to the prepoint discretizao
˙
tion of L, wherein θ M (t) → M (tv+1 ) − M (tv ) and M (t) → M (tv ). The
Stratonovich prescription corresponds to the midpoint discretization of L,
1
˙
wherein θM (t) → M (tv+1 ) − M (tv ) and M (t) → 2 M (tv+1 ) + M (tv ) . In
terms of the functions appearing in the FokkerPlanck Eq. (1.4), the Itˆ preo
scription of the prepoint discretized Lagrangian L, Eq. (1.5), is relatively
simple, albeit deceptively so because of its nonstandard calculus. In the absence of a nonphenomenological microscopic theory, the diﬀerence between a
Itˆ prescription and a Stratonovich prescription is simply a transformed drift
o
[1.48].
There are several other advantages to Eq. (1.5) over Eq. (1.3). Extrema
and most probable states of M G ,
M G , are simply derived by a variational principle, similar to conditions sought in previous studies [1.54]. In
the Stratonovich prescription, necessary, albeit not suﬃcient, conditions are
given by
δG L = L,G − L,G:t = 0,
˙ ˙
¨
L,G:t = L,GG M G + L,GG M G .
˙
˙
˙˙ (1.7) ˙
¯
¯
¯
For stationary states, M G = 0, and ∂ L/∂ M G = 0 deﬁnes
M G , where
the bars identify stationary variables; in this case, the macroscopic variables
¯
are equal to their mesoscopic counterparts. Note that L is not the stationary solution of the system, e.g., to Eq. (1.4) with ∂P/∂t = 0. However, in
¯
some cases [1.55], L is a deﬁnite aid to ﬁnding such stationary states. Many 1. Automated Internet Trading 13 times only properties of stationary states are examined, but here a temporal
˙
dependence is included. E.g., the M G terms in L permit steady states and
their ﬂuctuations to be investigated in a nonequilibrium context. Note that
Eq. (1.7) must be derived from the path integral, Eq. (1.5), which is at least
one reason to justify its development.
1.4.2 Algebraic Complexity Yields Simple Intuitive Results
It must be emphasized that the output of this formalism is not conﬁned to
complex algebraic forms or tables of numbers. Because L possesses a variational principle, sets of contour graphs, at diﬀerent longtime epochs of the
pathintegral of P over its variables at all intermediate times, give a visually intuitive and accurate decisionaid to view the dynamic evolution of the
scenario. For example, this Lagrangian approach permits a quantitative assessment of concepts usually only loosely deﬁned.
∂L
“Momentum = Π G =
,
(1.8a)
∂ (∂M G /∂t)
∂2L
“Mass = gGG =
,
(1.8b)
G /∂t)∂ (∂M G /∂t)
∂ (∂M
∂L
“Force =
,
(1.8c)
∂M G
∂L
∂L
∂
“F = ma : δL = 0 =
−
,
(1.8d)
G
∂M
∂t ∂ (∂M G /∂t)
where M G are the variables and L is the Lagrangian. These physical entities
provide another form of intuitive, but quantitatively precise, presentation of
these analyses. For example, daily newspapers use some of this terminology
to discuss the movement of security prices. In this paper, the Π G serve as
canonical momenta indicators (CMI) for these systems.
Derived Canonical Momenta Indicators (CMI). The extreme sensitivity of the CMI gives rapid feedback on changes in trends as well as the
volatility of markets, and therefore are good indicators to use for trading rules
[1.38]. A timelocked moving average provides manageable indicators for trading signals. This current project uses such CMI developed as a byproduct of
the ASA ﬁts described below.
Intuitive Value of CMI. In the context of other invariant measures, the
CMI transform covariantly under Riemannian transformations, but are more
sensitive measures of activity than other invariants such as the energy density,
eﬀectively the square of the CMI, or the information which also eﬀectively
is in terms of the square of the CMI (essentially integrals over quantities
proportional to the energy times a factor of an exponential including the
energy as an argument). Neither the energy or the information give details
of the components as do the CMI. In oscillatory markets the relative signs of
such activity can be quite important. 14 Lester Ingber et Radu P Mondescu The CMI present single indicators for each member of a set of correlated markets, “orthogonal” in the deﬁned metric space. Each indicator is a
dynamic weighting of shorttime diﬀerenced deviations from drifts (trends)
divided by covariances (risks). Thus the CMI also give information complementary to just trends or standard deviations separately.
1.4.3 Correlations
In this paper we report results of our onevariable trading model. However, it
is straightforward to include multivariable trading models in our approach,
and we have done this, for example, with coupled cash and futures S&P
markets.
Correlations between variables are modeled explicitly in the Lagrangian as
a parameter usually designated ρ. This section uses a simple twofactor model
to develop the correspondence between the correlation ρ in the Lagrangian
and that among the commonly written Wiener distribution dz .
Consider coupled stochastic diﬀerential equations for futures F and cash
C:
dF = f F (F, C )dt + g F (F, C )σF dzF ,
ˆ
C C dC = f (F, C )dt + g (F, C )σC dzC ,
ˆ
< dzi > = 0, i = {F, C }, < dzi (t)dzj (t ) > = dtδ (t − t ), i = j,
< dzi (t)dzj (t ) > = ρdtδ (t − t ), i = j, (1.9a)
(1.9b)
(1.9c)
(1.9d)
(1.9e) where < . > denotes expectations with respect to the multivariate distribution.
These can be rewritten as Langevin equations (in the Itˆ prepoint diso
cretization)
dF
= f F + g F σF (γ + η1 + sgnρ γ − η2 ),
ˆ
dt
dC
= g C + g C σC (sgnρ γ − η1 + γ + η2 ),
ˆ
dt
1
γ ± = √ [1 ± (1 − ρ2 )1/2 ]1/2 ,
2
ni = (dt)1/2 pi , (1.10a)
(1.10b)
(1.10c)
(1.10d) where p1 and p2 are independent [0,1] Gaussian distributions.
The equivalent shorttime probability distribution, P , for the above set
of equations is
P = g 1/2 (2πdt)−1/2 exp(−Ldt),
1
L = M † gM,
2 1. Automated Internet Trading M= dF
dt
dC
dt − fF
− fC 15 , g = det(g ). (1.11) g , the metric in {F, C }space, is the inverse of the covariance matrix,
g−1 = (g F σF )2 ρg F g C σF σC
ˆ
ˆˆ
ρg F g C σF σC (ˆC σC )2
ˆˆ
g . (1.12) The CMI indicators are given by the formulas
(dF/dt − f F )
ρ(dC/dt − f C )
−FC
,
F σ )2 (1 − ρ2 )
(ˆ F
g
g g σF σC (1 − ρ2 )
ˆˆ
ρ(dF/dt − f F )
(dC/dt − f C )
− CF
.
=C
2 (1 − ρ2 )
(ˆ σC )
g
g g σC σF (1 − ρ2 )
ˆˆ ΠF = (1.13a) ΠC (1.13b) 1.4.4 ASA Outline
The algorithm Adaptive Simulated Annealing (ASA) ﬁts shorttime probability distributions to observed data, using a maximum likelihood technique
on the Lagrangian. This algorithm has been developed to ﬁt observed data
to a theoretical cost function over a Ddimensional parameter space [1.15],
adapting for varying sensitivities of parameters during the ﬁt. The ASA code
can be obtained at no charge, via WWW from http://www.ingber.com/ or
via FTP from ftp.ingber.com [1.14].
General Description. Simulated annealing (SA) was developed in 1983 to
deal with highly nonlinear problems [1.56], as an extension of a MonteCarlo
importancesampling technique developed in 1953 for chemical physics problems. In 1984 [1.57], it was established that SA possessed a proof that, by
carefully controlling the rates of cooling of temperatures, it could statistically
ﬁnd the best minimum, e.g., the lowest valley of our example above. This was
good news for people trying to solve hard problems which could not be solved
by other algorithms. The bad news was that the guarantee was only good
if they were willing to run SA forever. In 1987, a method of fast annealing
(FA) was developed [1.58], which permitted lowering the temperature exponentially faster, thereby statistically guaranteeing that the minimum could
be found in some ﬁnite time. However, that time still could be quite long.
Shortly thereafter, Very Fast Simulated Reannealing (VFSR) was developed
in 1987 [1.15], now called Adaptive Simulated Annealing (ASA), which is
exponentially faster than FA.
ASA has been applied to many problems by many people in many disciplines [1.18][1.17][1.59]. The feedback of many users regularly scrutinizing the
source code ensures its soundness as it becomes more ﬂexible and powerful. 16 Lester Ingber et Radu P Mondescu Multiple Local Minima. Our criteria for the global minimum of our cost
function is minus the largest proﬁt over a selected training data set (or in
some cases, this value divided by the maximum drawdown). However, in
many cases this may not give us the best set of parameters to ﬁnd proﬁtable
trading in test sets or in realtime trading. Other considerations such as the
total number of trades developed by the global minimum versus other close
local minima may be relevant. For example, if the global minimum has just a
few trades, while some nearby local minima (in terms of the value of the cost
function) have many trades and was proﬁtable in spite of our slippage factors,
then the scenario with more trades might be more statistically dependable
to deliver proﬁts across testing and realtime data sets.
Therefore, for the outershell global optimization of training sets, we have
used an ASA OPTION, MULTI MIN, which saves a userdeﬁned number of
closest local minima within a userdeﬁned resolution of the parameters. We
then examine these results under several testing sets. 1.5 TRADING SYSTEM 1.5.1 Use of CMI
As the CMI formalism carries the relevant information regarding the prices
dynamics, we have used it as a signal generator for an automated trading
system for S&P futures.
While currently we are integrating fastresponse CMI signals into the
trading model, next we discuss averaged CMI signals characterizing longer
time scales.
Based on a previous work [1.39] applied to daily closing data, the overall
structure of the trading system consists in 2 layers, as follows: We ﬁrst construct the “shorttime” Lagrangian function in the Itˆ representation (with
o
the notation introduced in Section 3.3)
L(ii − 1) = 1
2σ 2 Fi2x1
− dFi
− fF
dt 2 (1.14) with i the postpoint index, corresponding to the one factor price model
dF = f F dt + σF x dz (t), (1.15) where f F and σ > 0 are taken to be constants, F (t) is the S&P future price,
and dz is the standard Gaussian noise with zero mean and unit standard
deviation. We perform a global, maximum likelihood ﬁt to the whole set of
price data using ASA. This procedure produces the optimization parameters
{x, f F } that are used to generate the CMI. One computational approach
was to ﬁx the diﬀusion multiplier σ to 1 during training for convenience, but 1. Automated Internet Trading 17 used as free parameters in the adaptive testing and realtime ﬁts. Another
approach was to ﬁx the scale of the volatility, using an improved model,
dF = f F dt + σ F
<F > x dz (t), (1.16) where σ now is calculated as the standard deviation of the price increments
∆F/dt1/2 , and < F > is just the average of the prices.
As already remarked, to enhance the CMI sensitivity and response time
to local variations (across a certain window size) in the distribution of price
increments, the momenta are generated applying an adaptive procedure, i.e.,
after each new data reading another set of {f F , σ } parameters are calculated
for the last window of data, with the exponent x—a contextual indicator of
the noise statistics—ﬁxed to the value obtained from the global ﬁt.
The CMI computed in this manner are fed into the outer shell of the trading system, where an AItype optimization of the trading rules is executed,
using ASA once again.
The trading rules are a collection of logical conditions among the CMI,
prices and optimization parameters that could be window sizes, time resolutions, or trigger thresholds. Based on the relationships between CMI and
optimization parameters, a trading decision is made. The cost function in the
outer shell is either the overall equity or the riskadjusted proﬁt (essentially
the return). The inner and outer shell optimizations are coupled through
some of the optimization parameters (e.g., time resolution of the data, window sizes), which justiﬁes the recursive nature of the optimization.
Next, we describe in more details the concrete implementation of this
system.
1.5.2 Data Processing
The CMI formalism is general and by construction permits us to treat multivariate coupled markets. In certain conditions (e.g., shorter time scales of
data), and also due to superior scalability across diﬀerent markets, it is desirable to have a trading system for a single instrument, in our case the S&P
futures contracts that are traded electronically on Chicago Mercantile Exchange (CME). The focus of our system was intraday trading, at time scales
of data used in generating the buy/sell signals from 10 to 60 secs. In particular, we here give some results obtained when using data having a time
resolution ∆t of 55 secs (the time between consecutive data elements is 55
secs). This particular choice of time resolution reﬂects the set of optimization
parameters that have been applied in actual trading.
It is important to remark that a data point in our model does not necessarily mean an actual tick datum. For some trading time scales and for noise
reduction purposes, data is preprocessed into sampling bins of length ∆t using either a standard averaging procedure or spectral ﬁltering (e.g., wavelets,
Fourier) of the tick data. Alternatively, the data can be deﬁned in block bins 18 Lester Ingber et Radu P Mondescu that contain disjoint sets of averaged tick data, or in overlapping bins of
widths ∆t that update at every ∆t < ∆t, such that an eﬀective resolution
∆t shorter than the width of the sampling bin is obtained. We present here
work in which we have used disjoint block bins and a standard average of the
tick data with time stamps falling within the bin width.
In Figs. 1 and 2 we present examples of S&P futures data sampled with
55 secs resolution. We remark that there are several time scales—from mins
to one hour—at which an automated trading system might extract proﬁts.
ESU0 data June 20
time resolution = 55 secs 1505 Futures
Cash 1500
1495 S&P 1490
1485 1480
1475
1470
1465 0620 10:46:16 0620 11:45:53 0620 12:45:30 0620 13:45:07 0620 14:44:44 TIME (mmdd hhmmss) Fig. 1.1. Futures and
cash data, contract ESU0
June 20: (solid line) —
futures; (dashed line) —
cash ESU0 data June 22
time resolution = 55 secs
1485
Futures
Cash 1480
1475 S&P 1470
1465
1460
1455
1450
0622 12:56:53 0622 13:56:30
TIME (mmdd hhmmss) 0622 14:56:07 Fig. 1.2. Futures and
cash data, contract ESU0
June 22: (solid line) —
futures; (dashed line) —
cash Fig. 1 illustrates that the proﬁtable regions are prominent even for data
representing a relatively ﬂat market period. I.e., June 20 shows an uptrend 1. Automated Internet Trading 19 region of about 1 hour 20 mins and several short and long trading domains
between 10 mins and 20 mins.
Fig. 2 illustrates the sustained short trading region of 1.5 hours and several
shorter long and short trading regions of about 10–20 mins.
In both situations, there are a larger number of opportunities at time
resolutions smaller than 5 mins.
The time scale at which we sample the data for trading is itself a parameter that is extracted from the optimization of the trading rules and of the
Lagrangian cost function Eq. (1.14). This is one of the coupling parameters
between the inner and the outershell optimizations. 1.5.3 InnerShell Optimization
A cycle of optimization runs has three parts, training and testing, and ﬁnally realtime use—a variant of testing. Training consists in choosing a data
set and performing the recursive optimization, which produces optimization
parameters for trading. In our case there are six parameters: the time resolution ∆t of price data, the length of window W used in the local ﬁtting
procedures and in computation of moving averages of trading signals, the
drift f F , volatility coeﬃcient σ and exponent x from Eq. (1.15), and a multiplicative factor M necessary for the trading rules module, as discussed below.
The optimization parameters computed from the training set are applied
then to various test sets and ﬁnal proﬁt/loss analyses are produced. Based
on these, the best set of optimization parameters are chosen to be applied
in realtime trading runs. We remark once again that a single training data
set could support more than one proﬁtable sets of parameters and can be a
function of the trader’s interest and the speciﬁc market dynamics targeted
(e.g., short/long time scales). The optimization parameters corresponding to
the global minimum in the training session may not necessarily represent the
parameters that led to robust proﬁts across realtime data.
The training optimization occurs in two interrelated stages. An innershell maximum likelihood optimization over all training data is performed.
The cost function that is ﬁtted to data is the eﬀective action constructed from
the Lagrangian Eq. (1.14) including the prefactors coming from the measure
element in the expression of the shorttime probability distribution Eq. (1.11).
This is based on the fact [1.48] that in the context of Gaussian multiplicative
stochastic noise, the macroscopic transition probability P (F, tF , t ) to start
with the price F (= Fi−1 ) at t (= ti−1 ) and reach the price F (= Fi ) at
t(= ti )y is determined by the shorttime Lagrangian Eq. (1.14),
1
P (F, tF , t ) =
(2πσ 2 Fi2x1 dti )1/2
−
N × exp − i=1 L(ii − 1)dti , (1.17) 20 Lester Ingber et Radu P Mondescu with dti = ti −ti−1 . Recall that the main assumption of our model is that price
increments (or the logarithm of price ratios, depending on which variables are
considered independent) could be described by a system of coupled stochastic,
nonlinear equations as in Eq. (1.9a). These equations are deceptively simple
in structure, yet depending on the functional form of the drift coeﬃcients and
the multiplicative noise, they could describe a variety of interactions between
ﬁnancial instruments in various market conditions (e.g., constant elasticity
of variance model [1.60], stochastic volatility models, etc.). In particular, this
type of models include the case of BlackScholes price dynamics (x = 1).
In the system presented here, we have applied the model from Eq. (1.15).
The ﬁtted parameters were the drift coeﬃcient f F and the exponent x. In the
case of a coupled futures and cash system, besides the corresponding values
of f F and x for the cash index, another parameter, the correlation coeﬃcient
ρ as introduced in Eq. (1.9a), must be considered.
1.5.4 Trading Rules (OuterShell) Recursive Optimization
In the second part of the training optimization, we calculate the CMI and
execute trades as required by a selected set of trading rules based on CMI
values, price data or combinations of both indicators.
Recall that three external shell optimization parameters are deﬁned: the
time resolution ∆t of the data expressed as the time interval between consecutive data points, the window length W (in number of time epochs or data
points) used in the adaptive calculation of CMI, and a numerical coeﬃcient
M that scales the momentum uncertainty discussed below.
At each moment a local reﬁt of f F and σ over data in the local window
W is executed, moving the window M across the training data set and using
the zeroth order optimization parameters f F and x resulting from the innershell optimization as a ﬁrst guess. It was found that a faster quasilocal code
is suﬃcient for computational purposes for these adaptive updates. In more
complicated models, ASA can be successfully applied recursively, although
in realtime trading the response time of the system is a major factor that
requires attention.
All expressions that follow can be generalized to coupled systems in the
manner described in Section 3. Here we use the one factor nonlinear model
given by Eq. (1.15). At each time epoch we calculate the following momentum
related quantities:
ΠF = 1
σ 2 F 2x F
Π0 = − dF
− fF
dt , fF
,
σ 2 F 2x ∆Π F = < (Π F − < Π F >)2 >1/2 = 1
√,
dt σF x (1.18) 1. Automated Internet Trading 21 where we have used < Π F >= 0 as implied by Eqs. (1.15) and (1.14). In the
F
previous expressions, Π F is the CMI, Π0 is the neutral line or the momentum
F
of a zero change in prices, and ∆Π is the uncertainty of momentum. The
last quantity reﬂects the Heisenberg principle, as derived from Eq. (1.15) by
calculating
√
∆F ≡ < (dF − < dF >)2 >1/2 = σF x dt,
∆Π F ∆F ≥ 1,
(1.19) where all expectations are in terms of the exact noise distribution, and the
calculation implies the Itˆ approximation (equivalent to considering nono
anticipative functions). Various moving averages of these momentum signals
are also constructed. Other dynamical quantities, as the Hamiltonian, could
be used as well. (By analogy to the energy concept, we found that the Hamiltonian carries information regarding the overall trend of the market, giving
another useful measure of price volatility.)
Regarding the practical implementation of the previous relations for trading, some comments are necessary. In terms of discretization, if the CMI are
calculated at epoch i, then dFi = Fi − Fi−1 , dti = ti − ti−1 = ∆t, and
all prefactors are computed at moment i − 1 by the Itˆ prescription (e.g.,
o
σF x = σFix 1 ). The momentum uncertainty band ∆Π F can be calculated
−
from the discretized theoretical value Eq. (1.18), or by computing the estimator of the standard deviation from the actual time series of Π F .
There are also two ways of calculating averages over CMI values: One
way is to use the set of local optimization parameters {f F , σ } obtained from
the local ﬁt procedure in the current window W for all CMI data within
that window (localmodel average). The second way is to calculate each CMI
in the current local window W with another set {f F , σ } obtained from a
previous local ﬁt window measured from the CMI data backwards W points
(multiplemodels averaged, as each CMI corresponds to a diﬀerent model in
terms of the ﬁtting parameters {f F , σ }).
The last observation is that the neutral line divides all CMI in two classes:
F
long signals, when Π F > Π0 , as any CMI satisfying this condition indicates
F
a positive price change, and short signals when Π F < Π0 , which reﬂects a
negative price change.
After the CMI are calculated, based on their meaning as statistical momentum indicators, trades are executed following a relatively simple model:
Entry in and exit from a long (short) trade points are deﬁned as points
where the value of CMIs is greater (smaller) than a certain fraction of the
uncertainty band M ∆Π F (−M ∆Π F ), where M is the multiplicative factor
mentioned in the beginning of this subsection. This is a choice of a symmetric trading rule, as M is the same for long and short trading signals,
which is suitable for volatile markets without a sustained trend, yet without
diminishing too severely proﬁts in a strictly bull or bear region.
Inside the momentum uncertainty band, one could deﬁne rules to stay in
a previously open trade, or exit immediately, because by its nature the mo 22 Lester Ingber et Radu P Mondescu mentum uncertainty band implies that the probabilities of price movements
in either direction (up or down) are balanced. From another perspective, this
type of trading rule exploits the relaxation time of a strong market advance
or decline, until a trend reversal occurs or it becomes more probable.
Other sets of trading rules are certainly possible, by utilizing not only
the current values of the momenta indicators, but also their localmodel or
multiplemodels averages. A trading rule based on the maximum distance
F
between the current CMI data ΠiF and the neutral line Π0 shows faster
response to markets evolution and may be more suitable to automatic trading
in certain conditions.
Stepping through the trading decisions each trading day of the training
set determined the proﬁt/loss of the training set as a single value of the
outersell cost function. As ASA importancesampled the outershell parameter space {∆t, W, M }, these parameters are fed into the inner shell, and a
new innershell recursive optimization cycle begins. The ﬁnal values for the
optimization parameters in the training set are ﬁxed when the largest net
proﬁt (calculated from the total equity by subtracting the transactions costs
deﬁned by the slippage factor) is realized. In practice, we have collected optimization parameters from multiple local minima that are near the global
minimum (the outershell cost function is deﬁned with the sign reversed) of
the training set.
The values of the optimization parameters {∆t, W, M, f F , σ, x} resulting
from a training cycle are then applied to outofsample test sets. During the
test run, the drift coeﬃcient f F and the volatility coeﬃcient σ are reﬁtted
adaptively as described previously. All other parameters are ﬁxed. We have
mentioned that the optimization parameters corresponding to the highest
proﬁt in the training set may not be the suﬃciently robust across test sets.
Then, for all test sets, we have tested optimization parameters related to
the multiple minima (i.e., the global maximum proﬁt, the second best proﬁt,
etc.) resulting from the training set.
We performed a bootstraptype reversal of the trainingtest sets (repeating the training runs procedures using one of the test sets, including the
previous training set in the new batch of test sets), followed by a selection
of the best parameters across all data sets. This is necessary to increase the
chances of successful trading sessions in realtime. 1.6 RESULTS 1.6.1 Alternative Algorithms
In the previous sections we noted that there are diﬀerent combinations of
methods of processing data, methods of computing the CMI and various sets 1. Automated Internet Trading 23 of trading rules that need to be tested — at least in a sampling manner —
before launching trading runs in realtime:
1. Data can be preprocessed in block or overlapping bins, or forecasted
data derived from the most probable transition path [1.49] could be used as
in one of our most recent models.
2. Exponential smoothing, wavelets or Fourier decomposition can be applied for statistical processing. We presently favor exponential moving averages.
3. The CMI can be calculated using averaged data or directly with tick
data, although the optimization parameters were ﬁtted from preprocessed
(averaged) price data.
4. The trading rules can be based on current signals (no average is performed over the signal themselves), on various averages of the CMI trading
signals, on various combination of CMI data (momenta, neutral line, uncertainty band), on symmetric or asymmetric trading rules, or on mixed
priceCMI trading signals.
5. Diﬀerent models (one and twofactors coupled) can be applied to the
same market instrument, e.g., to deﬁne complementary indicators.
The selection process evidently must consider many speciﬁc economic
factors (e.g., liquidity of a given market), besides all other physical, mathematical and technical considerations. In the work presented here, as we tested
our system and using previous experience, we focused toward S&P500 futures
electronic trading, using block processed data, and symmetric, localmodel
and multiplemodels trading rules based on CMI neutral line and stayin
conditions.
1.6.2 Trading System Design
The design of a successful electronic trading system is complex as it must incorporate several aspects of a trader’s actions that sometimes are diﬃcult to
translate into computer code. Three important features that must be implemented are factoring in the transactions costs, devising money management
techniques, and coping with execution deﬁciencies.
Generally, most trading costs can be included under the “slippage factor,”
although this could easily lead to poor estimates. Given that the margin of
proﬁts from exploiting market ineﬃciencies are thin, a high slippage factor
can easily result in a nonproﬁtable trading system. In our situation, for
testing purposes we used a $35 slippage factor per buy & sell order, a value
we believe is rather high for an electronic trading environment, although it
represents less than three ticks of a miniS&P futures contract. (The miniS&P is the S&P futures contract that is traded electronically on CME.) This
higher value was chosen to protect ourselves against the bidask spread, as our
trigger price (at what price the CMI was generated) and execution price (at
what price a trade signaled by a CMI was executed) were taken to be equal
to the trading price. (We have changed this aspect of our algorithm in later 24 Lester Ingber et Radu P Mondescu models.) The slippage is also strongly inﬂuenced by the time resolution of the
data. Although the slippage is linked to bidask spreads and markets volatility
in various formulas [1.61], the best estimate is obtained from experience and
actual trading.
Money management was introduced in terms of a trailing stop condition
that is a function of the price volatility, and a stoploss threshold that we
ﬁxed by experiment to a multiple of the miniS&P contract value ($200).
It is tempting to tighten the trailing stop or to work with a small stoploss
value, yet we found — as otherwise expected — that higher losses occurred
as the signals generated by our stochastic model were bypassed.
Regarding the execution process, we have to account for the response
of the system to various execution conditions in the interaction with the
electronic exchange: partial ﬁlls, rejections, uptick rule (for equity trading),
etc. Except for some special conditions, all these steps must be automated.
1.6.3 Some Explicit Results
Typical CMI data in Figs. 3 and 4 (obtained from realtime trading after a
full cycle of trainingtesting was performed) are related to the price data in
Figs. 1 and 2. We have plotted the fastest (55 secs apart) CMI values Π F , the
F
neutral line Π0 and the uncertainty band ∆Π F . All CMI data were produced
using the optimization parameters set {55secs, 88epochs, 0.15} of the secondbest net proﬁt obtained with a training set based on the March data of the
ESM0 contract (miniS&P June 2000 contract). We recall the meaning of the
optimization parameters from 1.5.4: the ﬁrst factor is the frequency of CMI
signals (or timestep between consecutive CMIs), the second parameter is the
width in timestep units of the timewindow used for local statistics, and the
third parameter is the scaling factor of the momentum uncertainty.
Canonical Momenta Indicators (CMI)
time resolution = 55 secs
F Π (CMI Futures) 8 Π F (neutral CMI) 0
F ∆Π (theory) CMI 4 0 4 8 0620 10:46:16 0620 11:45:53 0620 12:45:30 0620 13:45:07 TIME (mmdd hhmmss) 0620 14:44:44 Fig. 1.3. CMI data,
realtime trading June
20: (solid line) — CMI;
(dashed line) — neutral
line; (dotted line) —
uncertainty band 1. Automated Internet Trading 25 Canonical Momenta Indicators (CMI)
time resolution = 55 secs F 8 Π (CMI Futures)
Π F (neutral CMI) 0
F ∆Π (theory) CMI 4 0 4 8 0622 12:56:53 0622 13:56:30 0622 14:56:07 TIME (mmdd hhmmss) Fig. 1.4. CMI data,
realtime trading, June
22: (solid line) — CMI;
(dashed line) — neutral
line; (dotted line) —
uncertainty band Although the CMIs exhibit an inherently ragged nature and oscillate
around a zero mean value within the uncertainty band — the width of which is
decreasing with increasing price volatility, as the uncertainty principle would
also indicate — time scales at which the CMI average or some persistence
time are not balanced about the neutral line.
These characteristics, which we try to exploit in our system, are better
depicted in Figs. 5 and 6.
Canonical Momenta Indicators (CMI)
time resolution = 55 secs
1
F <Π 0> (local)
M ∆Π F CMI 0.5 0 0.5 1 0620 10:46:16 0620 11:45:53 0620 12:45:30 0620 13:45:07 TIME (mmdd hhmmss) 0620 14:44:44 Fig. 1.5. CMI trading
signals, realtime trading
June 20: (dashed line)
— localmodel average
of the neutral line; (dotted line) — uncertainty
band multiplied by the
optimization parameter
M = 0.15 One set of trading signals, the localmodel average of the neutral line
F
< Π0 > and the uncertainty band multiplied by the optimization factor
M = 0.15, and centered around the theoretical zero mean of the CMI, is
represented versus time. Note entry points in a short trading position (< 26 Lester Ingber et Radu P Mondescu
Canonical Momenta Indicators
time resolution = 55 secs 1
F < Π 0>
M ∆Π F CMI 0.5 0 0.5 1 0622 12:56:53 0622 13:56:30
TIME (mmdd hhmmss) 0622 14:56:07 Fig. 1.6. CMI trading
signals, realtime trading
June 22: (dashed line)
— localmodel average
of the neutral line; (dotted line) — uncertainty
band multiplied by the
optimization parameter
M = 0.15 F
Π0 > > M ∆Π F ) at around 10:41 (Fig. 5 in conjunction with S&P data
in Fig. 1) with a possible exit at 11:21 (or later), and a ﬁrst long entry
F
(< Π0 > < −M ∆Π F ) at 12:15. After 14:35, a stay long region appears
F
(< Π0 > < 0), which indicates correctly the price movement in Fig.1.
In Fig. 6 corresponding to June 22 price data from Fig. 2, a ﬁrst long
signal is generated at around 12:56 and a ﬁrst short signal is generated at
14:16 that reﬂects the long downtrend region in Fig. 2. Due to the averaging
process, a time lag is introduced, reﬂected by the long signal at 12:56 in Fig.
4, related to a past upward trend seen in Fig. 2; yet the neutral line relaxes
rather rapidly (given the 55 sec time resolution and the window of 88 ≈ 1.5
hour) toward the uncertainty band. A judicious choice of trading rules, or
avoiding standard averaging methods, helps in controlling this lag problem.
Recall that the trading rules presented are symmetric (the long and short
entry/exit signals are controlled by the same M factor), and we apply a staylong condition if the neutralline is below the average momentum < Π F >= 0
F
and stayshort if < Π0 >> 0. The drift f F and volatility coeﬃcient σ are
reﬁtted adaptively and the exponent x is ﬁxed to the value obtained in the
training set. Typical values are f F ∈ ±[0.003 : 0.05], x ∈ ±[0.01 : 0.03].
During the local ﬁt, due to the shorter time scale involved, the drift may
increase by a factor of ten, and σ ∈ [0.01 : 1.2].
We note that the most robust optimization factors — in terms of maximum cumulative proﬁt resulted for all test sets — do not correspond to the
maximum proﬁt in the training sets: For the localmodel rules, the optimum
parameters are {55, 88, 0.15}, and for the multiple models rules the optimum
set is {45, 72, 0.2}, both realized by a fourdays training set from the March
2000 miniS&P contract [1.3].
Other observations are that, for the data presented here, the multiplemodels averages trading rules consistently performed better and are more
robust than the localmodel averages trading rules. The number of trades is 1. Automated Internet Trading 27 similar, varying between 15 and 35 (eliminating cumulative values smaller
than 10 trades), and the time scale of the local ﬁt is rather long in the 30
mins to 1.5 hour range. In the current setup, this extended time scale implies
that is advisable to deploy this system as a traderassisted tool.
An important factor is the average length of the trades. For the type of
rules presented in this work, this length is of several minutes, up to one hour,
as the time scale of the local ﬁt window mentioned above suggested.
Related to the length of a trade is the length of a winning long/short trade
in comparison to a losing long/short trade. Our experience indicates that a
ratio of 2:1 between the length of a winning trade and the length of a losing
trade is desirable for a reliable trading system. Here, using the localmodel
trading rules seems to oﬀer an advantage, although this is not as clear as one
would expect.
More details regarding the data and results obtained with the trading
system are given in our work Ref. [1.3]. 1.7 CONCLUSIONS
1.7.1 Main Features
The main stages of building and testing this system were:
1. We developed a multivariate, nonlinear statistical mechanics model of
S&P futures and cash markets, based on a system of coupled stochastic
diﬀerential equations.
2. We constructed a twostage, recursive optimization procedure using
methods of ASA global optimization: An innershell extracts the characteristics of the stochastic price distribution and an outershell generates
the technical indicators and optimize the trading rules.
3. We trained the system on diﬀerent sets of data and retained the multiple minima generated (corresponding to the global maximum net proﬁt
realized and the neighboring proﬁt maxima).
4. We tested the system on outofsample data sets, searching for most robust optimization parameters to be used in realtime trading. Robustness
was estimated by the cumulative proﬁt/loss across diverse test sets, and
by testing the system against a bootstraptype reversal of trainingtesting
sets in the optimization cycle.
Modeling the market as a dynamical physical system makes possible a direct representation of empirical notions as market momentum in terms of
CMI derived naturally from our theoretical model. We have shown that
other physical concepts as the uncertainty principle may lead to quantitative signals (the momentum uncertainty band ∆Π F ) that captures
other aspects of market dynamics and which can be used in realtime
trading. 28 Lester Ingber et Radu P Mondescu 5. We presented and discussed the main aspects of developing an internetbased interface (API) for connecting a proprietary trading system to an
exchange.
1.7.2 Summary
We have presented an internetenabled trading system with its two components: the connection API and the computational trading engine.
The trading engine is composed of an outershell tradingrule model and
an innershell nonlinear stochastic dynamic model of the market of interest,
S&P500. The innershell is developed adhering to the mathematical physics
of multivariate nonlinear statistical mechanics, from which we develop indicators for the tradingrule model, i.e., canonical momenta indicators (CMI).
We have found that keeping our model faithful to the underlying mathematical physics is not a limiting constraint on proﬁtability of our system; quite
the contrary.
An important result of our work is that the ideas for our algorithms, and
the proper use of the mathematical physics faithful to these algorithms, must
be supplemented by many practical considerations en route to developing a
proﬁtable trading system. For example, since there is a subset of parameters,
e.g., time resolution parameters, shared by the inner and outershell models,
recursive optimization is used to get the best ﬁts to data, as well as developing multiple minima with approximate similar proﬁtability. The multiple
minima often have additional features requiring consideration for realtime
trading, e.g., more trades per day increasing robustness of the system, etc.
The nonlinear stochastic nature of our data required a robust global optimization algorithm. The output of these parameters from these training sets
were then applied to testing sets on outofsample data. The best models and
parameters were then used in realtime by traders, further testing the models
as a precursor to eventual deployment in automated electronic trading.
We have used methods of statistical mechanics to develop our innershell
model of market dynamics and a heuristic AI type model for our outershell tradingrule model, but there are many other candidate (quasi)global
algorithms for developing a cost function that can be used to ﬁt parameters
to data, e.g., neural nets, fractal scaling models, etc. To perform our ﬁts to
data, we selected an algorithm, Adaptive Simulated Annealing (ASA), that
we were familiar with, but there are several other candidate algorithms that
likely would suﬃce, e.g., genetic algorithms, tabu search, etc.
We have shown that a minimal set of trading signals (the CMI, the neutral
line representing the momentum of the trend of a given time window of data,
and the momentum uncertainty band) can generate a rich and robust set
of trading rules that identify proﬁtable domains of trading at various time
scales. This is a conﬁrmation of the hypothesis that markets are not eﬃcient,
as noted in other studies [1.62], [1.13], [1.39]. 1. Automated Internet Trading 29 1.7.3 Future Directions
Although this paper focused on trading of a single instrument, the futures
S&P 500, the code we have developed can accommodate trading on multiple
markets. For example, in the case of tickresolution coupled cash and futures
markets, which was previously prototyped for interday trading [1.38][1.39],
the utility of CMI stems from three directions:
(a) The innershell ﬁtting process requires a global optimization of all
parameters in both futures and cash markets.
(b) The CMI for futures contain, by our Lagrangian construction, the
coupling with the cash market through the oﬀdiagonal correlation terms of
the metric tensor. The correlation between the futures and cash markets is
explicitly present in all futures variables.
(c) The CMI of both markets can be used as complimentary technical
indicators for trading in futures market.
Several near term future directions are of interest:
– ﬁnalizing the productionlevel order execution API,
– orienting the system toward shorter trading time scales (1030 secs) more
suitable for electronic trading,
– introducing fast response “averaging” methods and time scale identiﬁers
(exponential smoothing, wavelets decomposition),
– identifying minicrashes points using renormalization group techniques,
– investigating the use of CMI in patternrecognition based trading rules,
– exploring the use of forecasted data evaluated from most probable transition path formalism.
1.7.4 Standard Disclaimer
We must emphasize that there are no claims that all results are positive or
that the present system is a safe source of riskless proﬁts. There as many
negative results as positive, and a lot of work is necessary to extract meaningful information.
ACKNOWLEDGMENTS
We thank Donald Wilson for his ﬁnancial support. We thank K.S. Balasubramaniam and Colleen Chen for their programming support and participation in formulating parts of our trading system. Data was extracted from
the DRW Reuters feed. 30 References References
1.1 E. Peters, Chaos and Order in the Capital Markets (Wiley & Sons, New
York,NY, 1991).
1.2 FIX Protocol Organization, URL http://www.ﬁxprotocol.org.
1.3 L. Ingber and R. P. Mondescu, IEEE Trans. Neural Net. 12 (4), 776 (2001),
URL http://www.ingber.com/markets01 optim trading.pdf.
1.4 E. Azoﬀ, Neural Network Time Series Forecasting of Financial Markets (Wiley
& Sons, New York,NY, 1994).
1.5 E. Gately, Neural Networks for ﬁnancial Forecasting (Wiley & Sons, New
York,NY, 1996).
1.6 L.
Ingber,
Physica
A
283,
529
(2000),
URL
http://www.ingber.com/markets00 highres.ps.gz.
1.7 B. Mandelbrot, Fractals and Scaling in Finance (SpringerVerlag, New York,
NY, 1997).
1.8 R. Mantegna and H. Stanley, Nature 383, 587 (1996).
1.9 L. Laloux, P. Cizeau, J.P. Bouchaud, and M. Potters, Phys. Rev. Lett. 83,
1467 (1999).
1.10 A. Johansen, D. Sornette, and O. Ledoit, J. Risk 1, 5 (1999).
1.11 K. Ilinsky and G. Kalinin, Technical Report No. http://xxx.lanl.gov/hepth/9712034, LANL (unpublished).
1.12 J. Hull, Options, Futures, and Other Derivatives, 4th Edition (Prentice Hall,
Upper Saddle River, NJ, 2000).
1.13 L.
Ingber,
Math.
Modelling
5,
343
(1984),
URL
http://www.ingber.com/markets84 statmech.ps.gz.
1.14 L. Ingber, Technical Report No. Global optimization Ccode,
Caltech Alumni Association, Pasadena, CA (unpublished), URL
http://www.ingber.com/#ASACODE.
1.15 L. Ingber, Mathl. Comput. Modelling 12, 967 (1989), URL
http://www.ingber.com/asa89 vfsr.ps.gz.
1.16 L. Ingber and B. Rosen, Oper. Res. Management Sci. 33, 523 (1993).
1.17 L. Ingber, Control and Cybernetics 25, 33 (1996a), Invited paper to Control
and Cybernetics on “Simulated Annealing Applied to Combinatorial Optimization.” URL http://www.ingber.com/asa96 lessons.ps.gz.
1.18 L. Ingber, Mathl. Comput. Modelling 18, 29 (1993b), URL
http://www.ingber.com/asa93 sapvt.ps.gz.
1.19 B. Cohen, Technical Report No. Training synaptic delays in a recurrent neural
network, TelAviv University, TelAviv, Israel (unpublished).
1.20 R. A. CozzioBu¨ler, The design of neural networks using a prior knowledge,
e
Ph.D. dissertation, Swiss Fed. Inst. Technol., Zurich, 1995.
1.21 G. Indiveri, G. Nateri, L. Raﬀo, and D. Caviglia, Technical Report No. Report,
University of Genova, Genova, Italy (unpublished).
1.22 D. Mayer et al., Proceedings Modelling, Simulation and Optimization Conference, International Association of Science and Technology for Development
(IASTED), 69 May 1996 Gold Coast (PUBLISHER, ADDRESS, 1996).
1.23 S. Sakata and H. White, Econometrica 66, 529 (1998).
1.24 G. Burghardt, Futures Industry February/March, 2001.
1.25 Archipelago, URL http://www.tradearca.com, 2001.
1.26 eSignal (Interactive Data Corporation), URL http://www.esignal.com, 2001.
1.27 CME Market Data API, URL http://www.cme.com/electronic/mdapi, 2001.
1.28 Eurex Values API, URL http://www.eurexchange.com/marketplace, 2001.
1.29 Rogue Wave Software, URL http://www.roguewave.com, 2001. References 31 1.30 E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements
of Reusable ObjectOriented Software
1.31 L. Bachelier, Annales de l’Ecole Normale Supe´ieure 17, 21 (1900).
r
1.32 M. C. Jensen, J. Finan. Econ. 6, 95 (1978).
1.33 B. B. Mandelbrot, Rev. Econ. Statist. 53, 225 (1971).
1.34 S. J. Taylor, J. Finan. Quant. Anal. 17, 37 (1982).
1.35 P. Brown, A. W. Kleidon, and T. A. Marsh, J. Fin. Econ. 12, 33 (1983).
1.36 G. Barabino, G. Barabino, B. Bianco, and M. Marchesi, Proc. IEEE Int. Conf.
Circuits and Computers (PUBLISHER, ADDRESS, 1980), pp. 1150–1153.
1.37 J. Nelder and R. Mead, Computer J. (UK) 7, 308 (1964).
1.38 L. Ingber, in Progress in Neural Information Processing, edited by S.I. Amari,
L. Xu, I. King, and K.S. Leung (Springer, New York, 1996b), pp. 777–784,
URL http://www.ingber.com/markets96 momenta.ps.gz.
1.39 L. Ingber, Mathl. Computer Modelling 23, 101 (1996c), URL
http://www.ingber.com/markets96 trading.ps.gz.
1.40 H. Haken, Synergetics, 3rd ed. (Springer, New York, 1983).
1.41 L.
Ingber,
Phys.
Rev.
A
44,
4017
(1991),
URL
http://www.ingber.com/smni91 eeg.ps.gz.
1.42 L. Ingber, M. Wehner, G. Jabbour, and T. Barnhill, Mathl. Comput. Modelling 15, 77 (1991), URL http://www.ingber.com/markets91 interest.ps.gz.
1.43 K. Cheng, J. Math. Phys. 13, 1723 (1972).
1.44 H. Dekker, Phys. Rev. A 19, 2102 (1979).
1.45 R. Graham, in Stochastic Processes in Nonequilibrium Systems, edited by L.
Garrido, P. Seglar, and P. Shepherd (Springer, New York, NY, 1978), pp.
82–138.
1.46 F. Langouche, D. Roekaerts, and E. Tirapegui, Phys. Rev. D 20, 419 (1979).
1.47 F. Langouche, D. Roekaerts, and E. Tirapegui, J. Phys. A 113, 449 (1980).
1.48 F. Langouche, D. Roekaerts, and E. Tirapegui, Functional Integration and
Semiclassical Expansions (Reidel, Dordrecht, The Netherlands, 1982).
1.49 H. Dekker, Phys. Lett. A 80, 99 (1980).
1.50 B. Oksendal, Stochastic Diﬀerential Equations (Springer, New York, NY,
1998).
1.51 J. Harrison and D. Kreps, J. Econ. Theory 20, 381 (1979).
1.52 S. Pliska, Introduction to Mathematical Finance (Blackwell, Oxford, UK,
1997).
1.53 C. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry and the
Natural Sciences (SpringerVerlag, Berlin, Germany, 1983).
1.54 R. C. Merton, Econometrica 41, 867 (1973).
1.55 L.
Ingber,
Phys.
Rev.
A
31,
1183
(1985),
URL
http://www.ingber.com/smni85 stm.ps.gz.
1.56 S. Kirkpatrick, J. C.D. Gelatt, and M. Vecchi, Science 220, 671 (1983).
1.57 S. Geman and D. Geman, IEEE Trans. Patt. Anal. Mac. Int. 6, 721 (1984).
1.58 H. Szu and R. Hartley, Phys. Lett. A 122, 157 (1987).
1.59 M. Wofsey, The Wall Street Journal 222, B1 (1993).
1.60 J. C. Cox and S. A. Ross, J. Fin. Econ. 3, 145 (1976).
1.61 P. Kaufman, Trading systems and Methods, 3rd edition (John Wiley & Sons,
New York, NY, 1998).
1.62 W. Brock, J. Lakonishok, and B. LeBaron, J. Finance 47, 1731 (1992). ...
View
Full
Document
This note was uploaded on 10/24/2011 for the course SCIENCE PHY 453 taught by Professor Barnard during the Winter '11 term at BYU.
 Winter '11
 BARNARD
 Physics, Electron, pH, The Land

Click to edit the document details