Combining Market Depth and Tick Files

What is MARKET Depth ?? Market Depth is another name for Order book information. Order Book displays 5-10 levels of Bid and Offer Prices along with the volume sizes. this is very useful information for high frequency trading. Order Book tells us about what state the market is in. The amount of data is huge. Here I give some tips and useful information in how to combine the Order book Data with the Time & sales Tick Data.

CME usually provides 5 levels of information for futures contracts.
So, To construct a sequential price flow, we need to combine the Depth and Tick Data. By doing this, we can reconstruct the price action that has taken place in the market.

It is best to work with the Best Bid Best Offer and Time and sales data to reduce the memory requirement.

Always, Market Depth Updates first and then trade gets updated. If one is collecting the data, one needs to note down the Time stamps (upto msec level) and the corresponding Data.

For Example:

Depth Update 08:30:01.324 1925.50 1925.75
Trade Update 08:30:01.325 1925.75 500 (volume)

As you can see, one needs to see the time stamp for the last trade and see what was the best bid and best offer right before it and then combine the both into one row as follows:

08:30:01.325 1925.50 1925.75 1925.75 500

This gives a better picture of what really happened.

Intraday Trading---Converting Ticks into Bars

Sometimes it is easier to work with bars instead of ticks. In such cases, If we have Raw Tick Data, The following matlab programs would do the work for you. It is a collection of various methods that could be used to construct bars.

Tick2bar program at the matlab file exchange:



http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=3398

Another method that was posted at a forum http://www.nuclearphynance.com/


% numberOfBarsPerDay: 24 = 1 hour bars (60 minutes)
% 48 = 30 minute bars (24*60)/30
% 72 = 20 minute bars (24*60)/20
% 96 = 15 minute bars (24*60)/15
% 144 = 10 minute bars (24*60)/10
% 288 = 5 minute bars (24*60)/5
% 1440 = 1 minute bars (24*60)/1
%%--------------------------------------------------------------------------

numberOfBarsPerDay=24;
x=Bid;
datetimeGrid=(floor(datetime.*numberOfBarsPerDay))./numberOfBarsPerDay;
timeChgPointIndex=find(diff(datetimeGrid)~=0);
intervalDatetimeStart=datetimeGrid(timeChgPointIndex);
intervalDatetimeEnd=datetimeGrid(timeChgPointIndex+1);
intervalDatetimeActual=datetime(timeChgPointIndex);
intervalData=x(timeChgPointIndex);







3. Another method using histc
[numberInInterval,intervalIndex]=histc(datetime,intervals);

Creating Market profile with MATLAB

Some Day Traders use CBOT Market Profile for Day Trading purposes. I am NOT a fan of Technical Analysis, but nevertheless, I thought why not write a matlab program to create one.

Here is a quote from CBOT----

Market Profile is a graphical organization of price and time information. Market Profile displays price on the vertical axis and time on the horizontal axis. Letters are used to symbolize time brackets. marketprofile is an analytical decision support tool for traders—not a trading system.

CBOT Overview:

http://www.cbot.com/cbot/pub/page/0,3181,1168,00.html

CBOT Examples:

MATLAB CODE:

% clear all
% clc;
% %assuming tick by tick data
% data = load('may01d.mat');
% ESdata = data.ESlast;
ESdata=close;

ESprices=unique(ESdata);
bars=20;
timertick=1;
periods=floor(size(ESdata,1)/(timertick*bars));
MP=zeros(size(ESprices,1),periods);
for i=1:periods
g=ESdata(timertick*bars*(i-1)+1:timertick*bars*i,1);
for m=1:size(g,1)
s=g(m);
s1=find(ESprices==s);
MP(s1,i)=i;
end
end

B = MP;%A(:,2:end) ;
B(B==0) = inf ;
B = sort(B,2) ; % sort each row
B(isinf(B)) = 0 ;

TPO=zeros(size(B));
for l=1:numel(B)
if B(l)~=0
TPO(l)=B(l)+64;
end
end

Final_MP={ESprices char(TPO)};

Estimate Historical Volatility

Here is the MATLAB code that one could use to estimate historical volatility using different methods

Historical Close-to-Close volatility
Historical High Low Parkinson Volatility
Historical Garman Klass Volatility
Historical Garman Klass Volatility modified by Yang and Zhang
Historical Roger and Satchell Volatility
Historical Yang and Zhang Volatility
Average of all the historical volatilities calculated above

function vol = EstimateVol(O,H,L,C,n)
% Estimate Volatility using different methods
% EstimateVol(O,H,L,C)gives an estimate of volatility based on Open, High,
% Low, Close prices.
% INPUTS:
% O--Open Price
% H--High Price
% L--Low Price
% C--Close Price
% n--Number of historical days used in the volatility estimate
% OUTPUT:
% Vol is a structure with volatilities using different methods.
% hccv -- Historical Close-to-Close volatility
% hhlv -- Historical High Low Parkinson Volatility
% hgkv -- Historical Garman Klass Volatility
% hgkvM -- Historical Garman Klass Volatility modified by Yang and Zhang
% hrsv -- Historical Roger and Satchell Volatility
% hyzv -- Historical Yang and Zhang Volatility
% AVGV -- Average of all the historical volatilities calculated above
% web: http://www.sitmo.com
try
OHLC = [O H L C];
catch %#ok
error('O H L C must be of the same size');
rethrow(lasterror);
end
if(n<=length(O))
fh = @(x) x(length(x)-n+1:end);
else
error('n should be less than or equal to the length of the prices')
end
open = fh(O); %O(length(O)-n+1:end);
high = fh(H); %H(length(H)-n+1:end);
low = fh(L); %L(length(L)-n+1:end);
close = fh(C); %C(length(C)-n+1:end);
Z = 252; %Number of trading Days in a year

vol.hccv = hccv();
vol.hhlv = hhlv();
vol.hgkv = hgkv();
vol.hgkvm = hgkvM();
vol.hrsv = hrsv();
vol.hyzv = hyzv();
vol.AVGV = mean(cell2mat(struct2cell(vol)));

function vol1 = hccv()
% historical close to close volatility
%Historical volatility calculation using close-to-close prices.

r = log(close(2:end)./close(1:end-1));
rbar = mean(r);
vol1 = sqrt((Z/(n-2)) * sum((r - rbar).^2));
end

function vol2 = hhlv()
%The Parkinson formula for estimating the historical volatility of an
%underlying based on high and low prices.

vol2 = sqrt((Z/(4*n*log(2))) * sum((log(high./low)).^2));
end

function vol3 = hgkv()
% The Garman and Klass estimator for estimating historical volatility
% assumes Brownian motion with zero drift and no opening jumps
%(i.e. the opening = close of the previous period). This estimator is
% 7.4 times more efficient than the close-to-close estimator.

vol3 = sqrt((Z/n)* sum((0.5*(log(high./low)).^2) - (2*log(2) - 1).*(log(close./open)).^2));

end

function vol4 = hgkvM()
%Yang and Zhang derived an extension to the Garman Glass historical
%volatility estimator that allows for opening jumps. It assumes
%Brownian motion with zero drift. This is currently the preferred
%version of open-high-low-close volatility estimator for zero drift
%and has an efficiency of 8 times the classic close-to-close estimator.
%Note that when the drift is nonzero, but instead relative large to the
%volatility, this estimator will tend to overestimate the volatility.

vol4 = sqrt((Z/n)* sum((log(open(2:end)./close(1:end-1))).^2 + (0.5*(log(high(2:end)./low(2:end))).^2) - (2*log(2) - 1)*(log(close(2:end)./open(2:end))).^2));
end

function vol5 = hrsv()
%The Roger and Satchell historical volatility estimator allows for
%non-zero drift, but assumed no opening jump.

vol5 = sqrt((Z/n)*sum((log(high./close).*log(high./open)) + (log(low./close).*log(low./open))));
end

function vol6 = hyzv()
%Yang and Zhang were the first to derive an historical volatility
%estimator that has a minimum estimation error, is independent of
%the drift, and independent of opening gaps. This estimator is
%maximally 14 times more efficient than the close-to-close estimator.
%It can be interpreted as a weighted average of the Rogers and Satchell
%estimator, the close-open volatility and the open-close volatility.
%The performance degrades to the classic close-to-close estimator when
%the price process is heavily dominated by opening jumps.
muO = (1/n)*sum(log(open(2:end)./close(1:end-1)));
sigmaO = (Z/(n-1)) * sum((log(open(2:end)./close(1:end-1)) - muO).^2);
muC = (1/n)*sum(log(close./open));
sigmaC = (Z/(n-1)) * sum((log(close./open) - muC).^2);
sigmaRS = hrsv();
sigmaRS = sigmaRS^2;
k = 0.34/(1+((n+1)/(n-1)));

vol6 = sqrt(sigmaO^2+(k*sigmaC^2)+((1-k)*(sigmaRS)));

end
end

MATLAB2IB

MATLAB2IB is the product I created along with Exchange Systems Inc, which Connects to the JAVA TWS API from Interactive Brokers. it is socket based. It is much better than ActiveX based programming that IB also Provides. It is more efficient.

MATLAB2IB

provides all the information that you would need.

It includes all the commands that are neccessary for building an automated trading system in MATLAB. (ATS). This gives you a powerful combination of MATHEMATICAL software such as MATLAB and a cheap commissions based broker such as Interactive Brokers.

MATLAB has wide variety of toolboxes available that one could use in building a trading system.

There are a lot of freely available toolboxes .

File Exchange


Interactive Brokers:

InteractiveBrokers


Interactive Brokers API (Application Programming Interface) :

IB API