view · edit · attach · print · history

Financial - 2005-2008 dataset

Benchmark data

CN550 Spring 2008

Financial Prediction (2005-2008)

Description:

In this benchmark, classifiers learn to predict whether the price of the Dow Industrial Average will go up or go down during the month of March, based on features extracted from trading data of the previous few years. The training data consist of 10 daily features from June 2005 till February 2008 (688 days), and the testing data consists of features from March 2008.
Binary classification problem: Predict whether the fund will go UP or DOWN on the following day.
Custom investment strategies: Try to make as much money as possible!

Data:

Data: http://cns.bu.edu/cn550/file_repository/matlab/findata_v2.mat

Use of the data

Load the file using the following command: load findata_v2.mat
Values of the features for all the training days: fin2.train_data
Values of the up/down for all the training days (1 for up, 0 for down): fin2.train_labels
Closing values for the training days: fin2.train_change

Values of the features for all the testing days: fin2.test_data
Values of the up/down for all the testing days (1 for up, 0 for down): fin2.test_labels
Closing values for the testing days: fin2.test_change

Format for Financial results

March 2008 -- xxx trading days: xxx UP, xxx DOWN

Largest UP = +xxx, Largest DOWN = -xxx, Smallest Change = xxx

# Name System Parameters-Settings Strategy 1 Strategy 2 % Correct C-index Runtime Details Link
1 Best All predictions correct   1220.56   100 1.0 0   
2 Worst All predictions wrong   -1220.56   0 0 0   
3 All UP All predictions UP   212.38   xxx 0.5 0   
4 All DOWN All predictions DOWN   -212.38   xxx 0.5 0   
6 Your name system parameters, comments xxx   xxx xxx xxx Details-LINK?  
7 Cloud Backprop w/ momentum 50 hidden neurons, lr = 0.05, momentum = 0.9, max epochs 600 with 0.01 minimum required change 624.68   83.3 0.667 8.9s (train+test, just for this parameter set) Furthur experiments find unstability in the results due to the initilization. So the result here should be considered just as one possible result. I report all 5*5*5 PoC, Cindex and Money made in 3 figures. Please see the details
8 Rohit KNN K=21, cityblock metric, nearest neighbor consensus rule, cross-validated with training set (first 3/4 was training, last 1/4 was test) 828.24  70.83 0.8021 0.229 sec (training + test) Details
9 Rohit fuzzy ARTMAP learning rate = 1, baseline vigilance = 0.9   79.1667 0.8507 3.66 sec (training) + 0.2129 sec (test) Details
10 Melissa Backprop w/momentum learning rate = 0.40, momentum = 0.60 899.94 455.5 70.83 0.5000 442.62 seconds Details
11 Melissa PNN sigma = 0.62 602.46 455.5 66.67 0.3333 1.25 seconds Details
12 Neel Genetic Simplified Fuzzy ARTMAP population size = 80, generations = 2000 748.56   79.17 0.813 4 hours (training) 0.01 sec (test) Details
13 Roger Fuzzy ARTMAP vigilance=.01 725.76   66.67   5.825 
13 Roger PNN spread=.4 1094.4  83.33   .9907 sec 
15 Peifeng SVM Using Platt's SMO C = 100, Kernel = rbf(5), using Matlab with C++ mex 780.54 455.5 83.3 0.687 2.24sec (train) .0079sec (test) Details
15 Peifeng Fuzzy ARTMAP Learning rate = 1, vigilance = 0.4 725.76 455.5 66.67 0.44 2.73sec (train) .0297sec (test) Details
16 Joe Backprop Learning rate = .3, 200 epochs    33   54.57 sec (test + train)  
17 Neel Simplified Fuzzy ARTMAP baseline vigilance = 0.75 -924.74   16.67 0.313 11.84 seconds (training) 0.05 sec (test) Details
18 Cloud Fuzzy KNN k=9 L2 635.96   83.33   0.0143 sec  
19 Jeff SVM libSVM package, poly kernel (3rd), qp 503.36   62.5 .375 0.0143 sec 118.24 secs (train) .01 secs (test)
20 Jeff MADALINE alpha=.2, MRII 259.04   .5 .1875 0.0143 sec 9.16 secs (train) .015 secs (test)
21 Todd SVM libSVM with polynomial kernel (3rd order) 1102.4 6075.3 87.5 .9236 0.056s (trn) 0.005s (test) Details
22 Todd EBF Centers = 25 568.4 -933.31 54.1667 0.6250 2.03s (trn) 0.055s (test) Details
23 Sean Fuzzy ARTMAP beta = 1, rho = 0.9 476.67 455.5 79.17   4.95 sec (train); 0.18 sec (test) Details
24 Charles Cascade Correlation In-house code, 5 cascades, 1 epoch 659.38 455.5 82.41 0.6643 5.00 sec Details
25 Charles Maximum Likelihood In-house code, severely hacked (see details) 201.4 455.5 53.33 0.3357 2.15 sec Details
26 Sean KNN K=1 567.75   70.83 455.5 0.0498 sec Details
26 Sean Radial Basis Function RBF Nodes = 5; sigma = 0.7 206.21   62.75   0.063215 sec Details

Format for Financial confusion matrices

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP TP: true UP FP: false UP TP + FP
2 # predicted DOWN FN: false DOWN TN: true DOWN TN + FN
3 total xxx xxx xxx

1 - Cloud (Backprop w/ momentum)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 9 2 11
2 # predicted DOWN 3 10 13
3 total 12 12 24

2 - Rohit(KNN Using K=21, city block metric)

# FIN 2 - Rohit # actual UP # actual DOWN total
1 # predicted UP 9 3 12
2 # predicted DOWN 4 8 12
3 total 13 11 24

3 - Rohit(learning rate = 1, baseline vigilance = 0.9)

# FIN 2 - Rohit # actual UP # actual DOWN total
1 # predicted UP 10 2 12
2 # predicted DOWN 3 9 12
3 total 13 11 24

4 - Melissa - Backprop w/Momentum

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 9 4 13
2 # predicted DOWN 3 8 11
3 total 12 12 24

5 - Melissa - PNN

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 12 8 20
2 # predicted DOWN 0 4 4
3 total 12 12 24

6 - Neel - GSFAM

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 9 2 11
2 # predicted DOWN 3 10 13
3 total 12 12 24

7 - ROger - FUzzy ARtmap

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 8 4 12
2 # predicted DOWN 4 8 12
3 total 12 12 24

7 - ROger - PNN

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 11 3 14
2 # predicted DOWN 1 9 10
3 total 12 12 24

8 - Peifeng - SVM

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 11 3 14
2 # predicted DOWN 1 9 10
3 total 12 12 24

9 - Peifeng - Fuzzy ARTMAP

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 8 4 12
2 # predicted DOWN 4 8 12
3 total 12 12 24

10 - Joe - Backprop

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 8 7 15
2 # predicted DOWN 9 0 9
3 total 17 7 24

Neel - SFAM

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 0 5 5
2 # predicted DOWN 9 4 13
3 total 9 9 18

12 - Cloud (Fuzzy KNN)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 10 2 12
2 # predicted DOWN 2 10 12
3 total 12 12 24

19 - Jeff (SVM)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 6 3 9
2 # predicted DOWN 6 9 15
3 total 12 12 24

19 - Jeff (MADALINE)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 9 9 18
2 # predicted DOWN 3 3 9
3 total 12 12 24

21 - Todd (SVM)

# confusion matrix # actual positive (+) # actual negative (–) __total__
1 # predicted + 11 2 13
2 # predicted – 1 10 11
3 total 12 12 24

22 - Todd (Elliptical Basis Function)

# confusion matrix # actual positive (+) # actual negative (–) __total__
1 # predicted + 10 9 19
2 # predicted – 2 3 5
3 total 12 12 24

23 - Sean (Fuzzy ARTMAP)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 10 2 12
2 # predicted DOWN 3 9 12
3 total 13 11 24

24 - Charles (Cascade Correlation)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 10 6 16
2 # predicted DOWN 2 6 8
3 total 12 12 24

25 - Charles (Maximum Likelihood)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 6 4 10
2 # predicted DOWN 6 8 14
3 total 12 12 24

26 - Sean (KNN)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 6 6 12
2 # predicted DOWN 1 11 12
3 total 7 17 24

26 - Sean (Radial Basis Function)

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP 4 8 12
2 # predicted DOWN 5 7 12
3 total 9 15 24
view · edit · attach · print · history
Page last modified on September 19, 2008, at 04:18 PM