view · edit · attach · print · history

Financial dataset - 2005-2006 dataset

Benchmark data

CN550 Spring 2008

Format for Financial results

January 2006 -- 21 trading days: 13 UP, 8 DOWN

Largest UP = +1.3, Largest DOWN = -2.26, Smallest Change = +0.08

# Name System Parameters-Settings Strategy 1 Strategy 2 % Correct C-index Runtime Details Link
1 Best All predictions correct   +12.37   100 1.0 0   
2 Worst All predictions wrong   –12.37   0 0 0   
3 All UP All predictions UP   -0.03   62 0.5 0   
4 All DOWN All predictions DOWN   +0.3   38 0.5 0   
6 Peifeng Fuzzy ARTMAP learning rate = 1 baseline vigilance = 0.57 8.89 3.83 80.95 0.57 2.3673sec(train) .1995sec (test) Details  
7 Peifeng SVM Using Platt's SMO C = positive infinity, Kernel = polynomial:(x1*x2 +1)^0.2, using Matlab with C++ mex 0.97 3.83 61.90 0.289 .8679sec (train) .0035sec (test) Details 170 support vectors/237 inputs 
8 Jeff SVM libSVM package, poly kernel (3rd), qp .901   47.62 .202 .448sec (train) .005sec (test) Details
9 Jeff MADALINE MRII algorithm, learning rate =.6 -2.94   42.87 .173 15.39secs (train) .51secs (test) Details
10 Rohit KNN K=22, city block metric, cross-validated with training set (first 3/4 was training and last 1/4 was testing) 11   97.92 0.9327 0.2554 sec (training + testing) Details
11 Roger Fuzzy Artmap learning rate=1, baseline vigilance=.5    47.6   0.513 sec (training + testing) Details
12 Melissa Backprop w/Momentum learning rate = 0.40, momentum = 0.60 7.57 3.83 85.71 0.7692 154.27 seconds Details
13 Karthik MADALINE Inhouse code MRII,LMS,learning rate = 0.4 -2.94   46 0.2 Sweeps over different learning rates dont seem to have much of effect, again just guess work Results
14 Karthik ISOMAP & LTSA Modified Original Source Code. Complete Failure of both manifold Learning Algorithms      Proves my Hypothesis! Dont work with data if it is Noisy. Details in writeup
15 Rohit Fuzzy ARTMAP learning rate = 1, baseline vigilance = 0.9 3.57   61.9048 0.7788 6.0585 sec (train) + 0.6835 sec (test) Details
16 Melissa PNN sigma = 0.62 9.09 3.83 80.95 0.6731 1.25 seconds Details
17 Roger PNN spread =>1.5 12.37  66.67   .78 seconds Details
18 Neel Simplified Fuzzy ARTMAP baseline vigilance = 0.75 2.71   33 0.519 4.99 sec (train) + 0.07 sec (test)Details
19 Neel Genetic Simplified Fuzzy ARTMAP population size = 80, generations = 2000 -6.79   23.81 0.394 47 minutes (training) 0.01 seconds (test) Details
20 Joe Backprop learning rate = .3, epochs = 200    62   46.76 sec (train + test)  
21 Karthik SVM MATLAB svmclassify with RBF -0.03   67 .68 1sec (train) .5sec (test) Always predicts up Results  
21 Karthik SVM svmlite, polynomial kernels,sequential minimal optimization,still exploring possibilities 6.75  72 0.7  Just started tweaking svmlite, more results forthcoming Results 
22 Todd SVM libSVM with polynomial kernel (3rd order) 1.63 19.27 61.90 0.7115 0.087s (trn) 0.017s (test) Details
23 Todd EBF Centers = 5 0.03 -3.83 38.0952 0.6538 1.971s (trn) 0.939 (test) Details
24 Sean Fuzzy Artmap beta=1, rho=0.9 1.31 3.83 57.14   14.95 sec (train); 1.36 sec (test) Details
25 Charles Cascade Correlation In-house code, 5 cascades, 1 epoch 6.13 3.83 66.67 0.8942 24.19 Details
26 Charles Maximum Likelihood In-house code, severely hacked <see details> 3.27 3.83 52.38 0.7885 1.785 sec Details
27 Sean KNN K=3 1.38 3.83 61.9   0.03478 sec Details
27 Sean Radial Basis Function RBF Nodes = 5; sigma = 0.7 4.74   57.14   0.036621 sec Details

# Financial confusion matrix # actual UP # actual DOWN total
1 # predicted UP TP: true UP FP: false UP TP + FP
2 # predicted DOWN FN: false DOWN TN: true DOWN TN + FN
3 total 13 8 21

Best

# Financial - Best # actual UP # actual DOWN total
1 # predicted UP 13 0 13
2 # predicted DOWN 0 8 8
3 total 13 8 21

Worst

# Financial - Worst # actual UP # actual DOWN total
1 # predicted UP 0 8 8
2 # predicted DOWN 13 0 13
3 total 13 8 21  

All UP

# Financial - All UP # actual UP # actual DOWN total
1 # predicted UP 13 8 21
2 # predicted DOWN 0 0 0
3 total 13 8 21

All DOWN

# Financial - All DOWN # actual UP # actual DOWN total
1 # predicted UP 0 0 0
2 # predicted DOWN 13 8 21
3 total 13 8 21

8 -Jeff (SVM)

# FIN 8 - Jeff # actual positive (+) # actual negative (–) ____total____
1 # predicted + 7 5 12
2 # predicted – 6 3 9
3 total 13 8 21

9 -Jeff(MADALINE)

# FIN 9 - Jeff # actual positive (+) # actual negative (–) ____total____
1 # predicted + 6 3 9
2 # predicted – 7 5 12
3 total 13 8 21

6 -Peifeng(Fuzzy ARTMAP)

# FIN 6 - Peifeng # actual IN # actual OUT total
1 # predicted IN 12 3 15
2 # predicted OUT 1 5 6
3 total 13 8 21

7 -Peifeng(SVM Using Platt's SMO)

# FIN 7 - Peifeng # actual UP # actual DOWN total
1 # predicted UP 10 5 15
2 # predicted DOWN 3 3 6
3 total 13 8 21

10 - Rohit(KNN Using K=22, city block metric)

# FIN 10 - Rohit # actual UP # actual DOWN total
1 # predicted UP 5 3 8
2 # predicted DOWN 1 12 13
3 total 6 15 21

11 - Roger(Fuzzy Artmap)

# FIN 11 - Rohit # actual UP # actual DOWN total
1 # predicted UP 4 2 6
2 # predicted DOWN 9 6 15
3 total 13 8 21

12 - Melissa(Backprop with Momentum)

# confusion matrix # actual IN # actual OUT total
1 # predicted IN TP: 10 FP: 0 10
2 # predicted OUT FN: 3 TN: 8 11
3 total 13 8 21

13 - Rohit (Fuzzy ARTMAP)

# confusion matrix # actual IN # actual OUT total
1 # predicted IN TP: 1 FP: 7 8
2 # predicted OUT FN: 1 TN: 12 13
3 total 2 19 21

15 - Melissa(PNN)

# confusion matrix # actual IN # actual OUT total
1 # predicted IN TP: 10 FP: 1 11
2 # predicted OUT FN: 3 TN: 7 10
3 total 13 8 21

16- Roger(PNN)

# Financial - Best # actual UP # actual DOWN total
1 # predicted UP 11 5 16
2 # predicted DOWN 2 3 5
3 total 13 8 21

17 - Neel (Simplified Fuzzy ARTMAP)

# confusion matrix # actual IN # actual OUT total
1 # predicted IN TP: 6 FP: 5 11
2 # predicted OUT FN: 2 TN: 1 3
3 total 2 19 21

18 - Neel (Genetic Simplified Fuzzy ARTMAP)

# confusion matrix # actual IN # actual OUT total
1 # predicted IN TP: 1 FP: 4 5
2 # predicted OUT FN: 12 TN: 4 16
3 total 13 8 21

19 - Joe (Backprop)

# confusion matrix # actual IN # actual OUT total
1 # predicted IN TP: 13 FP: 8 21
2 # predicted OUT FN: 0 TN: 0 0
3 total 13 8 21

21 - Karthik(MADALINE)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 6 5 11
2 # predicted – 6 4 10
3 total 12 9 21

22 - Karthik(SVMClassify)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 10 4 14
2 # predicted – 3 4 7
3 total 13 8 21

23 - Karthik(SVMLite)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 11 3 14
2 # predicted – 2 5 7
3 total 13 8 21

22 - Todd (SVM)

# confusion matrix # actual positive (+) # actual negative (–) __total__
1 # predicted + 10 5 15
2 # predicted – 3 3 6
3 total 13 8 21

23 - Todd (Elliptical Basis Functions)

# confusion matrix # actual positive (+) # actual negative (–) __total__
1 # predicted + 0 0 0
2 # predicted – 13 8 21
3 total 13 8 21

24 - Sean (Fuzzy ARTMAP)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 4 4 8
2 # predicted – 5 8 13
3 total 9 12 21

25 Charles (Cascade Correlation)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 6 0 13
2 # predicted – 7 8 8
3 total 6 15 21

26 Charles (Fuzzy ARTMAP)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 3 0 3
2 # predicted – 10 8 18
3 total 13 8 21

27 Sean (KNN)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 3 5 8
2 # predicted – 3 10 13
3 total 6 15 21

27 Sean (Radial Basis Function)

# Confusion Matrix # actual positive (+) # actual negative (–) ____total____
1 # predicted + 7 1 8
2 # predicted - 2 11 13
3 total 9 12 21

CN550 Spring 2007

# SVM Name % Correct C-index Cost_function More details
1 Gaussian kSVM Jing Bai 42.86   1.73 Methods and Results
2 SVM (Platt) Andy 71.43   6.75 details
3 SVM (Osuna) Eugene     
4 SVM Paramesh 61.9   1.41 Results

# ARTMAP Name % Correct C-index Cost_function More details
1 Fuzzy ARTMAP Tim 67 0.47 -0.03 (max 6.39) Vigilance
2 Fuzzy ARTMAP Gary 76.19 0.58 +4.71 conf parameters
3 Fuzzy ARTMAP Yohan 66.67 0.47 +8.17 details
4 Distr ARTMAP Rob 61.9   need training data!  
5 Default ARTMAP Hee 66.66 0.46 +4.07 Details_DefA_finance
6 Default ARTMAP Paramesh 76.19 0.375 +3.13 Results
7 ARTMAP-IC Jeff     * implementation in progress

# KNN Name % Correct C-index Cost_function More details
1 KNN Andy 66.67 0.43 8.07 details
2 KNN Hee 76.19 0.57 +6.07 Details_KNN_finance

# Bayesian Est. Name % Correct C-index Cost_function More details
1 Bayesian Est. Jing Xia 71.43 0.48 +3.55 10 mixed Gaussians result
2 Bayesian Est. Tim 71.4 .42 5.71 PCA Variation Validation

# RBF Name % Correct C-index Cost_function More details
1 RBF Network Jing Bai 52.38   2.81 methods and Results
2 RBF Jing Xia 71.43 0.35 +1.29 method result

view · edit · attach · print · history
Page last modified on May 12, 2008, at 11:44 AM