(Matlab) Codes and Datasets for Subspace Learning (Dimensionality Reduction)

We provide here some matlab codes of subspace learning algorithms, as well as some datasets in matlab format. All these codes and data sets are used in our experiments. The processed data in matlab format can only be used for non-commercial purpose.

If you have some problems or find some bugs in the codes, please email: dengcai2 AT cs DOT uiuc DOT edu

If you find these algoirthms and data sets useful, we appreciate it very much if you can cite our following works:

@article{CHH08,
author = {Deng Cai and Xiaofei He and Jiawei Han},
title = {SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis},
journal = {IEEE Trans. on Knowl. and Data Eng.},
volume = {20},
number = {1},
year = {2008},
pages = {1--12}}

@Inproceedings{CHH07b,
author = "Deng Cai and Xiaofei He and Jiawei Han",
title = "Spectral Regression for Efficient Regularized Subspace Learning",
booktitle = "IEEE International Conference on Computer Vision (ICCV)",
year = "2007"}

@Inproceedings{CHH07a,
author = "Deng Cai and Xiaofei He and Jiawei Han",
title = "Semi-Supervised Discriminant Analysis",
booktitle = "IEEE International Conference on Computer Vision (ICCV)",
year = "2007"}

@Article{CHHZ06,
author = "Deng Cai and Xiaofei He and Jiawei Han and Hong-Jiang Zhang",
title = "Orthogonal Laplacianfaces for Face Recognition",
journal = "IEEE Trans. Image Processing",
year = 2006,
volume = 15,
number = 11,
pages= "3608-3614"}

@Article{HYHNZ05,
author = "Xiaofei He and Shuicheng Yan and Yuxiao Hu and Partha Niyogi and Hong-Jiang Zhang",
title = "Face Recognition Using Laplacianfaces",
journal = "IEEE Trans. Pattern Anal. Mach. Intelligence",
year = 2005,
volume = 27,
number = 3,
pages= "328-340"}

@Inproceedings{HN03,
author = "Xiaofei He and Partha Niyogi",
title = " Locality Preserving Projections",
booktitle = "Advances in Neural Information Processing Systems 16",
year = "2003"}

Learning algorithms

All the codes have been rewritten. You can download the old version at here .

Spectral Regression (SRLDA, SRKDA, RLPP...)
Spectral Regression provides an efficient subspace learning framework. I strongly recommend using SR instead of the following ordinary subspace learning algorithms.
Some general functions
- EuDist2: Calculate the Euclidean distance matrix of two data matrix.
- constructKernel: Construct the kernel matrix.
PCA: Principal Component Analysis
KPCA: Kernel Principal Component Analysis
LGE: (Regularized) Linear Graph Embedding (Provides a general framework for graph based subspace learning. This function will be called by LPP, NPE, IsoProjection, LSDA, MFA ...)
OLGE: (Regularized) Orthogonal Linear Graph Embedding (Provides a general framework for graph based subspace learning (orthogonal basis vectors). This function will be called by OLPP. It is also very easy to develop ONPE, OIsoProjection, OLSDA, OMFA ...)
TensorLGE: Tensor Linear Graph Embedding (Provides a general framework for graph based tensor subspace learning. This function will be called by TensorLPP. It is also very easy to develop TensorNPE, TensorIsoProjection, TensorLSDA, TensorMFA ...)
KGE: (Regularized) Kernel Graph Embedding (Provides a general framework for graph based kernel subspace learning. This function will be called by KernelLPP. It is also very easy to develop KernelNPE, KernelIsoProjection, KernelLSDA, KernelMFA ...)
LDA: (Regularized) Linear Discriminant Analysis (Generally, LDA can also use LGE as a subroutine. However, we can use the special graph structure of LDA to obtain some computational benefits.)
KDA: (Regularized) Kernel Discriminant Analysis (Generally, KDA can also use KGE as a subroutine. However, we can use the special graph structure of KDA to obtain some computational benefits.)
LPP: Locality Preserving Projection (You need to download LGE.m as well as constructW.m).
OLPP: Orthogonal Locality Preserving Projections (You need to download OLGE.m as well as constructW.m)
TensorLPP: Tensor Locality Preserving Projections (You need to download TensorLGE.m as well as constructW.m)
KernelLPP: Kernel Locality Preserving Projections (You need to download KGE.m as well as constructW.m)
constructW: Function used to construct the affinity matrix.
NPE: Neighborhood Preserving Embedding (You need to download LGE.m)
IsoProjection: Isometric Projection (You need to download LGE.m)
dijkstra.dll (for Windows)
dijkstra.mexglx (for Linux): dijkstra algorithm (You can download the source code at here)
LSDA: Locality Sensitive Discriminant Analysis (You need to download LGE.m)
Semi-Supervised Discriminant Analysis
- SDA: Semi-Supervised Discriminant Analysis (Graph Embedding Way)
- SDA_MR: Semi-Supervised Discriminant Analysis (Manifold Regularization Way)
LaplacianScore for feature selection
- LaplacianScore: LaplacianScore
Other functions:
- Clustering Evaluation: Evaluate the clustering result by accuracy and normalized mutual information
  Deng Cai, Xiaofei He, and Jiawei Han, "Document Clustering Using Locality Preserving Indexing", in TKDE, 2005.
- bestMap
- hungarian
- MutualInfo
  %===========================================
  fea = rand(50,70);
  gnd = [ones(10,1);ones(15,1)*2;ones(10,1)*3;ones(15,1)*4];
  
  res = kmeans(fea,4);
  
  res = bestMap(gnd,res);
  %============= evaluate AC: accuracy ==============
  AC = length(find(gnd == res))/length(gnd);
  %============= evaluate MIhat: nomalized mutual information =================
  MIhat = MutualInfo(gnd,res);
  %===========================================

Data sets in matlab format

Reproducing experimental results

Deng Cai, Xiaofei He, Wei Vivian Zhang, and Jiawei Han, "Regularized Locality Preserving Indexing via Spectral Regression", in CIKM'07, 2007. ( pdf )
Deng Cai, Xiaofei He, and Jiawei Han, "Spectral Regression for Efficient Regularized Subspace Learning", in ICCV'07, 2007. ( pdf )
%=========Unsupervised=====================
options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'KNN';
options.WeightMode = 'Cosine';
options.k = 5;
W = constructW(fea,options);

options = [];
options.W = W;
options.ReguType = 'Ridge';
options.ReguAlpha = 0.1;

[eigvector] = SR_caller(options, fea);

[nSmp,nFea] = size(fea);
if size(eigvector,1) == nFea + 1
    newfea = [fea ones(nSmp,1)]*eigvector;
else
    newfea = fea*eigvector;
end
%=========Supervised=====================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.gnd = gnd_Train;
options.ReguType = 'Ridge';
options.ReguAlpha = 0.1;

[eigvector] = SR_caller(options, fea_Train);

[nSmp,nFea] = size(fea);
if size(eigvector,1) == nFea + 1
    newfea = [fea ones(nSmp,1)]*eigvector;
else
    newfea = fea*eigvector;
end
%===========================================
D. Cai, X. He, Y. Hu, J. Han, and T. Huang, "Learning a Spatially Smooth Subspace for Face Recognition", in CVPR'07, 2007. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.Regu = 1;
options.ReguAlpha = 0.1;
options.ReguType = 'Custom';
load('TensorR_32x32.mat');
options.regularizerR = regularizerR;

[eigvector, eigvalue] = LPP(W, options, fea_Train);
[eigvector, eigvalue] = LDA(gnd_Train, options, fea_Train);
...

newfea = fea*eigvector;
%===========================================
D. Cai, X. He, J. Han, and H. Zhang, "Orthogonal Laplacianfaces for Face Recognition", IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3608-3614, November, 2006. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.Regu = 0;
options.PCARatio = 1;
bSuccess = 0;
while ~bSuccess
[eigvector, eigvalue, bSuccess] = OLPP(W, options, fea_Train);
end

newfea = fea*eigvector;
%===========================================
X. He, D. Cai, and P. Niyogi, "Tensor Subspace Analysis", in NIPS'05, 2005. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.nRepeat = 10;
fea_Train = reshape(fea_Train',32,32,size(fea_Train,1));
[U, V, eigvalue_U, eigvalue_V, posIdx] = TensorLPP(fea_Train, W, options);

[nSmp,nFea] = size(fea);
fea = reshape(fea',32,32,nSmp);
nRow = size(U,2);
nCol = size(V,2);
newfea = zeros(nRow,nCol,nSmp);
for i=1:nSmp
newfea(:,:,i) = U'*fea(:,:,i)*V;
end
newfea = reshape(newfea,nRow*nCol,nSmp)';
newfea = newfea(:,posIdx);
%===========================================

Return to Deng Cai's Homepage