(Matlab) Codes and Datasets for Subspace Learning (Dimensionality Reduction)

We provide here some matlab codes of subspace learning algorithms, as well as some datasets in matlab format. All these codes and data sets are used in our experiments. The processed data in matlab format can only be used for non-commercial purpose.

If you have some problems or find some bugs in the codes, please email: dengcai2 AT cs DOT uiuc DOT edu

Learning algorithms

All the codes have been rewritten. You can download the old version at here .

Spectral Regression (SRLDA, SRKDA, RLPP...)
Spectral Regression provides an efficient subspace learning framework. I strongly recommend using SR instead of the following ordinary subspace learning algorithms.
Some general functions
- EuDist2: Calculate the Euclidean distance matrix of two data matrix.
- constructKernel: Construct the kernel matrix.
PCA: Principal Component Analysis
KPCA: Kernel Principal Component Analysis
LGE: (Regularized) Linear Graph Embedding (Provides a general framework for graph based subspace learning. This function will be called by LPP, NPE, IsoProjection, LSDA, MFA ...)
OLGE: (Regularized) Orthogonal Linear Graph Embedding (Provides a general framework for graph based subspace learning (orthogonal basis vectors). This function will be called by OLPP. It is also very easy to develop ONPE, OIsoProjection, OLSDA, OMFA ...)
TensorLGE: Tensor Linear Graph Embedding (Provides a general framework for graph based tensor subspace learning. This function will be called by TensorLPP. It is also very easy to develop TensorNPE, TensorIsoProjection, TensorLSDA, TensorMFA ...)
KGE: (Regularized) Kernel Graph Embedding (Provides a general framework for graph based kernel subspace learning. This function will be called by KernelLPP. It is also very easy to develop KernelNPE, KernelIsoProjection, KernelLSDA, KernelMFA ...)
LDA: (Regularized) Linear Discriminant Analysis (Generally, LDA can also use LGE as a subroutine. However, we can use the special graph structure of LDA to obtain some computational benefits.)
KDA: (Regularized) Kernel Discriminant Analysis (Generally, KDA can also use KGE as a subroutine. However, we can use the special graph structure of KDA to obtain some computational benefits.)
LPP: Locality Preserving Projection (You need to download LGE.m as well as constructW.m).
OLPP: Orthogonal Locality Preserving Projections (You need to download OLGE.m as well as constructW.m)
TensorLPP: Tensor Locality Preserving Projections (You need to download TensorLGE.m as well as constructW.m)
KernelLPP: Kernel Locality Preserving Projections (You need to download KGE.m as well as constructW.m)
constructW: Function used to construct the affinity matrix.
NPE: Neighborhood Preserving Embedding (You need to download LGE.m)
IsoProjection: Isometric Projection (You need to download LGE.m)
dijkstra.dll (for Windows)
dijkstra.mexglx (for Linux): dijkstra algorithm (You can download the source code at here)
LSDA: Locality Sensitive Discriminant Analysis (You need to download LGE.m)
MFA: Marginal Fisher Analysis (You need to download LGE.m)
Semi-Supervised Discriminant Analysis
- SDA: Semi-Supervised Discriminant Analysis (Graph Embedding Way)
- SDA_MR: Semi-Supervised Discriminant Analysis (Manifold Regularization Way)
LaplacianScore for feature selection
- LaplacianScore: LaplacianScore
Other functions:
- Clustering Evaluation: Evaluate the clustering result by accuracy and normalized mutual information
  Deng Cai, Xiaofei He, and Jiawei Han, "Document Clustering Using Locality Preserving Indexing", in TKDE, 2005.
- bestMap
- hungarian
- MutualInfo
  %===========================================
  fea = rand(50,70);
  gnd = [ones(10,1);ones(15,1)*2;ones(10,1)*3;ones(15,1)*4];
  
  res = kmeans(fea,4);
  
  res = bestMap(gnd,res);
  %============= evaluate AC: accuracy ==============
  AC = length(find(gnd == res))/length(gnd);
  %============= evaluate MIhat: nomalized mutual information =================
  MIhat = MutualInfo(gnd,res);
  %===========================================

Data sets in matlab format

Reproducing experimental results

Deng Cai, Xiaofei He, Wei Vivian Zhang, and Jiawei Han, "Regularized Locality Preserving Indexing via Spectral Regression", in CIKM'07, 2007. ( pdf )
Deng Cai, Xiaofei He, and Jiawei Han, "Spectral Regression for Efficient Regularized Subspace Learning", in ICCV'07, 2007. ( pdf )
%=========Unsupervised=====================
options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'KNN';
options.WeightMode = 'Cosine';
options.k = 5;
W = constructW(fea,options);

options = [];
options.W = W;
options.ReguType = 'Ridge';
options.ReguAlpha = 0.1;

[eigvector] = SR_caller(options, fea);

[nSmp,nFea] = size(fea);
if size(eigvector,1) == nFea + 1
    newfea = [fea ones(nSmp,1)]*eigvector;
else
    newfea = fea*eigvector;
end
%=========Supervised=====================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.gnd = gnd_Train;
options.ReguType = 'Ridge';
options.ReguAlpha = 0.1;

[eigvector] = SR_caller(options, fea_Train);

[nSmp,nFea] = size(fea);
if size(eigvector,1) == nFea + 1
    newfea = [fea ones(nSmp,1)]*eigvector;
else
    newfea = fea*eigvector;
end
%===========================================
D. Cai, X. He, Y. Hu, J. Han, and T. Huang, "Learning a Spatially Smooth Subspace for Face Recognition", in CVPR'07, 2007. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.Regu = 1;
options.ReguAlpha = 0.1;
options.ReguType = 'Custom';
load('TensorR_32x32.mat');
options.regularizerR = regularizerR;

[eigvector, eigvalue] = LPP(W, options, fea_Train);
[eigvector, eigvalue] = LDA(gnd_Train, options, fea_Train);
...

newfea = fea*eigvector;
%===========================================
D. Cai, X. He, J. Han, and H. Zhang, "Orthogonal Laplacianfaces for Face Recognition", IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3608-3614, November, 2006. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.Regu = 0;
options.PCARatio = 1;
bSuccess = 0;
while ~bSuccess
[eigvector, eigvalue, bSuccess] = OLPP(W, options, fea_Train);
end

newfea = fea*eigvector;
%===========================================
X. He, D. Cai, and P. Niyogi, "Tensor Subspace Analysis", in NIPS'05, 2005. ( pdf )
%===========================================
fea_Train = fea(trainIdx,:);
gnd_Train = gnd(trainIdx);

options = [];
options.Metric = 'Cosine';
options.NeighborMode = 'Supervised';
options.WeightMode = 'Cosine';
options.gnd = gnd_Train;
W = constructW(fea_Train,options);

options.nRepeat = 10;
fea_Train = reshape(fea_Train',32,32,size(fea_Train,1));
[U, V, eigvalue_U, eigvalue_V, posIdx] = TensorLPP(fea_Train, W, options);

[nSmp,nFea] = size(fea);
fea = reshape(fea',32,32,nSmp);
nRow = size(U,2);
nCol = size(V,2);
newfea = zeros(nRow,nCol,nSmp);
for i=1:nSmp
newfea(:,:,i) = U'*fea(:,:,i)*V;
end
newfea = reshape(newfea,nRow*nCol,nSmp)';
newfea = newfea(:,posIdx);
%===========================================

Return to Deng Cai's Homepage