Generating Feature Spaces for Linear Algorithms with Regularized Sparse Kernel Slow Feature Analysis. Bohmer, Grunewalder, Nickisch, Obermayer. Machine Learning 2012.

  1. Deals with a way of automatically constructing nonlinear basis functions via SFA
  2. “Real-world time series can be complex, and current SFA algorithms are

    either not powerful enough or tend to over-fit. We make use of thekernel trickin combination with sparsificationto develop a kernelized SFA algorithm which provides a powerful

    function class for large data sets.”

  3. Also uses regularization to prevent overfitting on small data sets
  4. Hypothesize that “…our algorithm generates a feature space that resembles a Fourier basis in the unknown space of latent variables underlying a given real-world time series.
  5. Assume that solutions are defined on a low dimensional space of latent variables Θ which is embedded in the high dimensional space X that is actually observed
  6. Look for a feature space Φ s.t.
    1. For all i = {1,…p} φi, is non-linear in X in order to encode Θ as opposed to X
    2. For all = {1,…p} φi, is “… a well behaving functional basis in Θ, e.g. a Fourier basis
    3. Size of p is as low as possible to represent Θ
  7. Although there have been numerous studies highlighting its resemblance to biological sensor processing (…), the method has not yet found its way in the engineering community that focuses on the same problems. One of the reasons is undoubtedly the lack of an easily operated non-linear extension that is powerful enough to generate a Fourier basis.”
  8. Based on Kernel SFA

<Wordpress ate the rest of this post… Grr.>

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: