An intuitive way to visualize how SVD works

SVD is the core algorithm of Latent Semantic Analysis. I have demonstrated how to perform LSA/LSI using SVD in blog here. However, SVD itself is not an easy thing to understand. To fully understand how it works requires quite a lot of linear algebra knowledge. We all agree math is important but it is not necessary for everyone to understand the math behind SVD in order to use SVD. Is there an intuitive way to see how it works? I mean, without any equations and formulas? Yes, there is. Today, i am going to let you SEE how it works ( :

I randomly downloaded a free picture from internet. An adorable parrot bird.


In order to convert the image into a matrix that can be applied with SVD, i need to convert it into gray scale first, and then perform SVD. The original gray-scaled image's dimension is 2000 x 3000 (see below)

This is how it looks like after we only keep the most important 300 components (2000 x 300). It doesn't look it changes that much, right?


This is how it looks like at (2000 x 50). This definitely makes the image more smudged and we can easily spot the difference now. However we still can see it is a parrot without any difficulty.
This is how it looks like at (2000 x 10). It is a very harsh reduction. A lot of background and bird face information have lost during this transformation. Even though we still can tell it is a bird, we might not be able to tell it is a parrot. 

Conclusion

The main idea of LSI is to perform a dimension reduction on our documents features. The more we reduce, the more information we lose. However, getting rid of some unimportant information is not a bad thing at all, i.e. the 2000 x 50 image still tells us pretty much all we can tell from its original image, and the reduction will make future computation much easier. 

Comments

Post a Comment

Popular posts from this blog

Perform efficient Latent Semantic Index using Python

Image
First of all, Latent Semantic Index(LSI) and Latent Semantic Analysis(LSA) are interchangeable terms. In E-discovery world, we often use LSI but in other fields, researchers often use LSA.  Latent semantic analysis   is widely used in information retrieval to search for similar documents. In E-discovery doc review, finding similar documents can drastically decrease the number of documents case team have to review so it is a powerful and essential tool in today's E-discovery industry. LSI is the thing behind Relativity conceptual analytics .  What is LSI and the intuition The main idea of LSI can be generalized in one simple sentence: Words with similar meaning will occur in similar documents .  Each document can have several topics and several words together can be used to express a topic. In another word, each document consists of a mixture of topics, and each topic consists of a collection of words. A word can appear several topics but the mixtures are dif...
Read more

SVM

Image
SVM has been a quite hot topic in recent E-discovery scene. The reason is Relativity has introduced Active Learning which is based on SVM. Now everybody knows it works especially well on classification. However what on earth is SVM? This blog is meant to take you through it briefly so you can know something more than just how to set up an active learning project in Relativity (: Introduction First of all, SVM is a supervised learning algorithm in general, meaning it is often used to classify data which is fundamentally different from LSI. You will need labelled data to train SVM model to predict on unseen data. In a nutshell, SVM is trying to find the hyperplane that can maximize the separation margins. So in another word, there might be many hyperplanes that can separate your data, but SVM is trying to find the best one for you! Nearly all the SVM tutorials will explain what is hyperplane and what is the margins, so i wont discuss it much about them here. The basic SVM is linear ...
Read more

Open Source to E-discovery

Image
As a data scientist who is working in E-discovery industry, I am glad to be able to use many good tools to work with E-discovery data, i.e. Nuix for processing, Relativity for review and some analytics, and so on. However, the limitation for any hard-coded software is they do not always work well when data is messy and unorganized. Coupled with the high license fees and server fees, those tools are not always a good option. So, is there anything free that you can rely on to perform all the core functionalities that those big tools provide? Well, the answer is yes and no. Yes, there are numerous open source tools that can do what you are looking for, and No, you still need to pay money for hiring a data scientist but not pay money for software. To demonstrate an open source approach to E-discovery, I decide to start writing blogs about this topic. When it comes to data, Python is hands down a very good option. (The other equivalent is R). Python has a very complete and robust standar...
Read more