Data Science: PCA and Feature Selection

I'm working on my MS in Analytics and someone recently asked about using Principal Component Analysis (PCA) for feature selection.

Real life story: 40 features

• The team takes a 4-column 2MM sample data set and turns that into 40 features.

  • Feature engineering enables you to build more complex models than you could with only raw data. It also allows you to build interpretable models from any amount of data.

• What next? Maybe both.

What is Feature Selection and why do you do it?

• Feature selection will help you limit these features to a manageable number.

• Methods

  • Forward, backward or stepwise. Tiny Algorithm: Each feature must meet target criteria or be dropped.

  • Lasso, Ridge, Elastic Net. Deep divers only, check out https://towardsdatascience.com/whats-the-difference-between-linear-regression-lasso-ridge-and-elasticnet-8f997c60cf29.

What is PCA and why do you do it?

• The professor's classic answer - PCA is a dimensionality reduction tool.

• PCA may help with feature selection is if the most important variables also have the most variation.

• This is a great tool to observe trends, clusters, outliers and reduce data sets for exploration.

• Here's a great in-depth article if you want to dig into it, https://towardsdatascience.com/pca-is-not-feature-selection-3344fb764ae6.

Drop me a note and tell me how you do it.

-DCN