Machine Learning
  • Introduction
  • Self LEarning
  • Why Statstics in ML Or Data Science
  • How important is interpretability for a model in Machine Learning?
  • What are the most important machine learning techniques to master at this time?
  • Learning
    • Supervised Learning
      • Evaluating supervised learning
        • K-fold cross validation
        • Using train/test to prevent overfitting of a polynomial regression
      • Regression
        • Linear regression
          • The ordinary least squares technique
          • The gradient descent technique
          • The co-efficient of determination or r-squared
            • Computing r-squared
            • Interpreting r-squared
          • Assumptions of linear regression
          • Steps applied in linear regression modeling
          • Evaluation Metrics Linear Regression
          • p-value
        • Ridge regression
        • Least absolute shrinkage and selection operator (lasso) Regression
        • Polynomial regression
        • Performance Metrics
        • Regularization parameters in linear regression and ridge/lasso regression
        • Comments
      • Classification
        • test
        • Logistic Regression
        • naïve Bayes
        • support vector machines (SVM)
        • decision trees
          • Split Candidates
          • Stopping conditions
          • Parameters
            • Non Tunable Or Specificable
            • Tunable
            • Stopping Parameters
        • Evaluation Metrics
      • Random Forest
        • Logistic Regression Versus Random Forest
        • Paramters
          • Non Tunable Parameters
          • Tunable
          • Stopping Param
        • Parameter Comparison of Decision Trees and Random Forests
        • Classification and Regression Trees (CART)
        • How random forest works
        • Terminologies related to random forest algorithm
        • Out-of-Bag Error
      • Decision Trees
        • Gini Index
    • Unsupervised learning
      • Clustering
        • test
        • KMeans Clustering
          • Params
          • Functions
        • Gaussian Mixture
          • Parameters
          • functions
    • Semi-supervised learning
    • Reinforcement learning
    • Learning Means What
    • Goal
    • evaluation metrics
      • Regression
        • MSE And Root Mean Squared Error (RMSE)
        • Mean Absolute Error (MAE)
      • Model Validation
        • test
      • The bias, variance, and regularization properties
        • Regularization
          • Ridge regression
        • Bias And Variance
      • The key metrics to focus
    • hyperparameters
  • Steps in machine learning model development and deployment
  • Statistical fundamentals and terminology
  • Statistics
    • Measuring Central Tendency
    • Probability
    • Standard Deviation , Variance
    • root mean squared error (RMSE)
    • mean Absolute Error
    • explained Variance
    • Coefficient of determination R2
    • Standard Error
    • Random Variable
      • Discrete
      • Continuous
    • Sample vs Population
    • Normal Distribution
    • Z Score
    • Percentile
    • Skewness and Kurtosis
    • Co-variance vs Correlation
    • Confusion matrix
    • References
    • Types of data
      • Numerical data
        • Discrete data
        • Continuous data
      • Categorical data
      • Ordinal data
    • Bias versus variance trade-off
  • Spark MLib
    • Data Types
      • Vector
      • LabeledPoint
      • Rating
      • Matrices
        • Local Matrix
        • Distributed matrix
          • RowMatrix
          • IndexedRowMatrix
          • CoordinateMatrix
          • BlockMatrix
    • Comparing algorithms supported by MLlib
      • Classification
    • When and why should you use MLlib (versus scikit-learn versus TensorFlow versus foo package)
    • Pipeline
    • References
    • Linear algebra in Spark
  • Terminology
  • Machine Learning Steps
    • test
  • Preprocessing and Feature selection techniues
  • The importance of variables feature selection/attribute selection
    • Feature Selection
      • forward selection
      • mixed selection or bidirectional elimination
      • backward selection or backward elimination
      • The key metrics to focus on
  • Feature engineering
  • Hyperplanes
  • cross-validation
  • Machine learning losses
  • When to stop tuning machine learning models
  • Train, validation, and test data
  • input data structure
  • Why are matrices/vectors used in machine learning/data analysis?
    • Linear Algebra
  • OverView
  • Data scaling and normalization
  • Questions
  • Which machine learning algorithm should I use?
Powered by GitBook
On this page

Was this helpful?

Why are matrices/vectors used in machine learning/data analysis?

Previousinput data structureNextLinear Algebra

Last updated 5 years ago

Was this helpful?

this question brings the usage of linear algebra

Another reason ... or maybe a practical example of the issue ... consider the neurons in a neural net and their corresponding weights. Many operations needed to train the network can be expressed through algebraic operations on the matrix of input feature values and the matrix of weights.

If one chooses an object oriented approach - just for example - instead of a matrix approach, in which the weights are instance variables inside an object, or instance variables inside a hierarchy of objects, then you may complicate the training of the network.

Highly optimized linear algebra libraries like BLAS and CUBLAS make the operations of Vector x Matrix or Matrix x Matrix operations extremely efficient versus piecewise operations on each element.

When I took Ng's ML class, I often first did the algorithms using standard nested for loops, then moved the solution to full matrix solutions. The performance differences on even these unoptimized solutions was dramatic. I recommend you try the equivalent in MatLab or Octave to see for yourself.

Add in the ability to move computations to GPUs via CUBLAS or the like and you have huge performance gains.

EDIT:

Here's a toy example which supposes inputting 5000 small images into the first layer of a neural net. The FOR loop implementation is very naive and does not implement any optimizations.

Computer are very adept at solving linear algebra problems efficiently, and writing algorithms in this framework (turning the calculus into linear algebra) allows for the exploitation of existing computational methods.

Because matrices/vectors are convenient for computer processing.

matrix operation is much faster than circulation.

Phil Glau has given a very good example!

A lot of ML algorithms rely heavily on linear algebra.

Linear regression, for example, is a linear algebra problem and can be solved purely by linear algebra.

Feed-forward neural networks: each layer has the form, whereis a matrix of weights. This not only allows for fast vectorized computation, but also makes backpropagation much easier.

Principal Component Analysis (PCA), probably the most popular dimensionality reduction technique, is pure linear algebra.

Collaborative filtering is a form of low-rank matrix factorization, a problem that comes up frequently in ML.

In fact, most ML models utilize a linear combination of features in some way. An understanding of linear algebra makes that much easier to deal with.

https://www.quora.com/Why-are-matrices-vectors-used-in-machine-learning-data-analysis
https://www.quora.com/How-is-linear-algebra-connected-with-machine-learning-and-big-data-analysis