**Machine Learning- Algorithm Summary**

There are a lot of algorithms, how to do a good job of grouping helps us better remember them, the author gives 2 algorithms grouping way:

• The first is a grouping of algorithms by the learning style. (learning by algorithm)

• The second is a grouping of algorithms by similarity in form or function (like grouping similar animals together). (by algorithmic function)

Here are two perspectives on machine learning algorithms.

**1. Supervised Learning (Supervised Learning)**

When the data of the input data set (which we call the training set) is labeled, such as good or bad labels, classification labels, etc., the prediction or classification model built from this data belongs to the supervised learning model.

**Classic questions: **classification and regression. (classification and regression)

**Classic algorithm:** Logistics Regression and The Back Propagation Neural Network. (Logic Regression Algorithm and BP Neural Network Algorithm)

**2. Unsupervised Learning (Unsupervised Learning)**

In contrast to supervised learning, the data in the training set is not labeled, which means that you need to extract their characteristic rules from this pile of unlabeled data, perhaps by systematically reducing redundancy through mathematical reasoning processes, or by organizing the data through data similarity.

**Classic questions: **clustering, dimensionality reduction and association rule learning. (clustering, dimensionality, rule learning)

**Classic algorithm:** The Apriori algorithm and k-Means. (This special term is not translated)

**3. Semi-Supervised Learning (semi-supervised learning)**

As the name implies, semi-supervised learning means that some of the training data are labeled, while some are not, in general, when the amount of training data is too small, the model effect obtained by supervised learning can not meet the needs, so the use of semi-supervised learning to enhance the effect.

Classic question:classification and regression.

Classic algorithms: semi-supervised SVM, Gauss model, KNN model

It is also a good way to distinguish algorithms according to their functional similarity, such as tree-based algorithms or neural network-based algorithms. So I think it would be better to understand these algorithms from this point of view.
Even if this is a good approach, it is by no means perfect, and there are algorithms that cannot be simply classified, such as Therning Vector Quantification (LVQ), which is both a neural network and a distance-based algorithm, so the following categorization is only suitable for most algorithms, but commonly used algorithms.

**1. Regression Algorithms (Regression Algorithm) **
Regression pays more attention to the relationship between arguments and affes, and models by measuring errors, and regression algorithm is a good application of mathematical statistics and is also included in statistical machine learning.
Common regression algorithms include:
• Ordinary Squares Regression (OLSR, normal least square regression)

• Linear Regression (Linear Regression)

• Logistics Regression (Logic Regression)

• Stepwise Regression (Progressive Return)

• Multivariate Adaptive Regression Splines (MARS, Multiple Adaptive Regression)

• Locally Estimated Scatterplot Smoothing (LOESS, local scatter smoothing estimate)

2. Instance-base Algorithms (distance-based algorithm)

Distance-based learning models are common in modeling training set data and comparing the distance from new data, which is measured in many ways, often euthon distance, Manhattan distance, and so on.

Common algorithms include:
• k-Nearest Neighbor (kNN)

• Learning Vector Quantization (LVQ, Learning Vector Quantification)

• Self-Organizing Map (SOM, Self-Organization Map)

• Locally Weighted Learning (LWL, Local weighted learning)

3. Generalization Algorithms (regular algorithm)
Normalization is an extension of another approach, usually regression, that punishes models based on their complexity and supports simpler models, which are also better at generalization capabilities.

Common normalization algorithms include:

• Ridge Regression (Ridge Regression Algorithm)

• Least Absolute Shrinkage and Select (LASSO algorithm, sparse constraint)

• Elastic Net (Elastic Network)

• Least-Angle Regression (LARS, Minimum Angle Regression Algorithm)

4. Decision Tree Algorithms (Decision Tree Algorithm)

Decision tree methods are modeled based on the actual values of attributes in the data, and decision trees are often trained to classify and regression problems, which are often fast and accurate and are the most popular in machine learning.

Common decision tree algorithms include:

• Classification and Regression Tree (CART, Classification Regression Tree Algorithm)

• Iterative Dichotomiser 3 (ID3)

• C4.5 and C5.0 (different versions)

• Chi-squared Automatic Interaction Detection (CHAID)

• Decision Stump (Decision Stump)

• MD5 (Message-Digest Algorithm, Message Digest Algorithm)

• Conditional Decision Trees (Conditional Decision Tree)

5. Bayesian Algorithms (Bayesian Algorithm)
Algorithms based on Bayes's theology, common language classification and regression problems.

Common Bayes algorithms include:

• Naive Bayes (Park Bayes)

• Gaussian Naive Bayes (Gossian Bayes)

• Multinomial Naive Bayes (Polynomial Plain Bayes)

• Averaged One-Dependence Estimators (AODE)

• Bayesian Belief Network (BBN, Bayesian Theology Network)

• Bayesian Network (BN, Bayesian Network)

6. Clustering Algorithms (Cluster Algorithms)

Cluster analysis, also known as group analysis, is a statistical analysis method for studying (samples or indicators) classification problems, and also an important algorithm for data mining.

Cluster analysis consists of several patterns, typically a vector of a measure, or a point in multi-dimensional space.

Clustering analysis is based on similarity, with more similarity between patterns in a cluster than between patterns that are not in the same cluster.

Common clustering algorithms include:
• k-Means。

• k-Medians。

• Expectation Maximization (EM, Expectation Maximization Algorithm, an iterative algorithm)

• Hierarchical Clustering (hierarchical clustering)

7. Association Rule Learning Algorithms (Association Rule Learning Algorithm)

The rules extracted by the Association Rule Learning Method best explain the relationships between variables in the data, and these rules can be exploited by organizations by finding important and commercially useful associations in large cubes.

The most common algorithms include:

• Apriori algorithm

• Eclat algorithm

8. Artificial Neural Network Algorithms (Artificial Neural Network Algorithms)

Artificial neural networks are models inspired by the structure and/or function of biological neural networks, which are a type of pattern matching that is commonly used for regression and classification problems, but are actually a huge sub-field that contains hundreds of algorithms and various types of problem types.

The most common algorithms include:

• Perceptron (Perceiver)

• Back-Propagation (Reverse Propagation)

• Hopfield Network (Hopfield Network)

• Radial Basis Function Network (RBFN, Radial Base Function Network)

9. Deep Learning Algorithms (Deep Learning Algorithm)

Deep learning methods are updates that take advantage of a large number of inexpensively calculated artificial neural networks, which are concerned with building larger and more complex neural networks, many of which, as mentioned above, are related to semi-supervised learning problems, where large data sets contain very little label data.

The most common algorithms include:

• Deep Boltzmann Machine (DBM)

• Deep Belief Networks (DBN)

• Convolutional Neural Network (CNN)

• Stacked Auto-Encoders

10. Dimensionality Reduction Algorithms (De-dimensional algorithm)

As with clustering methods, a decrease in dimensionality facilitates finding correlations to the data, but in this case, it is in an unsupervised manner, or to summarize or describe the data with less information.
Many of these methods can be used for classification and regression.

Common algorithms include:

• Principal Component Analysis (PCA)

• Principal Component Regression (PCR)

• Partial Least Squares Regression (PLSR)

• Sammon Mapping。

• Multidimensional Scaling (MDS)

• Projection Pursuit。

• Linear Discriminant Analysis (LDA)

• Mixture Discriminant Analysis (MDA)

• Quadratic Discriminant Analysis (QDA)

• Flexible Discriminant Analysis (FDA)

11. Ensemble Algorithms (Integrated Algorithms)

The integration approach is a model of several weaker models that are trained independently and whose predictions are to some extent combined to make overall predictions.
Such algorithms focus more on weak learners and how to combine them. This is a very powerful technology and therefore very popular.

Common algorithms include:

• Boosting。

• Bootstrapped Aggregation (Bagging)

• AdaBoost。

• Stacked Generalization (blending)

• Gradient Boosting Machines (GBM)

• Gradient Boosted Regression Trees (GBRT)

• Random Forest

12. Other Algorithms (Other Algorithms)
There are many algorithms that are not covered, and there are probably the following:

• Feature Selection Algorithms (Feature Selection Algorithm)

• Algorithm accuracy evaluation (algorithm accuracy estimation)

• Performance measures (effect assessment)

• Computational intelligence (evolutionary algorithms, etc.)

• Computer Vision (CV)

• Natural Language Processing (NLP)

• Recommender Systems。

• Reinforcement Learning。

• Graphical Models。

• And more...