|Figure 1: The oldest learning institution |
in the world; University of Bologna.
What is a machine in machine learning
First of all here, machine does not mean a machine in conventional sense, but computational modules or set of instructions. It is called machine because of thermodynamics can be applied to analyse this computational modules, their algorithmic complexity can be map to an energy consumption and work production, recall Landauer Principle. Charles Bennett has an article on this principle, here.
Learning curve for machines
The concept of learning curve and effect of sample size in training machine learning models for both experienced and novice practitioners. It is often this type of analysis is omitted in producing supervised learning solutions. In the advent of deep learning architecture, multi-layered neural networks, this concept becomes more pronounced.
Quantifying learning curve lies in measuring performance over increasing experience. If the performance of the machine, or human, increases over experience we denote that learning is achieved. We distinguish good learner.
On the misconception of unsupervised learning
|Figure 2: Donald Webb, |
father of unsupervised
A generic misconception appear on what unsupervised learning means. Clustering, or categorising unlabelled data, is not learning in Ebbinghaus sense. The goodness of fit or cluster validation exercise do not account an increase in experience, at least this is not established in the literature, judging from cluster validation techniques, see, Jain-Dubes’s Algorithms for clustering data. Wikipedia defines unsupervised learning “inferring a function to describe hidden structure from “unlabeled” data”, this is a function inference is not learning,.
Then, what is unsupervised learning? It originates from Hebbian Theory from neuroscience that “Cells that fire together wire together”. This implies, unsupervised learning is about how information is encoded, not how it is labelled. One good practical model that could be classified as unsupervised learning, so called spiking network model.
The concept of learning from Machine Learning perspective is summarised in this post. In data science, it is a good practice to differentiate what we call learning and function inference/optimisation. Being attentive to this details would help us.