People tend to think of robots when they hear the words ’Machine Learning’, but machine learning is not just a futuristic fantasy – it’s already here. It can be hard to understand this concept as it sounds quite abstract. In this blog post we delve into the basics of machine learning and give some important context as to where it came from and how we already use it in day-to-day life.
What is Machine Learning?
To give a brief overview, machine learning (ML), is a subdomain of Artificial Intelligence and is about teaching machines how to learn from a set of data. When applied properly, the machines learn to characterise patterns automatically in order to make strategic decisions. Not only is it a science, but it is also an art.
Why we use Machine Learning
Businesses often utilise ML to increase efficiency, improve outcomes and to get a better picture of their future state. Machine learning is often associated with complexity and high-tech businesses, but it can be utilised in most industries with great success. Some common use cases include:
- Manufacturing & Logistics: to estimate raw materials and man hours to complete orders.
- Client Experience in eCommerce: to improve automatic product recommendations.
- Chatbots: to automate responses to clients based on previous conversations.
The beginning of Machine Learning
The first ML application that really became popular and took over the world back in the 1990s changed the lives of millions of people. In fact, you probably don’t realise but it’s likely something you use every day; it was the email spam filter.
It was quickly followed by hundreds of ML applications that now quietly power a multitude of products and features that you use regularly, from better recommendations to voice search.
Here is a definition from 1959 by machine learning’s creator Arthur Samuel: [Machine Learning is the] field of study that gives computers the ability to learn without being explicitly programmed.
Here is a more modern and technical definition: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. – 1997, Tom Mitchell – American computer scientist known for his contributions to the advancement of machine learning.
How Machine Learning works
Let’s say your spam filter is a Machine Learning program that can learn to flag spam given examples of spam emails (e.g. flagged by users) and examples of regular (nonspam, also called ‘ham’) emails. The examples that the system uses to learn are called the training set. Each training example is called a training instance. In this case, the task T is to flag spam for new emails, the experience E is the training data, and the performance measure P needs to be defined; for example, you can use the ratio of correctly classified emails. This particular performance measure is called accuracy and it is often used in classification tasks.
A spam filter based on Machine Learning techniques automatically learns which words and phrases are good predictors of spam by detecting unusually frequent patterns of words in the spam examples compared to the ham examples.
Machine Learning can help humans learn
ML algorithms can be inspected to see what they have learned. For instance, once the spam filter has been trained on enough spam, it can easily be inspected to reveal the list of words and combinations of words that it believes are the best predictors of spam. Sometimes this will reveal unsuspected correlations or new trends, and thereby lead to a better understanding of the problem.
Machine Learning is great for:
- Problems for which existing solutions require a lot of hand-tuning or long lists of rules: one Machine Learning algorithm can often simplify code and perform better.
- Complex problems for which there is no good solution at all using a traditional approach: the best Machine Learning techniques can find a solution.
- Demand forecasting: ML algorithms can predict the number of products, services, power, and more factors the industries need to know for effective supply chain management.
- Fraud Detection: ML algorithms can learn from patterns, they are very fast to adapt to changes and can quickly identify patterns of fraud transactions. This means ML models can identify suspicious customers and as the number of transactions has increased over the years as a result of credit/debit cards, smartphones and e-wallets, it is important to take every possible step to prevent fraud.
- Fluctuating environments for which a Machine Learning system can adapt to new data.
- Getting quality insights about complex problems and large amounts of data
Main Challenges of Machine Learning
Since the main task is to select a learning algorithm and train it on some data, the two things that can go wrong are ‘bad algorithm’ and ‘bad data’.
Let’s start with examples of bad data.
Insufficient Quantity of Training Data
It takes a lot of data for most Machine Learning algorithms to work properly. Even for very simple problems you typically need thousands of examples and for complex problems such as image or speech recognition, you may need millions of examples.
Poor-Quality Data
Obviously, if your training data is full of errors and outliers, it will make it harder for the system to detect the underlying patterns, so your system is less likely to perform well. It is often well worth the effort to spend time cleaning up your training data. The truth is most data scientists spend a significant part of their time doing just that.
Now that we have looked at examples of bad data, let’s look at a couple of examples of bad algorithms. Please note, an ML model is the program that is created or implemented to find patterns or make decisions.
- Overfitting: This means that the model performs well on the training data, but it does not work efficiently when it is applied on the testing data (the part of the dataset the model has not seen before)
- Underfitting the Training Data: Is the opposite of overfitting. It occurs when your model is too simple to learn the underlying structure of the data. For example, a linear model of life satisfaction is prone to underfit; reality is just more complex than the model, so its predictions are bound to be inaccurate, even on the training examples.
Summarising, the model needs to be neither too simple (in which case it will underfit) nor too complex (in which case it will overfit).
Another challenge is scaling ML models. In brief, this is implemented to handle massive data sets and perform many computations in a cost-effective and time-saving way.
Testing and Validating ML
The only way to know how well a model will generalise to new cases is to test it. One way to do this is to split the data into two sets: the training set and the test set. As these names imply, you train your model using the training set, and you test it using the test set. The error rate on new cases is called the generalisation error, and by evaluating your model on the test set, you get an estimate of this error. This value tells you how well your model will perform on instances it has never seen before. If the training error is low but the generalization error is high, it means that your model is overfitting the training data.
Examples of Machine Learning in Real Life
Facial recognition
ML plays a pivotal role in facial recognition as it does everything from unlocking smart devices to identifying criminals.
Assessing Creditworthiness
Assessing the credibility of students as well as first-time credit card applicants who often do not have a credit history is not as difficult as it used to be. ML considers other factors like an applicants’ current financial health and habits.
Chatbots
A chatbot is a program created with ML algorithms that simulates and processes human conversation, allowing humans to interact with digital devices as if they were communicating with a real person. They can understand customers, handle their requests, and have natural responses. Chatbots help businesses take customer engagement to the next level.
Conclusion
Machine learning is growing in importance due to the increasingly enormous volumes and variety of data and the access and affordability of computational power. These digital transformation factors make it possible to rapidly and automatically develop models that can quickly and accurately analyse extraordinarily large and complex data sets.
Now many industries are developing more robust models capable of analysing bigger and more complex data while delivering faster and more accurate results. ML tools enable organisations to move more quickly to identify profitable opportunities and potential risks.
Industries that depend on vast quantities of data—and need a system to analyse it efficiently and accurately, have embraced ML as the best way to build models, strategise and plan.
If you’re ready to dive into the world of machine learning, our team of expert consultants know the ins and outs and can help you reap the benefits quickly. Get in contact with us using the form below and see the difference ML can make.
Source: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition. by Aurélien Géron. Chapters 1 and 2
