There are many issues called "machine learning," but many of them can be addressed using spreadsheets, database queries, or a set of rules. It's all about making the right choice at the beginning. If you make the wrong choice, then it will cost you: building models for simple issues wastes money; creating rule lists for complicated issues results in broken rules each month.
Start With a Simple Test
To see if you can use a simple rule-based approach to your problem; ask yourself if there is an easy way to state the rules for making a decision. If someone can clearly describe how they make a decision in just a few steps, then go ahead and implement those steps.
For example, tax calculations and shipping fees do not require models. Machine learning is useful when there are too many rules, the rules are too vague, or the rules are too obscure to be written down manually. Examples include identifying fake reviews, determining which customers are likely to cancel their subscription, and interpreting handwritten forms.
While it's impossible to identify all of the indicators of a fake review, a machine learning model can learn the patterns of fake reviews based upon thousands of previous examples.
Five Indicators that Machine Learning is Applicable
- The pattern is present in your data; however, you cannot clearly define this pattern. Experts will know when there is an issue, however they will not be able to identify what the issue is. An experienced loan officer may state a loan "appears risky," yet he/she cannot explain why.
- You have historical data. In order to create a model, you must have previous examples of both successful and unsuccessful outcomes (e.g., past orders identified as either fraudulent or non-fraudulent).
- The process occurs frequently. A model provides value after running thousands of times. If a decision is made only twice per year, then it would be best to use a human.
- Errors are acceptable. Models are rarely 100% accurate, therefore the cost associated with making an incorrect decision must be reasonable.
- The environment changes. Rules become outdated quickly, whereas models can be updated using new data.
Five Indicators That Your Project May Not Be Suitable For Machine Learning
- Your dataset is extremely small.
- By law, the decision must be completely understandable and reproducible; a "black box" cannot be used.
- An incorrect answer could cause harm and there is no way for a person to check the results.
- There is an existing simple rule that works well enough.
- No one is responsible for the problem or the outcome. If you notice any combination of these five items, take time to consider whether you need to gather additional data before proceeding with machine learning. Or, perhaps, machine learning is just not the right approach for this particular problem.
Rules, Analytics, or Machine Learning?
It can be helpful to consider three tiers of decision making. Rules work well for decisions that are both clearly defined and consistent.
For example, if an order exceeds a certain value, then it should go through for approval. Or, if a customer resides in a specific region, they should be charged a specific tax. Rules are inexpensive, quick, and easy to understand. Analytics is appropriate for analyzing historical data.
For instance, which product was the top seller during the previous quarter? Which region had the highest number of returns? A dashboard or a simple query will provide the answers to these types of questions without requiring a model. Machine Learning is best suited for making predictions and judgments based on complex patterns.
For example, which customer is most likely to cancel their subscription next month? Which component is most likely to fail? Which document is a contract versus an invoice? Many teams immediately begin working at the highest tier (Machine Learning).
However, it is recommended to start at the lowest tier (Rules) and only progress to the next tier once the previous one is no longer effective.
Common Machine Learning Use Cases
There are several common ways that businesses use machine learning. Here are some of those ways:
- Stock and staffing demand forecasting
- Subscription/service churn prediction
- Payment/claim fraud detection
- Retail/media product recommendation
- Document classification and data extraction
- Quality inspection from images taken on a production line
- Machine/vehicle predictive maintenance
These types of problems can be solved using machine learning because there is a lot of historical data available and there are many repeated decisions. If your problem is similar to one of these, then it is likely that machine learning will be able to help solve it.
However, if your problem does not resemble any of these, then you should continue searching for other options before deciding to use machine learning.
Check Your Data Before Anything Else
The quality of your data will determine the success of your project much more than the algorithm you choose. Ask yourself three questions about your data:
- Do we have enough examples?
- Are they labeled correctly?
- Will future data be similar to past data?
If any of these questions are "no," then address them first. Common issues include missing fields, inconsistent labeling, and data spread across multiple systems. When a model is trained on poor-quality data, it will produce poor-quality results; however, it will do so at a faster rate. You should also find out who owns the data and if you can legally use it.
Customer data may have privacy requirements, and some of it cannot be used for training purposes. It is best to resolve this issue as soon as possible so that you do not run into problems later on.
Define What Success Looks Like
Before you begin working on your project, establish a target. This target could be hours saved, errors avoided, additional sales, or fewer cancellations. If there is no target set, then no one will be able to determine if the project was successful. Establish a Baseline as well.
The baseline represents how the process is currently performing using either the existing rules or the current manual processes. Only when a model performs better than this baseline by an amount sufficient to cover the cost of developing and operating the model does the model have value.
Prove it On a Small Scale
Prior to developing an entire system, conduct a pilot project. Select a single, specific issue; develop a simple model to address this issue; and then evaluate the performance of this model against your existing approach. You should use some type of measurable metric such as hours saved or errors avoided.
If a simple model cannot outperform the baseline, then a more complicated model is unlikely to do so either. This is a very valuable finding because it indicates that you should either discontinue the project or alter your course of action prior to investing significant resources.
Although a pilot project may ultimately be deemed "not worth it," it has still accomplished its objective.
Questions to Consider Prior to Launching
This short list can be used in a planning meeting.
- What decision do we want to improve? Who currently makes this decision?
- How many times is this decision made per month?
- What is the cost of an incorrect answer?
- Do we have examples from the past with known outcomes?
- Who will be responsible for the model once launched?
If the team cannot answer most of these, then the problem is not ready. That is okay. It is better to know now than to discover later in the process.
Plan for Life After Launch
As customers' behaviors change, models will drift. For example, a churn model based upon last year's customers may not be able to identify the patterns of this year's customers.
Therefore, someone must monitor accuracy, retrain the model using new data, and address any anomalies (edge cases) that occur within the model. Without monitoring, businesses can adapt Machine Learning development services that will slowly degrade over time without anyone realizing it. Establish a schedule for reviewing results and determine in advance at what point an inaccuracy will require retraining.
Additionally, ensure there is always a human involved in making decisions that have real-world implications; such as denying a claim or identifying a customer as fraudulent.
Final Thought
Machine learning should be used in those instances where there are actual patterns; there is data available to analyze these patterns; and the decisions based upon those analyses occur frequently. In all other cases, keep it as simple as possible. It's always better to use a simple rule that works than to try to create a complex model that nobody will trust.
First define your problem; then look at the data; then conduct a small pilot study; and finally make decisions based on what the numbers tell you. If you need another perspective on a specific problem, WeblineIndia provides AI and Machine Learning solutions which start with a feasibility study of this type.
