Solutions Review’s Premium Content Series is a collection of contributed articles written by industry experts in enterprise software categories. In this feature, DoiT International Machine Learning Architect Jared Burns offers a brief on getting the most from machine learning.
In an effort to become more data-driven, companies have been experimenting with machine learning (ML). A subset of artificial intelligence (AI), ML takes in historical data, identifies patterns and outputs new values, enabling applications to quickly predict outcomes without the need for detailed programming.
Unfortunately, many early efforts have failed to fully harness ML due to incomplete plans to action that data, threatening investments and undermining progress. In a survey of corporate executives by NewVantage Partners, a resounding 99 percent of respondents said their firms had active investments in big data and AI. Yet just 39 percent said they are managing data as an asset and only 24.4% claimed to have created a data culture within their firms.
The following approaches to ML can help companies develop more effective data strategies and best use ML to drive actionable data insights that will produce valuable output.
H0w to Get the Most from Machine Learning
Build a Strong Foundation
It was just a few years ago that companies began venturing into ML. However, many rushed in too quickly and did not have the foundational aspects of their ML lifecycle set up properly. With the AI hype in full force, the speed with which companies moved led to major mistakes and insufficient planning.
For instance, even when teams had the right skill sets in place, there would often be engineers working separately on the same data tasks. This could corrupt data, return inconsistent conclusions and waste investments. While a number of companies have found some technical success with ML, the business value was not nearly as strong as expected, and many are looking for a more practical approach to ML technology based on long term, sustainable AI infrastructures.
The first steps to any data analysis strategy are to create a plan and build a strong foundation. Teams need to understand what value they’re looking to extract, how much they can invest and the expected timeframe for results. They should also have ample checkpoints built in to ensure they’re getting accurate, valuable results.
The ability for teams to collaborate effectively is key to success. Strategies to foster this should include:
- Moving workloads to the cloud so they are easily accessible for all collaborators and can scale according to need.
- Using source control or GitHub to track changes and deploy to separate environments for testing and production.
- Adopting a pipeline approach so the whole process as well as the data can be versioned, tracked and cataloged after each step in the process.
- Achieving replicability so that others can rerun a program and produce the same result, determine how it was reached and easily understand any changes to the data.
Make the Most of Your Data
The types of data that companies can extract value from vary, and the ways they process and store the data varies too. These data types can include:
- Unstructured data, such as raw text input that needs to be analyzed for patterns;
- Images companies want to extract insights from; and
- Operational data or tabular data, including any kind of measurements.
Fortunately, with all the technology and resources that Google and AWS have invested in, it’s easy to dive into these areas. In fact, with just a few clicks, a model can be created to analyze and understand data, making it easy for a company with any level of sophistication to begin to parse.
For customers seeking a more customized approach, tools like Python can help build a bespoke ML system from scratch that better aligns with their workflow. Ideally, a company should be able to blend the two pathways, building custom models while enlisting pre-built models to pressure test algorithms for accuracy.
Strive for High Quality
Above all else, the quality of the data and model is paramount – high quality input results in high quality output. Frequent testing is vital to operating a strong ML system and extracting real value.
If you don’t have a high performing data engineering team, you’re not going to develop good models. Data needs to be well cataloged and transformed in a way that’s easy for ML teams to adopt and audit. Monitoring data quality that feeds ML algorithms is critical. All of the features that go into a model need to maintain a similar output in the real world versus what was trained. If the model is delivering output that is different from reality, then either it or the data is flawed.
Along with tracking data and features that go into a model, validating models is also an important aspect of training and using it in a production scenario. Validation refers to the process of confirming the model actually achieves its intended purpose. In most situations, this will involve confirmation that it is predictive under the conditions of its planned use. You may have times when models find unintended consequences if not well tested.
Find a Balance Between Short & Long-Term Value
An approach has to be balanced between short and long term value. Every time you train a model, you need to think about ROI and if it’s worth investing more resources. The goal is always to operationalize a model that can reteach itself on its own and deliver insights that pay for themselves.
Models can be set up to automatically train and deploy based on changes to incoming data, new model architectures, or on a scheduled cadence. The resulting scalability means you can adopt new use cases or drive something more accurate. However, automation is not a substitute for human oversight. Consistent evaluation is essential to understanding if it’s worth the investment and effort of maintaining. Also, model explainability is critical to understanding whether the model’s predictions are causing bias and introducing unintended consequences, which also needs to be incorporated into the ROI of training a model.
Some data streams are very active and require constant evaluation and maintenance. Others are less urgent and data can be processed at less frequent intervals. Determining frequency is a balance of risk (what are the consequences if the AI is making mistakes?), input speed (how often is new data coming in?), and cost/value (how much is an AI operation costing each time it runs and how valuable is the output?).
Pin Down Cloud Costs
Always keep in mind the cloud-cost element and understand each component of that cost (automated or custom).
It’s important to consider both storage of the data and training time/budget of the model, as training the model will burn through compute budget just to retrain. Also, if using an automated ML model, remember that it’s easy to “fall asleep at the wheel” and for costs to mount.
If customizing a model, you need to think about data transformations, training and deployment, as each of these have different volumes of spend. Does a particular use case require an online scenario (constant input/output of data) or a batch scenario (generating predictions nightly/weekly)? You might eventually even scale up to using Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs), which can get very expensive. Consider enlisting partners that can help pin down ML spending and control operations to ensure you don’t receive any surprise bills.
Lean on Resources and a Solid Team to Get Started
Blogs from Google and Amazon offer a number of ways to get a quick start. Google, for example, has a quick lab repository of public data to quickly train and deploy a model. Both providers also have training tools that can help get project leaders up to speed.
Next, it’s important to build a strong team of data engineers experienced in ML. Once this is in place, you’ll be able to lean on this unit to build an intuitive model accessible to the wider team. The time frame for getting an ML program started varies based on scale, type and quality of data. Companies well positioned with a smart plan can get one up and running within a matter of weeks.
Tools are available within both Google and Amazon to help explain data and pinpoint issues. This can enable a company to tweak and adjust a model accordingly. But ultimately, the model will only be as good as its data, which will require ongoing care and validation.
- Lessons Learned: Getting the Most from Machine Learning - June 16, 2023
Jared Burns
Jared works within the CRE team at DoiT International supporting machine learning and data analytics. His directives range from helping individual, localized ML software to collaborative projects stored in the cloud. Before joining DoiT, Jared was a senior data science engineer at Pythian.
