Machine Learning provides us with tools to streamline business processes, improve decision-making and increase the accuracy of task solutions where humans are prone to error. All this, however, with the restriction that the required results do not have to be available immediately, because algorithms have to learn and the consumption of computing power can be immense. But not all machine learning applications and projects will be successful from the start. Many companies must first build the necessary end-to-end pipeline to support the continuous training of models.
The needs pyramid – prerequisites for the success of Data Science and Machine Learning
If you are not familiar with the Data Science pyramid of needs, Monica Rogati (former VP of Data at Jawbone and LinkedIn Data Scientist) describes the various requirements for success with Data Science and Machine Learning. Defining the basic technical and cultural requirements will probably take a long time. When it comes to the toolset, more comprehensive and “end-to-end” solutions such as Google’s Tensorflow and AWS Machine Learning will become attractive. However, it is likely that companies will have to compromise to adapt to their cloud strategy and use different tools until the market is mature.
Source: Hackernoon https://hackernoon.com/the-ai-hierarchy-of-needs-18f111fcc007
For the implementation of machine learning, organizational issues must first be clarified, as well as the recruitment or training of specialists. In addition, there are still many measures that need to be taken to maximize investment in machine learning and to quickly integrate functionality into front-ends and digital services.
So how can apps and digital services be developed today that will enable us to use machine learning most efficiently tomorrow? Machine Learning can be taken into account at the design stage when new digital services are created, so that implementation is easier and more effective later. With the ApiOmat platform, customers can quickly develop digital products and services that enable them to quickly take advantage of machine learning and at the same time develop digital solutions that impact their business today. This is our approach:
1. creation of minimal machine learning models
The goal of every digital solution is to offer the user added value. The aim should therefore be to create added value for the user with as little effort (time and resources) as possible. If a Minimum Viable Product (MVP) cannot generate added value for users without Machine Learning, it is likely that it will not generate added value with it.
If for some reason an MVP cannot add value without using some kind of AI or machine learning capability (e.g. image recognition), then the use of cloud APIs that cover the most basic functionality should be considered. Services from AWS, Microsoft Azure or Google may not provide the highest accuracy for your use case, but they should be sufficient to validate the use case and justify a larger investment in building and training your own machine learning model. There are also open source projects that you can use to get started with common machine learning tasks. Ultimately, these services or open source projects can fully meet the requirements.
With ApiOmat customers can quickly build new digital services for any front-end device. ApiOmat generates standardized REST APIs, provides project-specific SDKs for the most common frontend programming languages and offers a managed database. This allows our customers to focus on the added value for the users instead of reinventing standard components of each digital solution they develop.
2. collecting data and events
Since data is the digital currency of machine learning, it is important to ensure that your digital product collects everything that could be relevant to the application. A prime example that should be collected in any digital product is user analysis. If you are able to collect information about how users interact with your application, what actions certain user segments perform, then this is data that could potentially be used in the future to improve usability through machine learning.
Any analysis tools you use should not only collect data, but most importantly, they should be exportable in a structured, clean and standardized format. These user analyses will not be of much use if they cannot be used to train machine learning models. Also make sure that you have the ability to create custom in-app events. This will allow you to collect all the correct metrics specific to your application.
Another fact to consider is the data processing and its legal framework of the analytics solution. Many analytics tools are SaaS models and allow you to access your data but store and process it in locations where the legal framework is not always clear or compliant. Many ApiOmat customers appreciate the fact that they have complete control over their data and that it is not used or stored in a third party environment, but in their own instance.
3. aggregate and structure existing data
Because many companies already have a treasure trove of data packed in databases and other silos of corporate IT (CRMs, ERPs, etc.). Often the data is also distributed across different systems and to obtain the most complete data set it may be necessary to obtain data from multiple sources and orchestrate it as needed. This is a common implementation for our customers using ApiOmat and they also benefit from being able to add additional data to their data sets. Often this is a media file or simply information that was not stored in the past but is relevant to the current use case.
When integrating data sources with ApiOmat it is easy to aggregate and structure data from multiple sources. A popular example is the creation of a more complete view on the customer by combining data from multiple CRM or ERP systems. This way the data is almost ready for your machine learning pipeline. With ApiOmat you can easily add attributes if additional labels or results are to be written from your machine learning model. This way you can customize data sets without making changes in your existing or company wide IT systems.
4. marking and export of data
Since adding labels and attributes to your records is easy with ApiOmat, it is time to start with the actual labeling of the data. This can be done manually or automatically, based on the logic written in ApiOmat. No matter if the customer starts tagging his data in ApiOmat, he always has the possibility to export all data via our REST API.
Accessing the data through an API is essential as it allows you to automate the data export so that you can easily access the latest data set and use it to train new models. As ApiOmat customers have already integrated their data from multiple data sources or added additional attributes or data models to ApiOmat, the data is structured with labels and is ready for training. The extraction of specific data sets is also easily done using the ApiOmat and the supported query language.
5. use A/B tests when using new models
When deploying machine learning models, it is important that you do not assign 100% of your users to a new model, as errors or incorrect results would affect everyone. There are several approaches to implementing A/B testing, and ideally, it is then easy to revert to the old version or to roll out the new model to all your users.
Since ApiOmat supports modular backend development with advanced versioning controls, customers can deploy and run multiple versions of their models simultaneously. As ApiOmat is multi-client capable, each digital product can decide when and how to implement a new version of your machine learning service. They will be notified of the availability of new versions and can update their service or run the new service in parallel with the old service.
6. start collecting further data as soon as possible when needed
Data is the lifeblood of machine learning and your data can determine the success or failure of your machine learning activities. When creating and training models, access to additional attributes can be the key to improving the model. With ApiOmat, adding additional attributes to existing models or creating completely new data models takes only minutes thanks to API generation and the managed database provided by ApiOmat.
As both your machine learning initiative and your digital products grow, ApiOmat provides all the tools to quickly access more data from enterprise systems, cloud APIs or implement new functionality in the front-end to collect the required data.
Implement machine learning capable products with ApiOmat
So if you are planning to develop a digital product based on machine learning, start with an MVP and make sure you can deliver value to users. There’s nothing worse than making a huge investment in a digital product with – but also without – machine learning that offers no added value to users at all.
Together with our customers we have already developed machine learning products. Whether you’re just starting out and need advice on deploying a digital product or your existing machine learning models and want to accelerate the time to market of new digital products, contact us. We will be happy to discuss with you how Machine Learning and ApiOmat can impact your business.