Your data is a gold mine

By Michael Schuldenfrei January 22, 2019

Companies often face manufacturing problems to which there is no readily apparent solution. When combining parts from different suppliers, both internal and external, how can they maintain the highest level of quality at all stages?
Better yet, when quality issues do arise, how can they report them both upstream and downstream? Can they see the manufacturing performance in real time? Or is the data available for analysis only later on in the production process?

For each of these questions, companies today need a clear answer. They need highly effective tools geared towards precise predictions and analysis to bring about a real increase in ROI.
Ideally, they need to be able to gather multiple forms of data at each stage of a manufacturing process, process this data in real time, and spot trends that can predict various outcomes. And when a possible outcome indicates a potential failure or low-quality component, a company needs to know the exact stage at which such a flaw was introduced so that it can be stopped and fixed.

Performing a Quality Analysis after a product is manufactured can lead to potentially huge losses. Even worse, if a fault is discovered only after a product has been purchased by the end-customer, damages to the brand may have an even more severe, and lasting, financial impact.
But when you identify and stop production issues at the earliest stage possible, so that a potential flaw is never introduced in the first place, you can save on costs as well as rest assured that the quality of your end product will be high and your company’s image will remain intact. This is the beauty of Big Data.

Your manufacturing data is precious. And mining this data to gather knowledge is equivalent to digging up gold. After all, what you uncover will be realized as a solid increase in profits, reduced costs and substantially improved bottom line. Read on to learn more about the basics behind achieving these goals.

Basic Concepts: Big Data Tools

If you take a look at a typical factory, you will see plenty of electronic equipment, each of which has several measurable properties such as power consumption, operating temperature, localization, MTBF, or error rate.
Additionally, test equipment generates reams of measurements and shop-floor systems can also provide important contextual information. Analyzing this data can help ensure the quality and efficiency of the machines and ensure they are performing as expected. But if you can also “connect the dots” and track the data generated as each manufactured product makes its way through the assembly line, you can know whether to trust the product or not.

To do this, you need to gather all of these measurements in a massive database. Even better, you can further use that information to help prevent some malfunctions before they ever appeared. This is what we call Big Data solutions—spotting patterns in huge amounts of data in order to continuously improve production and prevent potential problems as well as any corresponding losses in revenue.

Analyzing these data sets is a difficult and complex task but also a profitable one. Consider that every type of equipment in a factory is a potential data source storing a large amount of information that you would like to compare and correlate. You’re not looking to guess; you want to make clear and informed decisions.

Data Stores

As you can see, there can be more than one database in which to store your data. That’s why there are different terms to refer to them, depending on how they are used.

Data Lake

A data lake is a mass storage of mixed data.
This doesn’t mean it is completely chaotic, but the general idea behind this type of store is to enable high availability of data at a low cost. Unfortunately, despite their clear need for this type of feature, many companies have also found that building data lakes is both expensive and long-running, indicating a clear need for systems which, out-of-the-box and with minimal effort, enable companies to benefit from the tremendous volume and variety of data they can collect.
Ideally, systems need to be able to work with all of the data in your data lake to maximize its value.

Data Warehouse

Data lakes store both structured and unstructured data and are easy to write to, but reading usually requires further processing before the data can be consumed.
A data warehouse, on the other hand, stores every bit of information in a tidy manner, just like ordered products awaiting shipment in real-life warehouses.
Here, most of the processing is done when writing, allowing for quick and easy access to any higher-level information required.

Data Mart

A related concept is the data mart, typically the presentation layer of a data warehouse. This layer does not store data in any different way but rather focuses on a specific slice or view of the data in question.
So, your data warehouse may contain everything you can measure, while your data mart may narrow it down to specific fields for further analysis, such as power consumption, production yield, or even marketing.

How Much Data Do You Produce?

Imagine all of the useful data that surrounds you. Starting from the supply chain and going through assembly, test, and all the way up to customer satisfaction, each stage can provide you with helpful insights.
In order to fully manipulate this data and deliver analysis, predictions, and fault-prevention, Big Data solutions are a must.

The range of data available for analysis and performance optimization goes well beyond the manufacturing process. Purchase information, conversion analytics of your marketing campaigns, or geographical location of your customers are all highly relevant data and also subject to the same principles of Big Data.
It is important to understand that proper and effective data analysis across an entire organization is critical for it to reach optimal performance and results.
Data is everywhere—make sure you are accessing it, storing it, and gleaning information from it for the greatest reliability, visibility, and ROI possible.

Leveraging Big Data: A Complex Task Made Simple

Nowadays, more and more products are developed via multiple departments in one large enterprise or even by various external contractors. Managing such production can be very challenging as each production tier has lower-level tasks that demand specific solutions in order to avoid issues across the entire process.
In order to get a proper picture of the whole manufacturing process, you need to gather and process data from many different sources as well as transform this data and make it easy to understand.

This processing can occur in real time so that the data streaming and analysis take place simultaneously. When manufacturing electronic devices, it may be necessary to collect and compare data regarding all critical semiconductor components together with their Lot Code, Date Code, etc.
Another record may contain test results of delivered components, documentation of failure analysis down to its roots, customer complaints, and returns.

This is a lot of data, yes. And in addition to all of this, remember that it all comes from different sources varying in structure. It is almost impossible to make high-level decisions without unifying available data and conducting proper analysis.
You need the tools to perform these functions and spot any possible inefficiencies and failings within your manufacturing process early on in order to prevent unwanted product defects down the line.

You need to leverage your Big Data to uncover the golden nuggets of knowledge necessary to optimize your production process and bottom line.

About the author
Corporate Technology Fellow,  joined OptimalPlus in 2006, and brings over 30 years of software and information technology experience. Since joining the company he has served in several leadership roles including Chief Software Architect and CTO.