How to Make Sense of Machine Data in Smart Factory 101

  /  Industrial IoT   /  Connected Industry   /  How to Make Sense of Machine Data in Smart Factory 101
big data in smart factory

How to Make Sense of Machine Data in Smart Factory 101

“Digital transformation” and “smart factory” are the terms that have been around for a while, but only advanced manufacturers apply them to practice. The idea of machine to machine communication has raised the initial excitement and high hopes but now owners of mid-size and small factories ask: “What’s next? How do I start adopting these technologies in my factory?”

While over 98% of business leaders understand the concept of the Internet of Things, many are not sure what IoT means for their business, according to an Aruba Networks research. This results in IoT not being implemented properly or being used only at a basic level instead of fully transforming processes.

However, when we look at production plants that have grasped the essence of the Industrial Internet of Things, it really boils down to the communication ability between the shop floor and business environment. In practice, it covers how machine data is transferred from equipment to standard business applications in the back office – ERP, MES, CRM systems; what conclusions are drawn from these data and how they are applied back to the production process. Overall, the success of digital transformation depends on the capability of an organization to make sense of numbers from the machines and turn them into a competitive advantage.

Emerging challenges for business

Charles Kettering, the famed inventor and head of research for GM, said “a problem well-stated is half-solved.” A winding road to the Smart Factory poses several emerging and existing challenges for companies, and manufacturers need to answer the following questions to understand how to run a digital business model at their production facilities.

Why do you need a Smart Factory?

First and foremost, machines can generate a lot of data but throwing numbers to the cloud does not automatically mean obtaining useful information out of it. You need to have a context.

Factory equipment can provide most accurate real-time information about production processes but how can people start taking advantage of it and apply the knowledge in their day-to-day job?

Is it going to be about maintenance – making sure that every part is up and running?

Or so-called predictive maintenance – resolving the breakdown before it occurs based on the aggregate data from many similar devices and early signs of the problem?

It can also be about the business side of the production: human operators making decisions based on the reports from the machines in the MES and ERP systems. Embracing Industry 4.0 dictates not only the equipment upgrade but an overall thinking paradigm shift.

Connecting across vertical silos

For a factory to be not “islands of things” but actually work along with the concept of the Internet of Things, production machines need to talk to each other and eventually to edge devices. Given the many systems involved, the question is how to set up such communication.

One of the challenges at a modern factory is that machines often communicate within individual vertical silos transferring data from a device to HMI/SCADA etc. where an operator keeps an eye at this particular part of the production. Managers have long known that valuable information resides in the systems that run their production facilities – but don’t have direct access to it. The problem has often been that both worlds operate in isolation, with little or no connection between them. However, if we are talking about blending data and carrying it across various departments, the factory data exchange infrastructure should be built differently.

Data travel to ERP

The next concern is data connectivity. How far down do you need to see in your production process? Is it only controllers, or you actually need the data from each sensor etc.? This will largely influence the amount of information you operate and the picture of your facility:

  • the whole plant in general,
  • an individual production line,
  • or even a particular device, how its every junction functioning – and where, for example, a breakdown is obstructing the production process.

Besides, putting data from the physical world into the cyberworld requires of planning of analytics. You need to understand if it is an operator who will use the data for achieving better local control, or the server automatically sends aggregate information up to the ERP or MES system. All in all, which numbers from the avalanche of data need to be sent forward for better analytics and planning at the factory level, and which data stay locally for quick control and decision making.

Sharing data across divisions and third parties is another important issue. At advanced factories, an ERP system can automatically order raw materials or inform the maintenance service about the need for a check-up, based on notifications coming from the production facilities. This information exchange can also work for the benefit of machine builders: based on automatic reports arriving from various factories, they plan maintenance, substitute details in advance and thus provide better customer service saving clients from the annoying and expensive downtime.

Machine data security

Finally, the ultimate issue for the “smart factory” is ensuring the security of open communication. Security occupies the first position among IoT developers’ concerns (31%), according to the 2019 report by Eclipse Foundation, with connectivity (21%) and data collection and analytics (19%) taking only the next two spots. Gartner reports that spending on IoT endpoint security solutions will increase to $631M in 2021. Opening up production facilities to the world requires corresponding protection measures.

From OPC to OPC Unified Architecture

Since late 90-s, production automation was built around the OPC server enabling data travel from a device to an operator’s computer in the control room whether the hardware and software were from the same provider or not: the OPC protocol standardized the communication and preconditioned a breakthrough in manufacture automation. OPC applied the Microsoft Distributed Component Object Model (DCOM), which is part of the Windows operating system. In the early 2000-s, the protocol launched industrial automation and enabled factories to transit to safer, more accurate and cost-effective production. In fact, many production plants around the world still rely on this technology.

However, soon after the software entered the market, the disadvantages of the OPC protocol have become clear. Firstly, DCOM makes OPC limited only to the Windows operating system. Next, experts complain about the difficulties in finding and fixing software failures, and the impossibility of exchanging data over the web. And finally, the OPC technology is not reliable in terms of information security.

This is why, in response to multiple requests from the industry, OPC Foundation created a new standard – OPC UA. It is based on the previous versions but without drawbacks and suggests a totally new approach to the machine to machine communication and industrial digital transformation.

Its new features are:

  • Platform independence. OPC UA doesn’t rely on OLE or DCOM technology from Microsoft anymore. You can launch it on any platform: Apple, Linux (JAVA) or Windows;
  • Easier configuration of data exchange between multiple computers. Instead of DCOM, use common data-exchange standards: TCP/IP, HTTP(S), and SOAP;
  • Easier data governance and maintenance. You can use structures or models, and group and give context to data tags or points. All of this provides much more flexibility and makes it possible to connect your factory across the departments, and not work only in vertical data silos;
  • Easy integration to SCADA. An OPC UA server can be a direct part of a PLC, which is being built in a device. It means, no need to buy and configure an OPC UA server;
  • Resource saving. With OPC UA, you don’t need many PCs that sit in the facility and translate between the systems;
  • Data encryption. OPC UA security is based on a multilayer concept covering application and transport layers. It contains authentication and authorization as well as encryption and data integrity by signing.
  • Flexibility of integration. UA APIs are available in several programming languages. Commercial SDK are available for C, C++, Java, and .NET. Open-source stacks are available at least for C, C++, Java, Javascript(node) and Python.
  • Cost and time effectiveness. Availability of OPC UA connectors which integrate your real-time machine data to CRM, ERP, BI, Exchange, document management systems like SharePoint for improved planning, maintenance, and boosted effectiveness of your production;
  • Connection to legacy infrastructure. Availability of mechanisms for migration from OPC to OPC UA; OPC UA wrappers.

How to start using machine data for digital transformation

Manufacturers call OPC Unified Architecture the first communication protocol that removes borders between the enterprise and the factory floor. But how to actually connect your production to the systems that help manage your company?

Customer Relationship Management (CRM), Enterprise Resource Planning (ERP), Manufacturing Execution System (MES), Document Management System (DMS), Business Intelligence (BI) system, warehouse management systems are the most common software that assists in managing a business. As many companies regardless of size use these applications in various combinations, it is safe to generalize that the life of a factory is planned, analyzed, predicted and reported in these systems. And being a fully automated smart factory means having accurate up-to-date information about the production process available anytime in these systems – for analysis, in-time alerts, maintenance notifications etc. depending on your priorities.

Integrating machine data in business systems can happen two ways: 1) via custom development through OPC UA APIs, and 2) via a ready-made OPC UA connector that links machine data to ERP, CRM systems etc.

In our opinion, the second scenario is better than custom programming. Firstly, you don’t waste time and money on expensive programmer’s labor, and secondly, don’t need to take care of further maintenance. A pre-built OPC UA connector is typically bug-free and ready to use from day one. It does require programming skills but very basic ones. Two-three hours are needed to understand the integration principles, and after that only build the logic for connecting your enterprise’s machine data to business systems.

From the device to the enterprise

Why do you need to connect your machine data to ERP, CRM, MES? An OPC-UA slogan is ‘from the device to the enterprise’, and in this case, “enterprise” means an intelligent enterprise resource planning system.

To make smart decisions, an intelligent ERP requires real-time IIoT data and analytics. They differentiate a company from the competition and constitute what is called a Smart Factory.

As ERP applications fulfill so many tasks: order management and processing, data collection, planning on an enterprise production level, business process management overall and in hooking up with other systems, timely performance of intelligent ERPs provides a lot of advantage.

When you need to speed up processes, reduce manual labor and errors, and move faster between steps, grabbing real-time data as close as possible to the source – from a particular production line or even a device, an integration of machine data via OPC UA to the ERP system is revolutionary.

Besides, such integration helps the culture shift we’ve discussed above, as employees adopt the new technology more willingly if it is presented via a familiar interface (and even is mobile-friendly!). Ensuring user adoption remains a big challenge in the IIoT, and presenting the data via the systems everyone knows and uses – ERP, CRM, MES etc. speeds up the transition to Industry 4.0.

Other components of an IIoT Smart Factory are predictive analytics, machine learning, blockchain report sealing, and semi-autonomous and autonomous decisions and actions. All of these can be a part of your ERP system which orders materials or takes shipments decisions based upon multiple sources and huge amounts of contextual fast data, analyzed and leveraged in real-time.


From being the Next Big Thing, IIoT has already turned into a demand for modern factories. To keep up with the competition, connecting your machine data to the management business systems is not optional anymore. OPC UA has been doing a good job in this area providing many more opportunities and security measures to open communication in a production facility, so it is time to reap the benefits of a connected smart factory.

More about: Security Architecture for Smart Factories


This article was written by Thomas Berndorfer, Founder and CEO of Connecting Software. A serial entrepreneur with over 20 years in IT, Thomas Berndorfer has extensive experience in relieving organizations from the pain of IT integration, synchronization and automation.