How should manufacturers maintain AI consistency across a global factory network? When a model update resets the trust operators have built, do you interrupt automation to re-validate? Sanjay Mahato of Amazon Web Services and Tauseef Ahmed of Grayhawk Consulting Group answer four audience questions on consistency at scale, graduated trust management, sensor data validation, and predictive model verification asked during the “Physical AI in Production: The Data Infrastructure Gap Holding Industrial Operations Back” session at the Industrial AI Summit 2026.
How Do You Preserve Consistency as AI Evolves Across Sites?
Audience question:”Our experience is, just as AI can amplify productivity, it also amplifies mistakes and inconsistency. How do we preserve consistency, while accommodating rapid evolution across multiple communicating harnesses?”
Sanjay Mahato, Global Partner Development Specialist at Amazon Web Services:
AI is an amplifier; it amplifies good decisions and bad ones, at scale and at speed. The approach is to tackle it via a three-layered approach. First, the semantic layer we discussed in our panel is the consistency foundation. If “Compressor_01” means the same thing at every site and every data pipeline speaks the same language, you’ve eliminated the single largest source of inconsistency: meaning drift across systems. Second, treat AI deployments like software releases, not science experiments. Version control your models. Stage rollouts: validate at one site, measure for drift, then propagate. Volkswagen does exactly this on their Digital Production Platform – a solution proven in Wolfsburg doesn’t go live at 42 other sites until it’s validated against local conditions. Third, close the feedback loop. Every model in production needs continuous monitoring for output drift. When the world changes, new product, new supplier, new process, the model must be retrained, not just trusted. The principle is to evolve fast; however, propagation is slow and deliberate.
Tauseef Ahmed, Founder and CEO of Grayhawk Consulting Group:
From a manufacturing perspective, the key is to separate what must remain consistent from what is allowed to evolve locally. Common definitions, semantic models, interfaces, security requirements, and governance should be standardized across the enterprise. Models and applications can evolve more rapidly within those boundaries. I would also avoid propagating a change across the network simply because it worked at one site. Validate it under representative operating conditions, monitor the outcome, and then promote it through a controlled deployment pattern. The objective is not to eliminate change. It is to create an architecture and operating model where change can happen without introducing inconsistency across the factory network.
Must AI Re-Earn Trust After Every Update?
Standardizing definitions and staging rollouts addresses the consistency problem, but every significant update still resets the confidence operators have in the system.
Audience question:”The challenge with autonomy is that confidence in an automation harness gets reset with every new version of the harness, new underlying model, new agent logic. So trust in autonomy oscillates as underlying automation evolves, adapts. Do we interrupt automation in order to build empirical trust in the new version of a harness, with new tool interaction, new capability, new APIs?”
Sanjay Mahato, Global Partner Development Specialist at Amazon Web Services:
This is a sophisticated observation – and the honest answer is yes, you do interrupt. But you interrupt surgically, not indiscriminately. Think of it as graduated re-trust. When a new model version or agent logic is introduced, you don’t flip from autonomous back to fully manual. You drop the system back one trust level. If it was executing autonomously, it goes to recommend-only. If it was recommend-only, it goes to shadow mode – running in parallel, invisible to operators. You re-earn trust through empirical validation against the same production conditions. It is like beta testing, i.e., roll the new version onto one line, one shift, one site. Run it alongside the incumbent. Compare outputs. Only when the new version matches or exceeds the previous version’s performance do you promote it. The principle: trust is not binary, and it’s not permanent. It’s a spectrum the system earns incrementally, and must re-earn with every significant change. Build the re-trust protocol into your deployment pipeline, not as an afterthought.
Tauseef Ahmed, Founder and CEO of Grayhawk Consulting Group:
I would treat trust in autonomy as something that must be continuously earned, not granted permanently. A significant change in the underlying model, agent logic, available tools, or operating environment should trigger a reassessment of the level of autonomy. That does not necessarily mean shutting the system down completely. Depending on the risk, you might temporarily move from autonomous action back to recommendation or shadow mode while validating the new version under real operating conditions. The higher the consequence of the decision, the stronger the evidence required before restoring full autonomy. This goes back to the principle I mentioned during the panel: the level of governance should correspond to the consequence of the decision.
How Should AI Evaluate Sensor Trustworthiness?
Graduated trust protocols manage the risk of model changes, but models are only as reliable as the sensor data feeding them.
Audience question:”In machine autonomy, AI depends on sensory information that are valid. How do we make AI capable to evaluate if sensors can be trusted, the health of the data? Remember what happened to the Boeing MAX since the problems arose from faulty sensors and having the wrong view of reality where the situation is more than critical (take off needs very quick response).”
Tauseef Ahmed, Founder and CEO of Grayhawk Consulting Group:
Before an AI system can trust sensor data, the system should have ways to assess the health and plausibility of that data continuously. That can include checking for missing or stale values, sensor drift, impossible readings, disagreement between redundant sensors, sudden changes outside known operating limits, and whether the signal is consistent with related process variables. For higher-consequence applications, I would be very cautious about allowing a single sensor to become the sole source of truth. Redundancy, cross-validation, and fail-safe behavior become increasingly important as the consequence of an incorrect decision increases. Most importantly, uncertainty in the input should affect the level of autonomy. If the system cannot establish sufficient confidence in the data, it should degrade gracefully, escalate to a human, or move to a safe operating state rather than continue acting as though the inputs are trustworthy.
How Do You Validate AI Models Before Production?
Trustworthy sensor data provides the foundation; the next question is how to identify which predictive model deserves production authority.
Audience question:”What if we took some of the historical data we already have, along with real-time data at the edge, and used it to build a few different predictive models? We could run them against a selected period of historical data, see which model does the best job predicting the actual process, and then run the best one alongside the live data to see how well it performs in the real world.”
Sanjay Mahato, Global Partner Development Specialist at Amazon Web Services:
Take your historical data, train multiple models – maybe a physics-based model, a statistical model, and a deep learning model – and back-test them against a known period where you already know the actual outcomes. Measure which one best predicts what happened. That’s your champion. Then run that champion in ‘shadow-mode’ alongside live production data. It’s making predictions but not acting. You’re comparing its real-time predictions against actual outcomes – building empirical confidence before you ever give it authority. Two critical additions to make this work at scale. First, the historical data must be contextualized. If your training data doesn’t carry asset identity, operating conditions, and maintenance state, you’re back testing against noise, and the model that “wins” may have learned the wrong patterns. Second, define your success criteria before you start. What prediction accuracy earns production trust? 90%? 95%? And over what time horizon? Set the bar, then let the data decide which model clears it. This is how great teams de-risk AI deployments – prove it in the past, validate it in the present, then trust it with the future.
These questions were submitted by attendees of the “Physical AI in Production: The Data Infrastructure Gap Holding Industrial Operations Back” session at the Industrial AI Summit 2026. Answers provided by Sanjay Mahato of Amazon Web Services and Tauseef Ahmed of Grayhawk Consulting Group. The session also featured Ian Clark of InfluxData, and it was moderated by Gaspard Duthilleul of Galyos.
