Trust, Security, and Human Factors in Agentic AI
Autonomous AI on the factory floor is not just a technical challenge — it’s a human one. Operators want to know they can trust the system. Executives need accountability. Security teams demand resilience. At the “Agentic AI in Manufacturing: From Copilots to Autonomous Systems” session during AI Frontiers 2025, organized by IIoT World, audience questions cut straight to these issues.
Human in the Loop, or Out of It?
Q: How to integrate agentic AI in decision-making workflows with a human in the loop?
Answer: A phased approach works best. Start with governance frameworks and sandbox environments such as digital twins. Keep humans involved early, using AI as an assistant, not a replacement. Over time, expand autonomy as operators build confidence.
Accountability in Case of Failure
Q: How do you handle accountability when autonomous AI systems make mistakes?
Answer: Accountability doesn’t disappear. Manufacturers must maintain audit trails, validation procedures, and explainability. If AI intervenes in production, decision-making must be transparent and traceable — so stakeholders can understand why a system acted the way it did.
Security and Trust in Manufacturing
Q: How to build and maintain trust and security in agentic AI in manufacturing?
Answer: Transparency is key. Communicate clearly that agents are augmentation tools, not replacements. Pair this with strong change management and operator training. Trust grows when workers see that agents reduce repetitive tasks and help them focus on higher-value work.
In regulated industries like pharma, compliance frameworks already exist. Extending them to agentic AI in manufacturing means focusing on validation, auditability, and governance. Security is non-negotiable — giving AI internet access without safeguards opens serious risks.
Operators as Partners, Not Competitors
Q: What is the most effective strategy to make operators see agents as partners, not threats?
Answer: Frame agentic AI explicitly as a tool for empowerment. Start with copilots handling routine tasks, then gradually expand autonomy while maintaining operator oversight. Involve experienced operators in rollouts, upskilling, and training programs so they remain central to the process.
What This Means for Manufacturers
The success of agentic AI in manufacturing depends as much on trust and governance as on data pipelines or infrastructure. Companies must:
- Keep humans in the loop early to build confidence.
- Ensure every AI action is explainable and auditable.
- Treat operators as partners through training and inclusion.
- Lock down security so autonomy doesn’t create new vulnerabilities.
Technology can be deployed quickly — but without operator trust, clear accountability, and robust safeguards, it won’t scale. For manufacturers of all sizes, trust and human factors are not side issues; they are central to making agentic AI cost-effective, safe, and sustainable.
Special thanks to the speakers contributing to these answers
- Madalina Ciortan, Director AI, OpsIT — AstraZeneca
- Sebastian Trolli, Research Manager & Head of Industrial Automation & Software — Frost & Sullivan
Related articles:
