2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》...

12 月 3 日,2023 IoTDB 用户大会在北京成功举行,收获强烈反响。本次峰会汇集了超 20 位大咖嘉宾带来工业互联网行业、技术、应用方向的精彩议题,多位学术泰斗、企业代表、开发者,深度分享了工业物联网时序数据库 IoTDB 的技术创新、应用效果,与各行业标杆用户的落地实践、解决方案,并共同探讨时序数据管理领域的行业趋势。

我们邀请到天谋科技欧洲研发负责人、ASF 董事会成员 Christofer Dutz 参加此次大会,并做主题报告——《纳其源:如何用 Apache PLC4X 构建极简工业数据采集》。以下为英文内容和中文翻译全文。

目录

Current State of OT Communication

OT (运营技术)通信现状

Introducing Apache PLC4X

Apache PLC4X 介绍

My Vision of PLC4X and IoTDB

PLC4X 与 IoTDB 集成愿景

很遗憾我不能亲自来到现场与大家见面,只能远程做这个报告,但我将尽力和大家分享最近令我十分感兴趣并且全力投入的事情。

我将探讨如何使用 Apache PLC4X 从工业硬件进行通用数据采集,那么这与 IoTDB 有什么关系呢?毕竟我们是在 IoTDB 用户大会。我想说的是,如果你能把数据存储下来,这当然很好,而如果你还能把数据读取出来,那就更好了。我认为如何读取数据仍然是工业自动化领域中最大、最主要的未解决问题之一。我们现在知道如何存储和处理大量数据,但获取它们仍然是一个严峻的问题

那么我将讲些什么呢?首先,我将为你们简要介绍工业运营技术领域通信的现状。之后,我将介绍 Apache PLC4X。也许你们中的一些人已经听说过它,但为了确保每个人都了解,并且都像我一样喜欢它,我还是要稍微聊聊 Apache PLC4X。最后,我还将分享我对如何将 PLC4X 和 IoTDB 结合起来,以解决真实工业问题的愿景。

I'm a bit sad that I'm not able to be with you right now, so I have to do this remotely, but I'll do my best to still get you on board with what currently really excites me.

So I'll be talking about universal data-acquisition from industrial hardware with Apache PLC4X, so what does that have to do with IoTDB? I mean we're at the IoTDB Summit. But the thing is, it's great if you can store data, but it's even better if you can actually get data, and that's what I think currently is still one of the biggest, mostly unsolved problems with industrial automation. We now know how to store and process huge amounts of data, but still, accessing them is quite a problem. 

So what will I be talking about? In general, first of all, I'm gonna give you a little overview of the current state of OT communication. After that, I'll be introducing Apache PLC4X. I would expect that the one or the other of you has already heard of it, but just to make sure that everybody has heard of it and everybody likes it as much as I do, I'll be talking a bit about Apache PLC4X. And in the end, I'm going to wrap up the session with my vision on how we can use PLC4X and IoTDB together to solve real industrial problems.

01

Current State of OT 

Communication

OT (运营技术)通信现状

第一部分,运营技术通信现状。对于那些不太熟悉这个领域的人来说,OT 类似于工业自动化领域的信息技术(IT)。在其他领域中,与 IT 相关的一切通常涉及软件、计算机和网络。类似地,在工业自动化领域中,OT 涉及到自动化硬件、自动化网络和自动化软件。概括而言,对于来自 IT 领域的我们来说,当我们希望与 OT 领域的设备进行通信时,问题就开始变得棘手了起来。

如果你相信工业界的说法,尤其是结合上个月我们在纽伦堡参加 SPS 智能生产解决方案工业展的经历,当你和与会人员沟通时,你可能会认为工业中的数据采集并不是问题,因为所有人都宣称 OPC-UA 已经解决了这个问题。

So, current state of OT communication. For those of you who are not that familiar with these topics, OT is sort of like the IT of the industrial automation. While in the rest of the world, everything with IT is sort of like software, computers, networks. Well, it's the same. OT is automation hardware, automation networks, and automation software. So in general, we currently come from the IT world, we want to communicate with devices from the OT world, and that's where it starts getting tricky.

If you believe the industry, especially last month, we were at the SPS fair in Nuremberg, and if you had a walk around, you would believe that this isn't actually a problem, because everybody was proclaiming that OPC-UA solved this problem. 

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第1张图片

但事实上,那只是市场宣传的一面,而如果你深入了解现实的话,可以参考 HMS 公司提供的数据,他们每年都会发布这样的图表。在过去的六七年里,我一直在演讲中使用他们的对比图。而总的来说,它们没有发生太大变化。

从图中可以看到,工业以太网的增长速度比现场总线快得多,因此可以认为,工业以太网就是 OT 通信的主,即 OT 领域的通信主要是基于普通网络基础设施的,而现场总线通常具有一些专有的有线连接。


除了应用方面,工业以太网比现场总线的增长速度快以外,你可能还会注意到一点,即图中并没有出现 OPC-UA,这是因为 OPC-UA 实际上还并没有被广泛地使用。我推测,OPC-UA 被统计在了“其他以太网连接”这个类别中。也许 OPC-UA 在这里的占比并不是很小,但它依旧是和 MQTT 等其他通讯协议共享这个市场份额。在 PLC4X 中,我们支持许多其他基于以太网的通信协议,它们都在这里共享这些很少的份额。

But the thing is, that's the marketing side of things, but if you have a deeper look at reality, and this is from a company called HMS, they publish this chart every year. So I've been using their comparison charts in my presentations for the last 6 or 7 years, and in general, they haven't been changing that much. 

The one thing you can see is that industrial ethernet is growing faster, a lot than the Fieldbus, so you can imagine that industrial ethernet is all the, let's say, OT communication based on normal network infrastructure. Fieldbus usually has some proprietary wire connections. 

While industrial ethernet is growing in comparison with Fieldbuses, one thing you should probably notice, there is one topic you can't see on this chart, and that's OPC-UA, and that's just because its adoption isn't actually that big. So right now, I would assume that OPC-UA is among others, all allocated in the “Other Ethernet”, which is not a small portion, but still, it's sharing its shares there with other protocols like MQTT. Well, in PLC4X we have loads of other ethernet-based protocols, so they're all sharing these very few shares down there.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第2张图片

为什么会这样?我认为 OPC-UA 在过去几年失去了很多信任。一个原因是它相当昂贵。实际中,你不得不用新设备替换故障或者过时的旧设备,而通常,支持 OPC-UA 部署的可编程逻辑控制器(PLC)设备要比不支持的贵一些。而且,如果你想使用 OPC-UA,你必须购买许可证。印象中我上次看到的支持 OPC-UA 部署的设备单价大约在 400 欧元。价格可能因厂商而异,但总的来说,即使只是为了部署在 PLC 设备上,你也必须付费。

但从我的角度来看,最糟糕的是它的性能非常差,你可以拥有一台性能非常强大的 PLC 设备,但只要并发很多简单的请求,它就可能被拖垮。或许可以举一个例子作对比,在一台西门子 S7 设备上,我们能够每 2 秒采集 2600 个数据点,而对 PLC 的负载几乎没有明显影响。然而,当使用 OPC-UA 时,只要每 2 秒采集 200 个数据点,你就可能让它完全宕机,这或许能让你对它的低性能有一些概念。

此外,因为引入 OPC-UA 而导致的新的兼容性问题也越来越常见。这个问题不是在通信层面,而是在更高的系统层级。因为 OPC-UA 引入了配套规范的概念,有点像某个特定领域设备的标准规范。但现在出现了关于哪个配套规范应该主导其他规范的争论,因此我们只是将问题挪到了一个更高的层级,并没有彻底解决。

上述这些原因导致了 OPC-UA 的采用率相对较低,与其所宣称的情况差距较大。我认为目前唯一可行的替代方案是 MQTT,但即便是 MQTT,仍然要求对设备进行改装

Why is that the case? From my perspective, OPC-UA has lost a lot of trust in the last few years. One reason is it's pretty expensive. You have to replace your hardware with new devices, and usually, these PLCs that support OPC-UA, they're a bit more expensive than the ones that don't. And if you want to use OPC-UA, you usually have to buy a license. I think the last number I had was like 400 Euros per device to enable OPC-UA. It may vary from vendor to vendor, but in general, you have to pay even if it's just for the PLC.

But from my perspective, the worst thing is it's really bad performance, so you can really have a very powerful PLC, and you can still get it to fall on its knees by just asking too many simple questions at a time. Maybe as a comparison on one Siemens S7 device, we were able to pull 2600 data points every 2 seconds without having a significant impact on the load of the PLC. Well, you can completely make it shut down with OPC-UA by just asking 200 data points every 2 seconds, so maybe that gives you a little bit impression on how low-performance it actually is. 

Also, another thing that's been coming up more and more is that it actually introduces new compatibility issues. Not on communication layers, but on higher levels, because OPC-UA also introduces a concept of a companion spec. It's sort of like a standard specification of, let's say, devices of a certain domain. But here, it feels like there are now fights on whose companion spec is the one to dominate the others, so we just lifted the things one level higher. 

All this causes for its adoption to be relatively low, compared with all the proclamations they do. And the only current serious alternative to it, I think, is MQTT, but even MQTT still requires you to retrofit your device.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第3张图片

那么如何解决这个问题呢?你可以在网关引入支持 OPC-UA 和 MQTT 的 PLC 设备,但仍然需要为它们支付一定费用。还有一些支持协议转换的网关,不过通常它们的价格也相对较高。你也可以购买一些商业驱动程序,并编写直接与工业硬件通信的软件,但我想上次我看到商业驱动程序的主要供应商,西门子 S7 设备在 Linux 节点上运行驱动程序的价格,大概是每个节点 5000 欧元,所以可以想象这个价格还在急剧上升,非常昂贵

2017 年我启动 PLC4X 项目时,虽然已经存在一些开源驱动程序,但它们通常有一些许可证方面的问题。这些程序大多采用了 GPL 许可证,这使得在商业应用中使用它们变得困难。此外,很多驱动程序的代码写得很糟糕,从有些程序的代码中你甚至可以看出,开发者是如何不经任何结构上的修改,而将 C 代码直接转换成 Java 的,而且多数驱动程序在当时(2017 年)就已经没人维护了。

So how to solve this problem? You could introduce new PLCs that use OPC-UA and MQTT as gateways, but, yet still have to pay a bit of money for them. There are a number of protocol conversion gateways. They also usually come with a quite steep price tags. You could buy some commercial drivers, and write software for directly communicating within the industrial hardware. But I think the last time I checked drivers from the major vendors of commercial drivers, for an S7 device to run on a Linux node, I think we were at something near 5000 Euros per node. So you can imagine that this price goes steeply up, so it's very expensive. 

Then, in 2017 when I started PLC4X, there were also some open-source drivers, but they were usually badly licensed. Most of them were a GPL license, which makes it difficult for using them in commercial settings. Most of them were also badly written. Some of them you could really see how they just took C code and converted that to Java without changing anything in the structure. And most of them were, even then, already no longer maintained.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第4张图片

02

Introducing Apache PLC4X

Apache PLC4X 介绍

为了解决这个问题,我有幸启动了一个新项目,那就是 Apache PLC4X。

PLC4X 是什么?我认为我们的项目描述是一个很好的介绍。PLC4X 是一组用一个共享 API 使用各类协议,与工业 PLC 通信的库。尤其是该描述的“共享 API”是 PLC4X 最与众不同的部分。因为在 PLC4X 出现之前,你需要为你使用的每个驱动程序都单独定义软件架构。而我们尝试构建类似于 JDBC 或 ODBC 的工具,其中你有一个共享 API,并且可以在各种不同的产品中使用。

In order to solve that problem, I had the joy and luck to be able to start a new project, and that was to become Apache PLC4X.

What is PLC4X? I think our project statement wraps it up quite nicely. PLC4X is a set of libraries for communicating with industrial programmable logic controllers using a variety of protocols but with a shared API. And especially the last part of that sentence is the one that actually makes quite a difference, the shared API. Because till then, you need to for every driver you use, it's like defining the structure of your software. And we try to build something similar to JDBC or ODBC, where you have a shared API, and you could use that with a variety of different products.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第5张图片

PLC4X 的核心概念是,当你编写应用程序时,只需使用 API 模块来编写软件。该模块定义了所有用于通信的数据类型和数据结构。对于要支持的每个协议,都有一个驱动程序实现相应的逻辑,并完全负责将该协议的 PLC 数据类型映射为标准数据类型。还有一些集成模块可用于将 PLC4X 集成到其他项目中,如 NiFi、Camel、StreamPipes 等,以及许多其他可用的项目。

之所以称为 PLC4X 而不是 PLC4J 是有原因的,即我们不仅仅是专注于编写 Java 驱动程序,因为我们知道在自动化行业,Java 并不是主导语言。理解一个驱动程序通常是比较困难的,因此我们构建了一个代码生成框架,使我们能够根据我们编写好的模板,用任何我们选择的语言编写驱动程序。目前可用的语言包括 Java、Go 和 C,而社区正努力开发 C#、Python 甚至 Rust 版本的驱动程序。这样做的目标是编写独立于使用的 PLC 的软件

The core concepts of PLC4X are that you write applications and you only use the API module for writing your software. That defines all the data types and the data structures for communicating. Then for each protocol that you want to support, a driver implements that logic, and it completely takes care of mapping the PLC data types of that particular protocol into the standard data types. There is a number of integration modules available that allow you to integrate PLC4X into other projects, such as NiFi, Camel, StreamPipes, but there is a huge number of other projects available.

It's called PLC4X and not PLC4J, and that's for a reason, because we're not only focusing on writing Java drivers, because we knew that in the automation industry, Java is not, let's say, the dominant language. And generally, understanding a driver is the tricky part, so we built a code generation framework that allows us to write drivers for any language that we write templates for. Right now, the usable versions are in Java, Go and C, while the community is working hard on some C#, Python, and even Rust versions of the driver. The goal of this is to write software that is almost independent of the actual PLC used.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第6张图片

PLC4X 支持哪些操作呢?首先,它支持发现功能。如果某些 PLC 或工业硬件、自动化硬件支持发现功能,我们的 Discover 模块肯定会捕捉到。一旦找到并连接到设备,你可能会想了解该设备提供了哪些数据,这就是 Browse API 的作用。现在你了解了设备上有哪些数据,可能会想要读取它,这可以通过 Read API 完成,或者你可能想用 Write API 进行写入。一些协议甚至支持订阅功能,因此你还可以订阅变量的变化情况,然后定期地或在变量变化时看到。

还有一个用扳手图标展示的 API,是我们目前正在努力开发的 Publish API。这通常用于 Fieldbus 协议,如 ProfiNet 或 EtherCAT,这些协议中的应用程序也会在一定的时间间隔上报值。不过这个 API 功能仍然正在开发中。

Which operations does PLC4X support? Currently, it supports discovery. Some PLCs or industrial hardware, automation hardware, support discovery, so if they support that, our Discover module definitely will pick them up. Once you've found a device and you've connected to the device, you sometimes want to know what data does this device provide, so that is provided by the Browse API. Now that you know what's on the device, you might want to read it. That's done with the Read API, or you want to write it with the Write API. Some protocols even support subscriptions, so you can also subscribe for variables, and then you get them in regular intervals or whenever they change.

And one API, that is indicated by the wrench there, that is what we are currently working on, is a Publish API. That's generally used for Fieldbus protocols such as ProfiNet or EtherCAT, where the application also emits values in certain intervals, but that's still something we're currently working on.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第7张图片

我提到 PLC4X 基于一些概念进行构建,其中之一就是协议,它通常涵盖并实现协议的一般逻辑。它不处理传输,因为我们有单独的传输的概念,但是协议本身处理序列化、反序列化、给定协议的模型类型等事务。它处理协议状态,如何编写标签地址或字段地址,还负责将使用 PLC 的数据类型映射为 PLC4X 的标准数据类型。

对于不熟悉工业协议的人来说,这些协议的复杂度非常不一样。例如 Modbus 协议,通常在为新语言实现驱动程序时,我们会首先使用它,因为它非常简单。

I mentioned PLC4X was built around a number of concepts, and one of them is that of a protocol, and that generally wraps and implements the general logic of a protocol. It doesn't handle transfer, because for that we have the concept of a transport. But the protocol itself, it handles serialization, deserialization, the model types for a given protocol. It handles the protocol state, how tag addresses or field addresses are written down. And it takes care of the mapping of the data types using the PLC to the PLC4X standard data types.

For those of you who are not familiar with the industrial protocols, there are, let's say, there's a really wide variety of complexities here. For example, if you take the Modbus protocol, that's usually one we use as first protocol to implement when implementing drivers for a new language, just because it's very, very simple.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第8张图片

为了让大家对工业协议的复杂性有一点了解,这里有一个简单的示例,展示了 Beckhoff ADS 协议的连接状态图。在黄色部分的一侧,表示连接建立的过程。你可以看到有许多状态和条件,决定了控制流的不同走向。这里面还有,比如红色代表读取,橙色代表写入,还有用于浏览、订阅等等的流程,可以看到这相当复杂。过去,通常你需要在程序中自己处理这种复杂的流程,但 PLC4X 完全替你处理了这一切。你只需要告诉它,比如“我想读取这个变量”,PLC4X 就会处理所有这个请求需要的复杂路径。

To give you a little impression on how complex things could be, here's a little demonstration on this state graph of a Beckhoff ADS protocol, of the connections. On the one side where the yellow thing is, that's the connection establishment. You can see there are a lot of states and conditions where the control flow takes a different path. But then we have, let's say, I think the red is for reading, the orange is for writing, and for browsing, and for subscriptions, and whatever. You can see it's quite complex, and usually in the past, you would had to have this complexity in your program, and that's what PLC4X takes away from you completely. You only need to say, "I want to read this variable", then PLC4X takes care of all the magic paths needed for that.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第9张图片

截至目前,我们已经支持了哪些协议呢?西门子 S7 协议绝对是第一个。Beckhoff ADS 也已经支持。Modbus 我们支持 TCP 和 Serial 两种方式,即 Modbus/TCP、Modbus/ASCII 和 Modbus/RTU。我们支持 EtherNet/IP,其中还包括 Logix 变体,它增加了一些仅在 Allen-Bradley 设备上可用的附加功能。此外,我们还支持在我们自己的驱动上实现 OPC-UA。我们正在努力开发 BacNet。KnxNet/IP 的支持已经相当稳定。Firmata 是一种在 Arduino 等设备上使用的非常简单的协议,我们也已支持。我们支持了 CAN 协议。你们有些人可能已经知道,去年或今年,我迅速实现了用于燃气轮机的 IEC-60870 协议的支持。目前我正在开发支持 ProfiNet。我的一位同事正在开发支持 C-Bus。Atlas Copco 集团的 Open-Protocol,过去称为 Torque-Tools,是一种用于汽车制造业的自动扳手的协议,我们也在开发中。我目前还在开发支持 Bosch Rexroth CtrlX 连接器。Emerson DeltaV 相对有点特殊,因为我已经没有可用的硬件继续开发,但实际上我们是第一批能够对 Emerson DeltaV 协议进行逆向工程的人。我还有一个协议在计划开发中,而且我真的希望随着最近的改变能够尽快实现,那就是新的西门子 S7 协议,但可能还需要几个月的时间,我才能开始着手处理。

So, which protocols do we support yet? Siemens S7, that was definitely the first. Beckhoff ADS, Modbus, we support in both TCP and Serial, so Modbus/TCP, Modbus/ASCII and Modbus/RTU are supported. We support EtherNet/IP, and here even the Logix variant that adds quite a number of additional features that are only available on Allen-Bradley devices. And we support OPC-UA with our own driver implementation. We're working hard on BacNet. KnxNet/IP is quite stable. Firmata, that is sort of a very simple protocol for using on Arduinos or something like that. We have an implementation for the CAN protocol. Some of you might know, last year or this year, I quickly implemented an IEC-60870 protocol for gas turbines. I'm currently working on ProfiNet. A colleague of mine is working on the C-Bus. Open-Protocol is from Atlas Copco. It was called Torque-Tools in the past. It's like a protocol for, let's say, automated wrenches, used in automotive and car manufacture. I'm also currently working on Bosch Rexroth CtrlX connector. Emerson DeltaV, that is like a special one, because I no longer have the hardware available to continue working on that, but we were actually the first people to be able to reverse-engineer the Emerson DeltaV protocol. One protocol I've still got on my to-do, and I really hope that with recent changes that'll be possible soon, is the new Siemens S7 protocol, but that'll probably take another few months or so, till I will be able to get started on that.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第10张图片

03

My Vision of PLC4X and IoTDB

PLC4X 与 IoTDB 集成愿景

正如我之前提到的,我对如何将 PLC4X 和 IoTDB 结合使用,以真正在工业领域产生影响有许多构想。

我对于 PLC4X 和 IoTDB 的愿景是构建一个使用 PLC4X 的网关系统,可在边缘侧的小型计算机甚至 PLC 本身上运行。这个网关系统使用 PLC4X 与设备进行通信,并以 Sparkplug B 格式生成数据。Sparkplug B 可能有些人听说过,但它是一个相对较新的协议,在其官网上表示旨在为工业自动化提供即插即用的功能,它实现的效果与 OPC-UA 希望通过配套规范实现的效果有些类似。通过 Sparkplug B,我们不仅可以发送数据,还可以发送数据的结构图。而这个网关系统与 IoTDB 之间的通信应该完全基于 MQTT,这也是 Sparkplug B 数据支持传输的方式。在 IoTDB 中,我们将使用一个新的 Sparkplug B 适配器,能够消费 Sparkplug B 数据,并将其直接写入到 IoTDB 中

As I mentioned, I have loads of visions of how we can use PLC4X and IoTDB together to really have an impact in the industry.

My vision for PLC4X and IoTDB is to build a gateway software using PLC4X that's runnable on small edge computers or even on the PLC itself. It uses PLC4X for communicating with the machines, and it produces data in a format called Sparkplug B. Sparkplug B, some of you might have heard, but it's a pretty new protocol that, I think on the website they aim to be offering the plug and play for industrial automation, and it sort of does what OPC-UA is trying with the companion specs. And with Sparkplug B, we will be not only emitting data, but also the schematics of the data, and the communication between this edge device and IoTDB should be purely based on MQTT. Well, Sparkplug B implies MQTT, but I just wanted to lay emphasis on this. In IoTDB, we would be using a new Sparkplug B adapter that is able to consume Sparkplug B data, and directly insert that into IoTDB.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第11张图片

下面是这个架构的示意图。你可以看到,有一些设备可以支持 Sparkplug B 协议,并且这样的设备会越来越多。如果你拥有这类设备,它的数据将发送到 MQTT 代理,而 IoTDB 的 Sparkplug B 适配器将直接消费这些消息。如果你有一些传统的硬件,那么你可以使用 Apache PLC4X 边缘网关与这些设备的协议进行通信,并通过 MQTT 发送 Sparkplug B 消息。然后,这些消息将被 Sparkplug B 适配器消费并写入到 IoTDB 中

It is a little schematic on how this generally would look like. So you can see, there are some Sparkplug B-enabled devices, and there will be more and more, so if you've got one of those, it will be sending to an MQTT broker, and the IoTDB Sparkplug B-adapter will be just directly consuming that. If you've got some legacy hardware, then you would be using the Apache PLC4X Edge Gateway that simply communicates with the other protocols and publishes Sparkplug B messages via MQTT, which are then consumed by the Sparkplug B-adapter and inserted into IoTDB.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第12张图片

那么想要实现这一架构,还需要哪些工作呢?为了实现这一目标,Apache IoTDB 需要能够连接到外部的 MQTT 代理。目前,如果我们想在 IoTDB 中使用 MQTT,我们会启用嵌入式的 MQTT 代理,但我认为这在设备大规模部署的场景下,可扩展性不够好。因此,我认为我们需要尽快做的一件事情是重构当前的 MQTT 适配器,不再使用嵌入式代理,而是构建一个 MQTT 代理客户端,并基于这些变化,为 IoTDB 实现一个 Sparkplug B 适配器。目前我们的 MQTT 适配器消费的是特殊格式的 JSON 消息,而 Sparkplug B 消费的是使用 Protobuf 编码的二进制有效载荷,因此如果想要实现 Sparkplug B 通信,需要构建一个完全不同的组件。

So what's needed in order to do that? Well, in order to be able to do that, Apache IoTDB needs to be able to connect to an external MQTT broker. Right now, if we want to use MQTT in IoTDB, we enable an embedded MQTT broker, but I think that's not really scalable in large installations, so I think one of the things we really need to change pretty soon is to refactor the MQTT adapter that we currently have, to not use the embedded broker, but to become a client, and based on these changes, to implement a Sparkplug B adapter for IoTDB. Right now our MQTT adapter consumes JSON messages that are formatted in a special format, while Sparkplug B consumes binary payload that is Protobuf encoded, so that will be a completely different component that will be able to consume Sparkplug B communication.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第13张图片

接下来的问题是,需要对 Apache PLC4X 的功能进行哪些调整或扩展?一方面,就像我前面提到的,PLC4X 支持订阅功能,但目前我们只在那些支持订阅的协议上实现了订阅功能,有 Beckhoff ADS、ProfiNet、KnxNet、BacNet 协议,基本上就是这些了。不过我们一直计划在不支持订阅的驱动程序上实现模拟订阅功能。例如在 Modbus 协议中,我们希望在后台轮询某个值,并根据订阅的设置,只有在这个值改变时才触发一个事件。

另外一点是,PLC4X 中广泛使用的是一个我们称之为 “scraper” 的模块。你可以将其视为一个调度程序,负责在给定链路上,以给定时间间隔,处理多类数据收集任务。这个模块还需要做很多更新,以支持订阅甚至是混合任务。所以我认为我们非常需要的是让 PLC4X 能够通过基于订阅的触发器进行轮询,对一组变量进行实时读取。这种功能当前是不支持的,也是绝对需要解决的问题。

最后一方面是,如果为了实现这个构想中的边缘网关系统,我在 IoTDB 上搭建了一个 Sparkplug B 的消费端,那么我将需要实现一个 Sparkplug B 的发送端。不过这应该是一个比较简单的任务,因为已经有一个 Eclipse 项目提供了实现这个发送端所有必要的基础设施。

Next thing is, which changes or extensions to Apache PLC4X are needed? Well, on the one hand side, I said that we support subscriptions. Well, right now we support subscriptions only on protocols that actually support that, right now that's the Beckhoff ADS protocol, ProfiNet, KnxNet, BacNet, I think that was all of them. But the thing is, we always planned to emulate subscriptions on drivers that actually don't support that. For example, on Modbus, in the background, we would be polling a certain value, and depending on the settings of the subscription, we would only be firing an event if the value actually changed.

Another part that is widely used in PLC4X, is what we call scraper. You can think of it as a scheduler that handles various tasks of collecting sets of data in a given interval on a given connection. That would greatly need an update for supporting subscriptions and sometimes even mixtures. So one thing that I think we really need is the ability to have a set of variables that are actively read by polling them, but to have a trigger for polling them that might be based on the subscription. And that's one thing the current implementation doesn't support, and one thing that definitely needs being addressing.

And the last thing, well, if I'm implementing a Sparkplug B consumer on the IoTDB side, for this proposed edge gateway, I would need to implement a Sparkplug B emitter. But that should actually be quite simple task, because there's already an Eclipse Project that's providing all the necessary infrastructures for that.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第14张图片

通过这些改变,我认为 IoTDB 有望成为工业自动化系统的核心数据枢纽我们可以利用它来实现在工业自动化领域被称为“统一命名空间”的架构。统一命名空间是一个可以获取有关设备信息的单独层级。它替代了当前逐级向上,每个应用程序都需要单独向工业硬件请求相同信息的等级架构。统一命名空间实现后,这些应用程序将不再直接连接到工业硬件,而是连接到我们的系统,并方便地通过 IoTDB 查询当前信息与数据

一旦我们实现了这一点,我认为 IoTDB 也可以成为未来构建定制化 MES 系统的真正核心。因为实际上,目前市场上的 MES 系统都不具备高扩展性,至少我不知道有哪一个系统能够跨多个节点进行扩展。它们都是单节点系统,所以通常用户在工厂中的应用效果会受到能够购买的服务器的限制,这并不理想。我过去曾看到,如果用户尝试将 IoT 或 IT 概念应用于工业自动化领域,例如微服务或只是写入服务,应用效果都比现有的 MES 系统要好很多。这就是为什么我非常希望我们能够在未来针对这一方面做出改变。

希望我的分享能为你们提供一些有趣的概念和想法。感谢聆听我的报告,期待下次能够与大家见面,祝各位收获愉快的参会体验。

With these changes, I think IoTDB can become the central data-hub for industrial automation systems. We can use it for implementing a software that provides what in the industrial automation field is often called the Unified-Namespace. It's a single place that you go to get information about your equipment. It's like replacing the current hierarchy, where one layer builds on top of the other, and every application separately asks for the same bits of information from industrial hardware. It would change to that these applications no longer connect directly to the industrial hardware, but they connect to our system, and simply ask IoTDB for the current values.

And with that, as soon as we have that in place, I think IoTDB can become the real central piece for building future custom MES systems. Because let's face it, the MES systems available out there, they just don't scale. At least I don't know a single one that actually scales across multiple nodes. They're all single-node systems that usually what you can do in your factory is limited by the size of server that you're able to purchase, and that doesn't work nicely. I've seen in the past that if you try applying modern IoT or modern IT concepts, something like microservices or just write services to industrial automation, you can do things a lot better. And that's why I'm really hoping for that we will be able to change in the future.

I hope I was able to provide you with some interesting ideas and concepts. Thank you for watching and I think I'll be available for questions, I hope. But looking forward to seeing you all and have a great time.

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第15张图片

可加欧欧获取大会相关PPT

微信号:apache_iotdb

2023 IoTDB 用户大会:天谋科技 Christofer Dutz《如何用Apache PLC4X构建极简工业数据采集》..._第16张图片

你可能感兴趣的:(iotdb,科技,apache)