超维术士_超维计算及其在人工智能中的作用

超维术士

它是什么？ (What is it?)

Don’t be intimidated by the name hyperdimensional computing…Even though it sounds like something from Star Trek, this simply means:

不要被超维计算这个名字所吓倒 ……尽管听起来像是《星际迷航》中的东西，但这仅意味着：

Computing using large vectors (say of 10000 values each)

使用大向量进行计算(每个向量有10000个值)

Why would we do that?

我们为什么要这样做？

As it turns out, such vectors have interesting properties that are appealing to the realm of AI:

事实证明，这些向量具有吸引AI领域的有趣特性：

• We can pick any number of random vectors from the hyperspace of 10k dimensions and they will all be approximately orthogonal to each other. This nearly garanties that we can generate a fresh vector at any time that do not resemble any previous ones.

  我们可以从10k维的超空间中选择任意数量的随机向量，并且它们将彼此近似正交。 这样几乎可以保证我们可以在任何时候生成一个与以前的相似的新矢量。

• We can add two vectors to obtain a vector similar to them.

  我们可以添加两个向量以获得类似于它们的向量。

• We can multiply two vectors to obtain a vector dissimilar to them.

  我们可以将两个向量相乘以获得与它们不同的向量。

• We can represent data holistically (meaning the value is distributed among many data points), making our vector redundent and robust to corruption (we can reconstruct the meaning of a vector as long as we have 60% of its content).

  我们可以整体表示数据(意味着值分布在许多数据点之间)，从而使向量变得冗余且对破坏具有鲁棒性(只要向量的内容有60％，就可以重构向量的含义)。

让我们探讨一下我们可以做什么！ (Let’s explore what we can do with this!)

示例1：猜测任意文本的语言 (Example 1: Guessing the language of arbitrary text)

Problem: we have text (of any size and content) and we want to guess if it’s French or English.

问题 ：我们有文本(任何大小和内容)，我们想知道它是法语还是英语。

Solution: compute vectors for each language and input and compare their angles.

解决方案 ：计算每种语言和输入的向量，并比较它们的角度。

Steps:

步骤 ：

1. compute a single vector for each language: one for French, one for English.
   为每种语言计算一个向量：一个用于法语，一个用于英语。
2. compute a single vector for the input text.
   计算输入文本的单个向量。

3. compare the input text vector with both language vectors using the cosine similarity. The closest language vector to our input vector is most likely the input’s language.

   使用余弦相似度比较输入的文字向量和两种语言的向量。 与我们的输入向量最接近的语言向量很可能是输入的语言。

Ok… let’s go into each step one by one.

好吧……让我们一步一步地进行。

步骤1：计算一种语言的10k向量 (Step 1: compute a 10k vector for a language)

Given some sample text for say English, the vector generated with it will “represent” the English language (naturally the larger the text the better it will represent it).

给定一些用于说英语的示例文本，用它生成的向量将“代表”英语(自然，文本越大，代表它越好)。

The encoding process works as follows:

编码过程如下：

• Generate a random 10k vector for each letter and store it. We can use +1 and -1 as the available values, which I’ll call “bipolar vectors”, so the vector will look something like this: [ +1 -1 +1 +1 -1 -1 +1 … ]
  为每个字母生成一个随机的10k向量并将其存储。 我们可以使用+1和-1作为可用值，我将其称为“双极向量”，因此向量将如下所示：[+1 -1 +1 +1 -1 -1 -1……]

• Encode trigrams using rotate and multiply operations. This summarizes short sequences of 3 letters into 10k vectors.

  使用旋转编码三字母组合 并乘以运算。 这将 3个字母的短序列汇总为10k个向量。

  Example: the input “something” will generate one 10k vector for each 3 sequential characters: “som”, “ome”, “met”, “eth”, “thi”, “hin”, “ing”.

  示例：输入“ something”将为每个3个连续字符生成一个10k矢量：“ som”，“ ome”，“ met”，“ eth”，“ thi”，“ hin”，“ ing”。

  Each trigram (say “som”) will be calculated as

  每个三联词(例如“ som”)将计算为

  som = rr(s)×r(o)×m

  som = rr( s )×r( o )× m

  - We use

  - 我们用

  rotate r(x) to encode the position of a letter within the trigram.

  旋转 r( x )以编码字母在三字母组合词内的位置。

  - We use

  - 我们用

  multiply to produce a new unique vector dissimilar to any of the letter vectors.

  乘以产生一个新的唯一向量，该向量与任何字母向量都不相同。

  multiply to produce a new unique vector dissimilar to any of the letter vectors. Note that we can replace trigrams by “quadgrams” and it will still work. You can experiment to see what works best (even mix sizes), I found that sets of 3 yield good results.

  乘以产生一个新的唯一向量，该向量与任何字母向量都不相同。 注意，我们可以用“ quadgrams”代替trigrams，它仍然可以使用。 您可以尝试查看哪种方法效果最好(甚至混合大小)，我发现3组效果很好。

• Take all of the trigrams and add them together to get one 10k vector that is the representation of the language. This vector is going to be similar to all of the trigrams combined.

  取所有三元组并将它们加在一起，以获得一个表示语言的10k向量。 该向量将类似于所有组合的三字母组。

步骤2：为输入文本计算10k向量 (Step 2: compute a 10k vector for the input text)

The process is exactly the same for encoding a language vector.

编码语言向量的过程完全相同。

步骤3：比较输入和语言向量 (Step 3: compare the input and language vectors)

Using the cosine similarity we can compare the angles between vectors:

使用余弦相似度，我们可以比较向量之间的角度：

• cos(A, B) = 1 means both vectors are equal.
  cos(A，B)= 1表示两个向量相等。
• cos (A, B) = -1 means they are equal, but opposite.
  cos(A，B)= -1表示它们相等，但相反。
• cos (A, B) = 0 means they are orthogonal.
  cos(A，B)= 0表示它们是正交的。

The closest cosine to 1 wins!

最接近1的余弦值！

释放更多力量！ (Unlock more power!)

You can also query your language vectors for information. Say you want to know what letter is most frequently seen after a sequence of letters in a particular language? Just ask the vector:

您也可以查询语言向量以获得信息。 假设您想知道在使用特定语言的一系列字母之后最常看到的字母？ 只需问矢量：

Problem: give me the most commonly seen letters after the sequence “th” in the English language.

问题：给我最常用的英文字母“ th”后的字母。

1. Compute the query vector Q = rr(t)×r(h)

   计算查询向量Q = rr( t )×r( h )

   (remember that a

   (请记住，

   trigram is encoded as rr(a)×r(b)×c)

   Trigram编码为rr( a )×r( b )× c

2. Compute the answer vector A = Q×en(where en is the language vector for english)

   计算答案向量A = Q × en (其中en是英语的语言向量)

Result would be “e”, followed by “space”, “a”, “i”, “r”, “o”, etc…

结果将是“ e”，然后是“空格”，“ a”，“ i”，“ r”，“ o”等。

Find the code example under examples/languageGuessing: https://github.com/gokadin/hyperdimensional-computing

在examples / languageGuessing下找到代码示例： https : //github.com/gokadin/hyperDimension-computing

示例2：使用HD向量进行语义绑定 (Example 2: Using HD vectors for semantic binding)

Problem: given the phrase “I want to run”, encode its semantic structure within a 10k vector.

问题 ：给定短语“我要跑步”，将其语义结构编码在10k向量内。

Solution: pick a random vector for each component (subject, verb, object, I, want, run) and mash them together.

解决方案 ：为每个成分(主语，动词，宾语，I，要，跑)选择一个随机向量并将它们混在一起。

In his book “How to build a brain”, Dr. Chris Eliasmith demonstrates how you can generate such a vector with the following formula:

克里斯·埃里亚史密斯(Chris Eliasmith)博士在他的《如何建立大脑》一书中演示了如何使用以下公式生成这样的向量：

P = subject ⊛ I + verb ⊛ want + object ⊛ run

P = 主题 ⊛I + 动词 ⊛ 想 + 对象 ⊛ 运行

where ⊛ represents circular convolution.

其中⊛表示圆形卷积 。

Then, if you want to query P for “what is the verb?”, you can unbind like so:

然后，如果要在P上查询“动词是什么？”，可以这样取消绑定：

want ≈ P ⊛ verb’

想 ≈P⊛ 动词”

where verb’ is the approximate inverse (or involution) of verb.

其中动词”是动词的近似逆(或退化)。

Of course you can take this as far as you like. Nothing keeps you from building complex syntactic structures and complex queries.

当然，您可以随心所欲。 没有什么可以阻止您构建复杂的语法结构和复杂的查询。

Find the code example under examples/semanticBinding: https://github.com/gokadin/hyperdimensional-computing

在examples / semanticBinding下找到代码示例： https : //github.com/gokadin/hyperDimension-computing

期待 (Looking forward)

The properties of hyperdimensional computing make it a good fit for many functions in AI. A big part of intelligence is about detecting, storing, binding and unbinding noisy patterns. HD vectors are well suited to represent noisy, redundent and robust patterns and there are many operations to bind and unbind them.

超维计算的属性使其非常适合AI中的许多功能。 情报的很大一部分是关于检测，存储，绑定和解除绑定噪声模式。 HD向量非常适合表示噪声，冗余和鲁棒的模式，并且有很多操作可以对其进行绑定和取消绑定。

Associative memory is the first application that comes to mind, however there are many less obvious uses for HD computing such as learning and fluid intelligence (i.e. solving logical problems using induction). Dr. Chris Eliasmith describes all of these in his book “How to build a brain” very well.

联想记忆是我想到的第一个应用程序，但是对于高清计算来说，有许多不太明显的用途，例如学习和流体智能(即使用归纳法解决逻辑问题)。 克里斯·埃里亚史密斯(Chris Eliasmith)博士在他的《如何建立大脑》一书中很好地描述了所有这些。

HD computing can even prove useful in non-AI domains. Example, I recently participated in a system design exercise with my colleagues where we needed to store 100s of thousands of documents to compare against each student’s submission of an essay to detect plagiarism. The solution involved transforming the documents into HD vectors and using the cosine similarity to detect how close the student’s submission was to each document.

高清计算甚至可以在非AI领域发挥作用。 例如，我最近与同事一起参加了系统设计练习，我们需要存储成千上万的文档，以与每个学生提交的论文进行比较以检测窃。 解决方案包括将文档转换为高清矢量，并使用余弦相似度来检测学生提交的文档与每个文档的接近程度。

I believe that hyperdimensional computing has a significant part to play in AI’s research and the future of computing.

我相信超多维计算在AI的研究和计算的未来中将发挥重要作用。

The possibilities are limitless!

可能性是无限的！

补充笔记 (Additional notes)

• Besides cosine similarity for comparing vectors, you can use the hamming distance.

  除了用于比较向量的余弦相似度之外，还可以使用汉明距离 。

• Why 10k dimensions? Why not 1k or 1M? 10k dimensions contains billions of nearly orthogonal vectors. This nearly garanties us an orthogonal vector when choosing one at random to all previously chosen ones, therefore we don’t need higher dimensions for this. Based on the use case, lower dimensions can also work.
  为什么尺寸为10k？ 为什么不选择1k或1M？ 10k维度包含数十亿个几乎正交的向量。 当随机选择一个先前选择的所有矢量时，这几乎使我们正交了，因此我们不需要更大的尺寸。 根据用例，较小的尺寸也可以使用。
• You don’t have to use bipolar (+1, -1) vectors. You can do binary (1, 0) or anything else you want as long as it works for your use case and the operations you use.
  您不必使用双极性(+1，-1)向量。 您可以执行二进制(1、0)或任何其他操作，只要它适用于您的用例和所使用的操作即可。

• Why did we use rotate and multiply for binding in example 1, but circular convolution in example 2? I could have demonstrated both examples using any of the operations. The important thing is that the operations are reversible for binding/unbinding to work. For binary vectors, the same thing can be achieved with XOR for example. The decision of which operations, type of vectors (bipolar, binary, etc…) you use depends on the use case, scalability and computing efficiency considerations.

  为什么在示例1中使用旋转和乘法进行绑定，而在示例2中为什么使用圆卷积呢？ 我可以使用任何操作演示两个示例。 重要的是操作对于绑定/解除绑定是可逆的。 对于二进制向量，例如，可以通过XOR实现相同的操作。 使用哪种运算，矢量类型(双极性，二进制等)的决定取决于用例，可伸缩性和计算效率方面的考虑。

参考资料和资源 (References and resources)

• Paper on hyperdimensional computing by Pentti Kanerva: http://www.rctn.org/vs265/kanerva09-hyperdimensional.pdf

  Pentti Kanerva撰写的有关超维计算的论文： http ://www.rctn.org/vs265/kanerva09-hyperDimension.pdf

• http://web.stanford.edu/class/ee380/Abstracts/171025-slides.pdf

  http://web.stanford.edu/class/ee380/Abstracts/171025-slides.pdf

• How to Build a Brain: A Neural Architecture for Biological Cognition by Chris Eliasmith

  如何建立大脑：克里斯·埃利亚史密斯(Chris Eliasmith)的神经认知神经体系结构

• My github demonstrating the examples.

  我的github演示了示例。

翻译自: https://medium.com/dataseries/hyperdimensional-computing-and-its-role-in-ai-d6dc2828e6d6

超维术士