系统工程与电子技术 ›› 2023, Vol. 45 ›› Issue (7): 1938-1956.doi: 10.12305/j.issn.1001-506X.2023.07.04
刘文波1,2, 姚翼荣1,2, 张弓3,4,*, 胡文3,4
收稿日期:
2022-03-14
出版日期:
2023-06-30
发布日期:
2023-07-11
通讯作者:
张弓
作者简介:
刘文波(1968—), 女, 教授, 博士, 主要研究方向为控制与测量技术、信号处理、深度学习、非线性系统控制基金资助:
Wenbo LIU1,2, Yirong YAO1,2, Gong ZHANG3,4,*, Wen HU3,4
Received:
2022-03-14
Online:
2023-06-30
Published:
2023-07-11
Contact:
Gong ZHANG
摘要:
超维计算是一种受大脑工作机制启发的新兴认知模型, 使用信息的高维、随机、全息分布式表示作为处理对象, 具有低运算成本、快速学习过程、高硬件友好性、强鲁棒性、不依赖大数据和优异的模型可解释性等优势, 在分类识别、信号处理、多任务学习、信息融合、智能决策等领域有着良好的应用前景。近年来, 超维计算受到的关注量持续增加, 展现出巨大的发展潜力, 为研究人员提供了一种新选择。本文详细介绍了超维计算的发展历史、基本原理和模型框架, 给出超维计算的典型应用实例, 并对超维计算现阶段存在的问题和未来可能的发展方向进行了探讨。
中图分类号:
刘文波, 姚翼荣, 张弓, 胡文. 超维计算概念、应用及研究进展[J]. 系统工程与电子技术, 2023, 45(7): 1938-1956.
Wenbo LIU, Yirong YAO, Gong ZHANG, Wen HU. Concept, application, and research progress of hyperdimensional computing[J]. Systems Engineering and Electronics, 2023, 45(7): 1938-1956.
1 |
ROSENBLATT F . The perceptron: a probabilistic model for information storage and organization in the brain[J]. Psychological Review, 1958, 65 (6): 386.
doi: 10.1037/h0042519 |
2 | RUMELHART D E , MCCLELLAND J L . Parallel distributed processing: explorations in the microstructure of cognition[M]. Cambridge: MIT Press, 1986. |
3 | ANDERSON J A . A simple neural network generating an inte-ractive memory[J]. Mathematical Biosciences, 1972, 14 (3/4): 197- 220. |
4 | KOHONEN T . Correlation matrix memories[J]. IEEE Trans.on Computers, 1972, 100 (4): 353- 359. |
5 |
FUKUSHIMA K . Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36, 193- 202.
doi: 10.1007/BF00344251 |
6 |
OLSHAUSEN B A , FIELD D J . Emergence of simple-cell receptive field properties by learning a sparse code for natural images[J]. Nature, 1996, 381 (6583): 607- 609.
doi: 10.1038/381607a0 |
7 |
BELL A J , SEJNOWSKI T J . An information-maximization approach to blind separation and blind deconvolution[J]. Neural Computation, 1995, 7 (6): 1129- 1159.
doi: 10.1162/neco.1995.7.6.1129 |
8 | KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]//Proc. of the Conference on Advances in Neural Information Processing Systems, 2012, 25: 1097-1105. |
9 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778. |
10 | SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Proc. of the Conference on Advances in Neural Information Processing Systems, 2014: 3104-3112. |
11 |
HINTON G E , DENG L , YU D , et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups[J]. IEEE Signal Processing Magazine, 2012, 29 (6): 82- 97.
doi: 10.1109/MSP.2012.2205597 |
12 |
LI F F , FERGUS R , PERONA P . One-shot learning of object categories[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2006, 28 (4): 594- 611.
doi: 10.1109/TPAMI.2006.79 |
13 | LIU Z, LI J G, SHEN Z Q, et al. Learning efficient convolutional networks through network slimming[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2736-2744. |
14 | ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//Proc. of the European Conference on Computer Vision, 2014: 818-833. |
15 |
ZHANG Y , TINˇO P , LEONARDIS A , et al. A survey on neural network interpretability[J]. IEEE Trans.on Emerging Topics in Computational Intelligence, 2021, 5 (5): 726- 742.
doi: 10.1109/TETCI.2021.3100641 |
16 |
THOMAS A , DASGUPTA S , ROSING T . Theoretical foundations of hyperdimensional computing[J]. Journal of Artificial Intelligence Research, 2021, 72, 215- 249.
doi: 10.1613/jair.1.12664 |
17 | KANERVA P . Sparse distributed memory[M]. Cambridge: MIT Press, 1988. |
18 |
KANERVA P . Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors[J]. Cognitive Computation, 2009, 1 (2): 139- 159.
doi: 10.1007/s12559-009-9009-8 |
19 |
MASSE N Y , TURNER G C , JEFFERIS G S X E . Olfactory information processing in Drosophila[J]. Current Biology, 2009, 19 (16): R700- R713.
doi: 10.1016/j.cub.2009.06.026 |
20 |
CARON S J C , RUTA V , ABBOTT L F , et al. Random convergence of olfactory inputs in the Drosophila mushroom body[J]. Nature, 2013, 497 (7447): 113- 117.
doi: 10.1038/nature12063 |
21 |
SQUIRE L R . Memory and the hippocampus: a synthesis from findings with rats, monkeys, and humans[J]. Psychological Review, 1992, 99 (2): 195- 231.
doi: 10.1037/0033-295X.99.2.195 |
22 | RAHIMI A, KANERVA P, RABAEY J M. A robust and energy-efficient classifier using brain-inspired hyperdimensional computing[C]//Proc. of the International Symposium on Low Power Electronics and Design, 2016: 64-69. |
23 | ZHANG S, WANG R, ZHANG J J, et al. Assessing robustness of hyperdimensional computing against errors in associative memory[C]//Proc. of the IEEE 32nd International Conference on Application-specific Systems, Architectures and Processors, 2021: 211-217. |
24 |
HERSCHE M , LIPPUNER S , KORB M , et al. Near-channel classifier: symbiotic communication and classification in high-dimensional space[J]. Brain Informatics, 2021, 8 (1): 16.
doi: 10.1186/s40708-021-00138-0 |
25 |
KARUNARATNE G , SCHMUCK M , LE GALLO M , et al. Robust high-dimensional memory-augmented neural networks[J]. Nature Communications, 2021, 12 (1): 2468.
doi: 10.1038/s41467-021-22364-0 |
26 |
YAO Y R , LIU W B , ZHANG G , et al. Radar-based human activity recognition using hyperdimensional computing[J]. IEEE Trans.on Microwave Theory and Techniques, 2022, 70 (3): 1605- 1619.
doi: 10.1109/TMTT.2021.3134992 |
27 |
GE L L , PARHI K K . Classification using hyperdimensional computing: a review[J]. IEEE Circuits and Systems Magazine, 2020, 20 (2): 30- 47.
doi: 10.1109/MCAS.2020.2988388 |
28 | HASSAN E , HALAWANI Y , MOHAMMAD B , et al. Hyper-dimensional computing challenges and opportunities for AI applications[J]. IEEE Access, 2021, 10, 97651- 97664. |
29 | IMANI M, KONG D, RAHIMI A, et al. VoiceHD: hyperdimensional computing for efficient speech recognition[C]//Proc. of the IEEE International Conference on Rebooting Computing, 2017. |
30 | JOSHI A, HALSETH J T, KANERVA P. Language geometry using random indexing[C]//Proc. of the International Symposium on Quantum Interaction, 2016: 265-274. |
31 |
IMANI M , HWANG J , ROSING T , et al. Low-power sparse hyperdimensional encoder for language recognition[J]. IEEE Design and Test, 2017, 34 (6): 94- 101.
doi: 10.1109/MDAT.2017.2740839 |
32 | MA D N, GUO J M, JIANG Y, et al. HDTest: differential fuzz testing of brain-inspired hyperdimensional computing[C]//Proc. of the ACM/IEEE Design Automation Conference, 2021: 391-396. |
33 | YAO Y R , LIU W B , ZHANG G , et al. Fast SAR image recog-nition via hyperdimensional computing using monogenic mapping[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19, 4508205. |
34 | RAHIMI A , KANERVA P , BENINI L , et al. Efficient biosignal processing using hyperdimensional computing: network templates for combined learning and classification of ExG signals[J]. Proceedings of the IEEE, 2018, 107 (1): 123- 143. |
35 | RAHIMI A, BENATTI S, KANERVA P, et al. Hyperdimensional biosignal processing: a case study for EMG-based hand gesture recognition[C]//Proc. of the IEEE International Conference on Rebooting Computing, 2016. |
36 | BURRELLO A , SCHINDLER K , BENINI L , et al. Hyperdimensional computing with local binary patterns: one-shot learning of seizure onset and identification of ictogenic brain regions using short-time iEEG recordings[J]. IEEE Trans.on Biome-dical Engineering, 2019, 67 (2): 601- 613. |
37 | KIM Y, IMANI M, MOSHIRI N, et al. GenieHD: efficient DNA pattern matching accelerator using hyperdimensional computing[C]//Proc. of the Design, Automation and Test in Europe Conference and Exhibition, 2020: 115-120. |
38 |
CHANG C Y , CHUANG Y , CHANG E J , et al. MulTa-HDC: a multi-task learning framework for hyperdimensional computing[J]. IEEE Trans.on Computers, 2021, 70 (8): 1269- 1284.
doi: 10.1109/TC.2021.3073409 |
39 | RÄSÄNEN O J , SAARINEN J P . Sequence prediction with sparse distributed hyperdimensional coding applied to the analysis of mobile phone use patterns[J]. IEEE Trans.on Neural Networks and Learning Systems, 2015, 27 (9): 1878- 1889. |
40 |
RASANEN O , KAKOUROS S . Modeling dependencies in multiple parallel data streams with hyperdimensional computing[J]. IEEE Signal Processing Letters, 2014, 21 (7): 899- 903.
doi: 10.1109/LSP.2014.2320573 |
41 | SCHLEGEL K, MIRUS F, NEUBERT P, et al. Multivariate time series analysis for driving style classification using neural networks and hyperdimensional computing[C]//Proc. of the IEEE Intelligent Vehicles Symposium, 2021: 602-609. |
42 | RAHIMI A, KANERVA P, MILLAN J R, et al. Hyperdimensional computing for noninvasive brain-computer interfaces: blind and one-shot classification of EEG error-related potentials[C]//Proc. of the ACM/EAI International Conference on Bio-inspired Information and Communications Technologies, 2017: 19-26. |
43 | IMANI M, MORRIS J, BOSCH S, et al. AdaptHD: adaptive efficient training for brain-inspired hyperdimensional computing[C]//Proc. of the IEEE Biomedical Circuits and Systems Conference, 2019. |
44 |
RAHIMI A , DATTA S , KLEYKO D , et al. High-dimensional computing as a nanoscalable paradigm[J]. IEEE Trans.on Circuits and Systems I: Regular Papers, 2017, 64 (9): 2508- 2521.
doi: 10.1109/TCSI.2017.2705051 |
45 | IMANI M, MESSERLY J, WU F, et al. A binary learning framework for hyperdimensional computing[C]//Proc. of the Design, Automation and Test in Europe Conference & Exhibition, 2019: 126-131. |
46 | SCHMUCK M , BENINI L , RAHIMI A . Hardware optimizations of dense binary hyperdimensional computing: remateria-lization of hypervectors, binarized bundling, and combinational associative memory[J]. ACM Journal on Emerging Technologies in Computing Systems, 2019, 15 (4): 32. |
47 |
SALAMAT S , IMANI M , ROSING T . Accelerating hyperdimensional computing on FPGAs by exploiting computational reuse[J]. IEEE Trans.on Computers, 2020, 69 (8): 1159- 1171.
doi: 10.1109/TC.2020.2992662 |
48 | LI H, WU T F, RAHIMI A, et al. Hyperdimensional computing with 3D VRRAM in-memory kernels: device-architecture co-design for energy-efficient, error-resilient language recognition[C]//Proc. of the IEEE International Electron Devices Meeting, 2016. |
49 | WU T F, LI H, HUANG P C, et al. Brain-inspired computing exploiting carbon nanotube FETs and resistive RAM: hyperdimensional computing case study[C]//Proc. of the IEEE International Solid-State Circuits Conference, 2018: 492-494. |
50 |
KARUNARATNE G , LE-GALLO M , CHERUBINI G , et al. In-memory hyperdimensional computing[J]. Nature Electro-nics, 2020, 3 (6): 327- 337.
doi: 10.1038/s41928-020-0410-3 |
51 |
HOPFIELD J J . Neural networks and physical systems with emergent collective computational abilities[J]. Proceedings of the National Academy of Sciences, 1982, 79 (8): 2554- 2558.
doi: 10.1073/pnas.79.8.2554 |
52 |
ACKLEY D H , HINTON G E , SEJNOWSKI T J . A learning algorithm for Boltzmann machines[J]. Cognitive Science, 1985, 9 (1): 147- 169.
doi: 10.1207/s15516709cog0901_7 |
53 |
HINTON G E , OSINDERO S , TEH Y W . A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18 (7): 1527- 1554.
doi: 10.1162/neco.2006.18.7.1527 |
54 | HINTON G E , ANDERSON J A . Parallel models of associative memory: updated edition[M]. New York: Psychology Press, 2014. |
55 | MINSKY M , PAPERT S . Perceptrons[M]. Cambridge: MIT Press, 1969. |
56 |
WERBOS P J . Backpropagation through time: what it does and how to do it[J]. Proceedings of the IEEE, 1990, 78 (10): 1550- 1560.
doi: 10.1109/5.58337 |
57 |
RUMELHART D E , HINTON G E , WILLIAMS R J . Learning representations by back-propagating errors[J]. Nature, 1986, 323 (6088): 533- 536.
doi: 10.1038/323533a0 |
58 |
LECUN Y , BOSER B , DENKER J S , et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1989, 1 (4): 541- 551.
doi: 10.1162/neco.1989.1.4.541 |
59 | HINTON G E . Mapping part-whole hierarchies into connectionist networks[J]. Artificial Intelligence, 1990, 46 (1/2): 47- 75. |
60 | SMOLENSKY P . Tensor product variable binding and the representation of symbolic structures in connectionist systems[J]. Artificial Intelligence, 1990, 46 (1/2): 159- 216. |
61 |
PLATE T A . Holographic reduced representations[J]. IEEE Trans.on Neural Networks, 1995, 6 (3): 623- 641.
doi: 10.1109/72.377968 |
62 | KANERVA P. Binary spatter-coding of ordered k-tuples[C]//Proc. of the International Conference on Artificial Neural Networks, 1996: 869-873. |
63 | GAYLER R W. Multiplicative binding representation operators and analogy[C]//Proc. of the Workshop on Advances in Analogy Research: Integration of Theory and Data from the Cognitive Computational and Neural Sciences, 1998. |
64 | PLATE T A . Holographic reduced representations: distributed representation for cognitive structures[M]. Stanford: CSLI, 2003. |
65 | LEVY S D, GAYLER R. Vector symbolic architectures: a new building material for artificial general intelligence[C]//Proc. of the Conference on Artificial General Intelligence, 2008: 414-418. |
66 | IMANI M, HUANG C, KONG D, et al. Hierarchical hyperdimensional computing for energy efficient classification[C]//Proc. of the ACM/ESDA/IEEE Design Automation Confe-rence, 2018. |
67 | QUASTHOFF U, RICHTER M, BIEMANN C. Corpus portal for search in monolingual corpora[C]//Proc. of the International Conference on Language Resources and Evaluation, 2006: 1799-1802. |
68 | KOEHN P. Europarl: a parallel corpus for statistical machine translation[C]//Proc. of the Machine Translation Summit X, 2005: 79-86. |
69 | KLEYKO D , OSIPOV E , SENIOR A , et al. Holographic graph neuron: a bioinspired architecture for pattern processing[J]. IEEE Trans.on Neural Networks and Learning Systems, 2016, 28 (6): 1250- 1262. |
70 | MANABAT A X, MARCELO C R, QUINQUITO A L, et al. Performance analysis of hyperdimensional computing for cha-racter recognition[C]//Proc. of the International Symposium on Multimedia and Communication Technology, 2019. |
71 | IMANI M, KIM Y, WORLEY T, et al. HDCluster: an accurate clustering using brain-inspired high-dimensional computing[C]//Proc. of the Design, Automation and Test in Europe Conference & Exhibition, 2019: 1591-1594. |
72 |
KLEYKO D , RAHIMI A , RACHKOVSKIJ D A , et al. Classification and recall with binary hyperdimensional computing: tradeoffs in choice of density and mapping characteristics[J]. IEEE Trans.on Neural Networks and Learning Systems, 2018, 29 (12): 5880- 5898.
doi: 10.1109/TNNLS.2018.2814400 |
73 | IMANI M, SALAMAT S, KHALEGHI B, et al. SparseHD: algorithm-hardware co-optimization for efficient high-dimensional computing[C]//Proc. of the IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines, 2019: 190-198. |
74 |
MITROKHIN A , SUTOR P , SUMMERS-STAY D , et al. Symbolic representation and learning with hyperdimensional computing[J]. Frontiers in Robotics and AI, 2020, 7, 63.
doi: 10.3389/frobt.2020.00063 |
75 | ALONSO P, SHRIDHAR K, KLEYKO D, et al. HyperEmbed: tradeoffs between resources and performance in NLP tasks with hyperdimensional computing enabled embedding of n-gram statistics[C]//Proc. of the International Joint Conference on Neural Networks, 2021. |
76 | KLEYKO D , KHEFFACHE M , FRADY E P , et al. Density encoding enables resource-efficient randomly connected neural networks[J]. IEEE Trans.on Neural Networks and Learning Systems, 2020, 32 (8): 3777- 3783. |
77 | MORRIS J, LUI H W, STEWART K, et al. HyperSpike: hyperdimensional computing for more efficient and robust spiking neural networks[C]//Proc. of the 2022 Design, Automation & Test in Europe Conference & Exhibition, 2022: 664-669. |
78 |
KLEYKO D , FRADY E P , KHEFFACHE M , et al. Integer echo state networks: efficient reservoir computing for digital hardware[J]. IEEE Trans.on Neural Networks and Learning Systems, 2022, 33 (4): 1688- 1701.
doi: 10.1109/TNNLS.2020.3043309 |
79 | HSU P K, YU S. In-memory 3D NAND flash hyperdimensional computing engine for energy-efficient SARS-CoV-2 genome sequencing[C]//Proc. of the IEEE International Memory Workshop, 2022. |
80 | DUBREUIL T, AMARI P, BARRAUD S, et al. A novel 3D 1T1R RRAM architecture for memory-centric hyperdimensional computing[C]//Proc. of the IEEE International Memory Workshop, 2022. |
81 |
DATTA S , ANTONIO R A G , ISON A R S , et al. A programmable hyper-dimensional processor architecture for human-centric IoT[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019, 9 (3): 439- 452.
doi: 10.1109/JETCAS.2019.2935464 |
[1] | 王湖升, 陈伯孝, 叶倾知. 基于箔条干扰实测数据的对抗方法研究[J]. 系统工程与电子技术, 2023, 45(7): 2010-2021. |
[2] | 王春政, 胡明华, 杨磊, 赵征. 空中交通延误预测研究综述[J]. 系统工程与电子技术, 2022, 44(3): 863-874. |
[3] | 张心宇, 刘源, 宋佳凝. 基于LSTM神经网络的短期轨道预报[J]. 系统工程与电子技术, 2022, 44(3): 939-947. |
[4] | 赵凡, 金虎. 基于GAN的通信干扰波形生成技术[J]. 系统工程与电子技术, 2021, 43(4): 1080-1088. |
[5] | 宋波, 叶伟, 孟祥辉. 基于多智能体强化学习的动态频谱分配方法综述[J]. 系统工程与电子技术, 2021, 43(11): 3338-3351. |
[6] | 彭芳, 吴军, 王帅, 向建军. 基于SVRGD的机载预警雷达自适应波束形成算法[J]. 系统工程与电子技术, 2021, 43(1): 83-90. |
[7] | 张菁, 何友, 彭应宁, 李刚. 新一代战斗机技术特征论证[J]. 系统工程与电子技术, 2018, 40(8): 1754-1759. |
[8] | 颛孙少帅, 杨俊安, 刘辉, 黄科举. 基于正强化学习和正交分解的干扰策略选择算法[J]. 系统工程与电子技术, 2018, 40(3): 518-525. |
[9] | 王桂胜, 任清华, 姜志刚, 刘洋, 徐兵政. 基于信号特征空间的TDCS干扰分类识别[J]. 系统工程与电子技术, 2017, 39(9): 1950-1958. |
[10] | 王邑, 孙金标, 肖明清, 罗继勋. 基于类型2区间模糊K近邻分类器的动态武器目标分配方法研究[J]. 系统工程与电子技术, 2016, 38(6): 1314-1319. |
[11] | 顾斌杰, 潘丰. 改进的v-支持向量回归机的v解路径算法[J]. 系统工程与电子技术, 2016, 38(1): 205-214. |
[12] | 唐宁, 蔡晋, 李原, 张开富. 基于扩散先验分布的成组技术分类识别方法[J]. Journal of Systems Engineering and Electronics, 2012, 34(4): 827-832. |
[13] | 王海华, 沈晓峰. 一种新的雷达辐射源信号脉内特征提取方法[J]. Journal of Systems Engineering and Electronics, 2009, 31(4): 809-811. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||