Jump to content

Hardware for artificial intelligence

From Wikipedia, the free encyclopedia

Specialized computer hardware is often used to execute artificial intelligence (AI) programs faster, and with less energy, such as Lisp machines, neuromorphic engineering, event cameras, and physical neural networks. Since 2017, several consumer grade CPUs and SoCs have on-die NPUs. As of 2023, the market for AI hardware is dominated by GPUs.[1]

Raspberry Pi AI Kit

As of the 2020s, AI computation is dominated by graphics processing units (GPUs) and newer domain-specific accelerators such as Google's Tensor Processing Units (TPUs), AMD's Instinct MI300 series, and various on-device neural-processing units (NPUs) found in consumer hardware.[2][3]

Scope

[edit]

For the purposes of this article, AI hardware refers to computing components and systems specifically designed or optimized to accelerate artificial-intelligence workloads such as machine-learning training or inference. This includes general-purpose accelerators used for AI (for example, GPUs) and domain-specific accelerators (for example, TPUs, NPUs, and other AI ASICs).[4]

Event-based cameras are sometimes discussed in the context of neuromorphic computing, but they are input sensors rather than AI compute devices. Conversely, components such as memristors are basic circuit elements rather than specialized AI hardware when considered alone.[5][6]

Lisp machines

[edit]
Computer hardware

Lisp machines were developed in the late 1970s and early 1980s to make artificial intelligence programs written in the programming language Lisp run faster.[7]

Dataflow architecture

[edit]

Dataflow architecture processors used for AI serve various purposes with varied implementations like the polymorphic dataflow[8] Convolution Engine[9] by Kinara (formerly Deep Vision), structure-driven dataflow by Hailo,[10] and dataflow scheduling by Cerebras.[11]

Component hardware

[edit]

AI accelerators

[edit]
AI Accelerators Component hardware

Since the 2010s, advances in computer hardware have led to more efficient methods for training deep neural networks that contain many layers of non-linear hidden units and a very large output layer.[12] By 2019, graphics processing units (GPUs), often with AI-specific enhancements, had displaced central processing units (CPUs) as the dominant means to train large-scale commercial cloud AI.[13] OpenAI estimated the hardware compute used in the largest deep learning projects from Alex Net (2012) to Alpha Zero (2017), and found a 300,000-fold increase in the amount of compute needed, with a doubling-time trend of 3.4 months.[14][15]

General-purpose GPUs for AI

[edit]

Since the 2010s, graphics processing units (GPUs) have been widely used to train and deploy deep learning models because of their highly parallel architecture and high memory bandwidth. Modern data-center GPUs include dedicated tensor or matrix-math units that accelerate neural-network operations.[16]

In 2022, NVIDIA introduced the Hopper-generation H100 GPU, adding FP8 precision support and faster interconnects for large-scale model training.[17] AMD and other vendors have also developed GPUs and accelerators aimed at AI and high-performance computing workloads.[18] Processing in memory devices can be used in place of industry standard HBM memory stacks to speed up computation and increase efficiency by placing some processing functions very close to the memory cells within the devices, at the cost of lower memory capacity per device due to the footprint required by the added processors within the device.[19]

Domain-specific accelerators (ASICs / NPUs)

[edit]

Beyond general-purpose GPUs, several companies have developed application-specific integrated circuits (ASICs) and neural processing units (NPUs) tailored for AI workloads. Google introduced the Tensor Processing Unit (TPU) in 2016 for deep-learning inference, with later generations supporting large-scale training through dense systolic-array designs and optical interconnects.[20] Other vendors have released similar devices—such as Apple's Neural Engine and various on-device NPUs—that emphasize energy-efficient inference in mobile or edge computing environments.[21] FPGAs are similar to ASICs in that they operate on a design that is not general purpose, however it is user defined and more suitable for low scale production which is the opposite of ASICs.[22] Neuromorphic and Analog AI accelerators promise to reduce the amount of power consumed by AI by either replacing digital computation with analog computation, or replicating neural networks directly in hardware.[23][24][25] Photonic processors have been explored.[26] Cerebras has launched a wafer scale processor to place large amounts of SRAM on a single processor and speed up memory bandwidth for AI,[27] and Groq has designed an LPU with large amounts of SRAM for low time to first token[28] unlike traditional NPUs, which focus on compute throughput. Taalas has developed ASICs that implement as hardware logic gates a specific AI model and are thus only capable of running that model, similarly to mask ROMs. The model is 'etched' as a circuit design onto the silicon via semiconductor device manufacturing and thus cannot be changed. However it has demonstrated speeds of up to 17,000 tokens per second.[29]

Memory and interconnects

[edit]

AI accelerators rely on fast memory and inter-chip links to manage the large data volumes of training and inference. High-bandwidth memory (HBM) stacks, standardized as HBM3 in 2022, provide terabytes-per-second throughput on modern GPUs and ASICs.[30] These accelerators are often connected through dedicated fabrics such as NVIDIA's NVLink and NVSwitch or optical interconnects used in TPU systems to scale performance across thousands of chips.[31]

References

[edit]
  1. ↑ "Nvidia: The chip maker that became an AI superpower". BBC News. 25 May 2023. Retrieved 18 June 2023.
  2. ↑ "NVIDIA H100 Tensor Core GPU Architecture Whitepaper". NVIDIA. 2022. Retrieved 4 November 2025.
  3. ↑ "Google Cloud TPU v5 Announcement". Google Cloud Blog. 2023. Retrieved 4 November 2025.
  4. ↑ Sze, Vivienne; Chen, Yu-Hsin; Yang, Tien-Ju; Emer, Joel (2017). "Efficient Processing of Deep Neural Networks: A Tutorial and Survey". Proceedings of the IEEE. 105 (12): 2295–2329. arXiv:1703.09039. doi:10.1109/JPROC.2017.2761740.
  5. ↑ Gallego, Guillermo (2022). "Event-based Vision: A Survey" (PDF). IEEE Transactions on Pattern Analysis and Machine Intelligence. 44 (1): 154–180. Bibcode:2022ITPAM..44..154G. doi:10.1109/TPAMI.2020.3008413. PMID 32750812. Retrieved 4 November 2025.
  6. ↑ Strukov, D. B.; Snider, G. S.; Stewart, D. R.; Williams, R. S. (2008). "The Missing Memristor Found". Nature. 453 (7191): 80–83. Bibcode:2008Natur.453...80S. doi:10.1038/nature06932. PMID 18451858. Retrieved 4 November 2025.
  7. ↑ Alan, Bawden; Richard, Greenblatt; Jack, Holloway; Thomas, Knight; David, Moon; Daniel, Weinreb (1977-08-01). "LISP Machine Progress Report". hdl:1721.1/5751. Archived from the original on 2026-02-18. {{cite journal}}: Cite journal requires |journal= (help)
  8. ↑ Maxfield, Max (24 December 2020). "Say Hello to Deep Vision's Polymorphic Dataflow Architecture". Electronic Engineering Journal. Techfocus media.
  9. ↑ "Kinara (formerly Deep Vision)". Kinara. 2022. Retrieved 2022-12-11.
  10. ↑ "Hailo". Hailo. Retrieved 2022-12-11.
  11. ↑ Lie, Sean (29 August 2022). Cerebras Architecture Deep Dive: First Look Inside the HW/SW Co-Design for Deep Learning. Cerebras (Report). Archived from the original on 15 March 2024. Retrieved 13 December 2022.
  12. ↑ Research, AI (23 October 2015). "Deep Neural Networks for Acoustic Modeling in Speech Recognition". AIresearch.com. Retrieved 23 October 2015.
  13. ↑ Kobielus, James (27 November 2019). "GPUs Continue to Dominate the AI Accelerator Market for Now". InformationWeek. Retrieved 11 June 2020.
  14. ↑ Tiernan, Ray (2019). "AI is changing the entire nature of compute". ZDNet. Retrieved 11 June 2020.
  15. ↑ "AI and Compute". OpenAI. 16 May 2018. Retrieved 11 June 2020.
  16. ↑ Lucian Armasu (2016-05-19). "Google's Big Chip Unveil For Machine Learning: Tensor Processing Unit With 10x Better Efficiency (Updated)". Tom's Hardware. Retrieved 2026-05-16.
  17. ↑ "NVIDIA H100 Tensor Core GPU Architecture". NVIDIA. 2022. Retrieved 4 November 2025.
  18. ↑ "AMD Instinct MI300X Accelerator". AMD. 2024. Retrieved 4 November 2025.
  19. ↑ https://spectrum.ieee.org/samsung-ai-memory-chips
  20. ↑ "Introducing Cloud TPU v5p and the AI Hypercomputer". Google Cloud Blog. 6 December 2023. Retrieved 4 November 2025.
  21. ↑ "Apple Neural Engine". Apple Machine Learning Research. Retrieved 4 November 2025.
  22. ↑ https://ieeexplore.ieee.org/abstract/document/9730377
  23. ↑ https://ieeexplore.ieee.org/abstract/document/9864008
  24. ↑ https://www.science.org/doi/10.1126/sciadv.adg3289
  25. ↑ https://venturebeat.com/ai/mythic-launches-analog-ai-processor-that-consumes-10-times-less-power
  26. ↑ https://www.wired.com/story/chip-ai-works-using-light-not-electrons/
  27. ↑ https://arxiv.org/abs/2503.11698
  28. ↑ https://developer.nvidia.com/blog/inside-nvidia-groq-3-lpx-the-low-latency-inference-accelerator-for-the-nvidia-vera-rubin-platform/
  29. ↑ https://www.theregister.com/systems/2026/08/06/amd-acquires-ai-chip-startup-taalas-to-boost-inference-performance-by-etching-models-into-silicon/5284344
  30. ↑ "JESD238A: High Bandwidth Memory (HBM3) Standard". JEDEC. January 2023. Retrieved 4 November 2025.
  31. ↑ "NVIDIA Hopper Architecture In-Depth". NVIDIA Developer Blog. 22 March 2022. Retrieved 4 November 2025.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Hardware_for_artificial_intelligence&oldid=1368088487"