Hardware/software co-design
Hardware/software co-design (often shortened as HW/SW co-design) is a system-level design methodology in which hardware and software components are developed together rather than in isolation. Its goal is to improve system-level metrics such as performance, power consumption, cost, and flexibility by trying out different ways of splitting functionality between hardware and software early in the design.[1][2]
Hardware/Software co-design is widely used in embedded systems,[3] cyber-physical systems,[4] and heterogeneous computing platforms,[5] where software execution and hardware acceleration need to work closely together.[3][6]
Overview
[edit]Hardware/software co-design is based on the principle that hardware and software constraints are considered at the same time, so that decisions on one side can be made with the other in mind. A function can be placed either in software on a processor or in dedicated hardware logic, with effects on performance, power, area, and cost at the same time. By keeping both sides open during early design, co-design explores combinations that a one-sided approach would miss.[1][2]
In co-design, the boundary between hardware and software is treated as a design variable. Computationally heavy, regular work is given a dedicated hardware implementation, while control logic and anything likely to change remains in software. The parameters that define this split, such as bit widths, buffer sizes, and the degree of parallelism, are shared between the two sides rather than fixed independently.[2][3][7]
History
[edit]Hardware/software co-design took shape in the late 1980s and early 1990s, when embedded systems were growing more complex and the usual practice of designing hardware and software separately began to break down. Many early embedded products, such as telecommunications equipment, industrial controllers, and consumer electronics, had tight performance and power budgets that could not be met by tuning the software alone.[8]
Early research looked at how to design digital hardware and embedded software at the same time, so that a given function could be placed either on a processor or in dedicated logic. Researchers developed formal methods for hardware/software partitioning, system-level specification, and performance estimation, and electronic design automation (EDA) tools started to support these combined workflows.[1][2]
During the 1990s, co-design became closely tied to embedded systems and system on chip design. As processors, memories, peripherals, and custom logic were integrated onto a single chip, designing the hardware and software together became necessary to meet cost, performance, and power targets.[8][9]
The increasing adoption of field-programmable gate arrays (FPGAs) in the 2000s widened the range of problems co-design could address. Unlike application-specific integrated circuits (ASICs), FPGAs can be reprogrammed after manufacturing, so the hardware can be modified alongside the software.[10] High-level synthesis (HLS) tools lowered the barrier further by letting parts of a program written in high level languages such as C and C++ be turned into digital hardware.[11]
More recently, hardware/software co-design is being adopted for artificial intelligence (AI), cloud computing, and heterogeneous computing. AI accelerators, data center architectures, and domain-specific processors can be built with co-design methods to raise performance-per-watt while keeping flexibility and development cost under control.[5][12]
Comparison with other methodologies
[edit]
Hardware/software development methodologies are commonly described as ranging from independent development to fully integrated co-design.[2][3]
Independent hardware/software design
[edit]
In the traditional model, hardware and software are developed largely on their own.[2][3] Hardware engineers build a target platform around architectural and manufacturing constraints, while software developers write applications on the assumption that the hardware interface will not change. The two teams communicate only through predefined instruction sets, programming interfaces, or hardware abstraction layers.[13]
Because the two sides are decoupled, each can proceed at its own pace. This works well for general-purpose computing platforms, desktop software, operating systems, and off-the-shelf hardware, where a single platform has to run a wide range of unrelated programs.[13]
A limitation of this approach is that the separation between hardware and software restricts cross-layer optimization. Software cannot exploit hardware capabilities that were not incorporated into the design, while the hardware cannot be tailored to the behavior of specific applications.[2][3]
Optimizing software for the hardware
[edit]
A common approach is to pick or build the hardware first, then tune the software to extract the most performance. The hardware is treated as fixed, and developers adapt their algorithms and code to make the best use of the available resources.[2][3]
Typical techniques include compiler optimizations, vectorization, cache-aware programming, parallel programming, instruction scheduling, and optimized memory layouts.[13] Additionally, algorithms can be rewritten to fit features such as single instruction, multiple data (SIMD) instructions, graphics processing units (GPUs), digital signal processors (DSPs), or specialized instruction set extensions.[6]
This is a common industrial approach for commercial processors, mainly because redesigning hardware once it is in production is very expensive. As a result, the software has to work within whatever the architecture provides, even when it limits the ideal implementation.[2][3][13]
Optimizing hardware for the software
[edit]
An alternative approach begins with a specific application or workload and then designs hardware to accelerate it. Here the software acts as the specification, and the hardware is built to match.[5] Typical applications include signal processing kernels, neural network inference, cryptographic algorithms, video encoding and decoding, and image processing pipelines.[5]
This approach has long been used in embedded and real-time systems, where low power and resource budgets make custom hardware worthwhile.[2][3] Building hardware around a known workload can give better performance-per-watt than a general-purpose processor. The same idea has since moved into large-scale cloud computing and data centers, where AI accelerators such as tensor processing units (TPUs) are used to reduce power and raise performance-per-watt.[12]
Hardware/software co-design
[edit]
Co-design differs from the three approaches above in that neither side is fixed first. Hardware and software are treated as an iterative, collaborative effort, and both are refined together against shared goals.[2][3] Starting from a system-level specification, alternative hardware/software partitions are evaluated against metrics such as performance, power consumption, area, and cost. The process is typically iterative, with the results of each evaluation guiding subsequent design decisions.[2]
Applications
[edit]Hardware/software co-design is widely used in systems that combine general-purpose processors with application-specific hardware. Typical application domains include:
- Embedded systems, where power consumption, chip area, and manufacturing cost are primary design constraints.[3]
- Automotive and aerospace, where safety-critical control loops must satisfy real-time requirements.[4]
- Telecommunications and digital signal processing, where functions such as filtering, modulation, and error correction are commonly implemented in dedicated hardware.[3]
- Computer vision and image processing, where computationally intensive pixel-processing kernels are often accelerated in hardware.[5]
- Artificial intelligence accelerators, including TPUs, neural processing units (NPUs), and GPU tensor cores, which are developed together with the software frameworks and numerical formats they support.[5][12]
- Data centers and cloud computing, where FPGA- and ASIC-based accelerators are used for networking, storage, database, and machine-learning workloads.[5][14]
- Scientific computing, where application-specific accelerators are used for computational kernels such as stencil computations, molecular dynamics, and sparse linear algebra.[5][15]
Design workflow
[edit]
A hardware/software co-design flow typically begins with a system-level specification, followed by hardware/software partitioning, design space exploration, and performance estimation. These activities are generally performed iteratively, as the results of one stage often influence decisions made in the others.[2][7]
Partitioning
[edit]Partitioning decides, for each function or kernel, whether it runs as software on a processor or as a dedicated hardware block. Kernels built from regular, data-parallel arithmetic, such as convolution, matrix multiplication, or fast Fourier transform (FFT) stages, are good candidates for hardware. Control-heavy code, rare operations, and anything expected to change over the product's life are typically left in software.[2][7]
Partitioning is normally treated as a multi-objective optimization problem that trades off execution time, power, silicon area, memory bandwidth, and development effort. Reported approaches include integer linear programming, graph partitioning, evolutionary algorithms, simulated annealing, and more recently machine learning heuristics.[7]
Design space exploration
[edit]The number of possible partitions and hardware configurations grows very quickly, so design space exploration (DSE) is used to narrow things down. DSE tools vary parameters such as accelerator count, memory hierarchy, interconnect topology, and clock frequency, then score analyzed configurations against the system goals.[5]
These scores can be computed through analytical models, instruction-set simulators, transaction-level SystemC models, and FPGA prototypes. There is a trade-off between accuracy and speed: rough analytical models can evaluate thousands of options quickly, while slow but accurate cycle-level simulation and FPGA prototyping are saved for a short list of promising designs.[16]
| Method | Relative speed | Relative accuracy | Typical use |
|---|---|---|---|
| Analytical models | Very fast | Low | Early screening of many configurations |
| Instruction-set simulation | Medium | Medium | Evaluation of candidate processors |
| Cycle-accurate or SystemC | Slow | High | Detailed evaluation of selected configurations |
| FPGA prototyping | Slow to set up, fast to run | Very high | Final validation |
Co-design choices
[edit]Beyond the basic "hardware or software" question, co-design involves a set of joint choices that are relevant to both hardware and software. Many of them rely on operations that are cheaper in hardware than in software, such as shifts and masks, bit-level parallelism, and arithmetic with a fixed, known latency.[2]
| Choice | Hardware effect | Software implications |
|---|---|---|
| Reduced precision | Reduced arithmetic cost and memory bandwidth | Numerical algorithms adapted to reduced precision (like quantization-aware training for AI) |
| Loop tiling | Improved locality through on-chip memory reuse | Loop structure and data access patterns matched to the hardware organization |
| Processing-element count | Degree of hardware parallelism | Parallelism exposed through vectorization, threading, or loop transformations |
| Fixed versus programmable | Trade-off between efficiency and flexibility | Flexibility to accommodate evolving algorithms and workloads |
| Data layout | Efficient memory access and accelerator interfaces | Data structures organized to match the hardware memory layout |
Numerical precision and quantization
[edit]General-purpose processors offer only a small number of standard number formats, typically 32- and 64-bit floating point and 8- to 64-bit integers. Custom hardware is not tied to these and can use any bit width, fixed-point arithmetic, or reduced-precision floating point such as bfloat16, FP16, FP8, or block floating point. Narrower operands reduce multiplier area roughly with the square of the width and cut memory bandwidth in proportion, which is an important consideration in neural network accelerators.[12][17]
Reducing numerical precision changes the results of arithmetic operations, so software algorithms may need to be adapted accordingly. For neural network accelerators, this typically involves quantization-aware training, calibration, or other techniques that maintain model accuracy at the target precision. The choice of numerical precision therefore becomes a shared design decision between the hardware implementation and the software model.[12][17]
Loop tiling and memory hierarchy
[edit]On-chip memory provides substantially higher bandwidth and lower energy consumption than off-chip DRAM, but its capacity is limited. In hardware/software co-design, tile and blocking sizes are selected so that the application's working set fits within on-chip scratchpad memories or caches. Tile dimensions, on-chip memory capacity, and the number of parallel processing elements are therefore determined together. For example, in a matrix multiplication accelerator, the dimensions of the systolic array, the sizes of the on-chip buffers, and the software's tiling strategy are derived from the same blocking parameters.[5][13][17]
Parallelism and processing-element count
[edit]The number of processing elements (PEs), SIMD lanes, or accelerator instances is chosen according to the available parallelism in the workload. In many designs, selecting a power-of-two configuration simplifies address decoding, reduction trees, and interconnect design, and aligns with tile sizes used in data partitioning. Software kernels are typically structured to expose this level of parallelism, using techniques such as vectorization, loop unrolling, or compiler-driven transformations.[7]
Fixed-function versus programmable acceleration
[edit]Co-design also determines the degree of flexibility required in an accelerator. A fully fixed-function implementation minimizes area and power consumption but supports only a single operation. A programmable engine with a domain-specific instruction set trades reduced efficiency for the ability to support changes in algorithms without redesigning the hardware. Reconfigurable platforms such as FPGAs occupy an intermediate position, allowing the hardware implementation to be modified after fabrication alongside software changes.[5]
Hardware/software interface
[edit]The interface between the processor and an accelerator is a key factor in determining overall system performance and is typically designed jointly with the target computation. Common mechanisms include memory-mapped registers, shared memory, direct memory access (DMA) engines, interrupts, and streaming FIFOs. When substantial amounts of data are transferred between processor and accelerator, data movement and cache coherence overhead can reduce or eliminate the performance benefits of acceleration.[5][11]
Example
[edit]
One example of a hardware/software co-design decision is choosing parameters that are powers of two. In digital hardware, multiplication or division by a constant power of two can be implemented as a constant bit shift, while the remainder after division by a power of two can be computed with a bitwise AND mask, operations that require significantly less hardware logic.[13][18] Similarly, addressing circular buffers or memories whose size is a power of two can be simplified into selecting the lower address bits instead of performing a modulo operation.[13][18]
When a hardware implementation exploits such properties, corresponding constraints are imposed on the software. Array dimensions, buffer sizes, tile sizes, or loop bounds may be selected as powers of two, with unused elements handled through padding or masking. In this way, numerical parameters become shared design choices that influence both the hardware implementation and the software that executes on it.[2]
Challenges and limitations
[edit]The combined hardware/software design space is typically very large, making automated partitioning and design space exploration important tools, although practical design decisions involve human guidance and domain knowledge.[7][16] Verification is harder as well, because the hardware, the software, and their interaction all have to be checked at once. Implementation errors can be in timing, concurrency, or interface mismatches that would not appear in either domain alone.[16]
Performance benefits may be reduced or eliminated if data movement between processors and accelerators is not carefully considered during design, since communication overhead can dominate computation time in some workloads.[12]
In ASIC design, co-design can lead to long development cycles due to the cost and time required for hardware fabrication, although reconfigurable platforms such as FPGAs mitigate this limitation by allowing post-fabrication updates.[10]
Finally, tightly coupled hardware/software systems can be more difficult to maintain and evolve, since changes in software may require corresponding changes in hardware assumptions such as data widths, memory layouts, or parallelization structure.[1]
Tools and design automation
[edit]Because the hardware/software design space is typically very large, hardware/software co-design depends heavily on electronic design automation tools. System-level modeling languages and hardware description languages are used to represent both hardware and software within a unified framework, while high-level synthesis enables hardware generation from high-level programming languages such as C and C++.[11] Simulation tools, SystemC-based models, and FPGA prototypes are commonly used to evaluate design choices and guide design space exploration.[7]
Current research in hardware/software co-design is largely focused on improving automation in the design process and extending co-design methodologies to emerging computing platforms. Active areas of work include machine-learning-assisted partitioning and design space exploration,[19] compiler-assisted accelerator generation, runtime reconfiguration for FPGAs,[20] and energy-aware optimization strategies for both edge and data-center systems.[5][7]
More recently, co-design techniques have also been applied to AI accelerators and other domain-specific computing architectures, including systems targeting large language models and generative AI workloads. In these settings, the emphasis is increasingly on improving energy efficiency and resource utilization in both edge and data-center deployments.[5][21]
See also
[edit]References
[edit]- 1 2 3 4 De Micheli, Giovanni; Sami, Mariagiovanna (1996). Hardware/software co-design.
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 De Michell, G.; Gupta, R.K. (1997-03-31). "Hardware/software co-design". Proceedings of the IEEE. 85 (3): 349–365. Bibcode:1997IEEEP..85..349D. doi:10.1109/5.558708.
- 1 2 3 4 5 6 7 8 9 10 11 12 Wolf, W.H. (1994-07-31). "Hardware-software co-design of embedded systems". Proceedings of the IEEE. 82 (7): 967–989. Bibcode:1994IEEEP..82..967W. doi:10.1109/5.293155. ISSN 1558-2256.
- 1 2 Zhou, Yuchen; Baras, John; Wang, Shige (2016). "Hardware Software Co-design for Automotive CPS using Architecture Analysis and Design Language". arXiv:1603.05069 [cs.SE].
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Cong, Jason; Fang, Zhenman; Huang, Muhuan; Wei, Peng; Wu, Di; Yu, Cody Hao (2019-01-01). "Customizable Computing—From Single Chip to Datacenters". Proceedings of the IEEE. 107 (1): 185–203. doi:10.1109/JPROC.2018.2876372. ISSN 0018-9219.
- 1 2 Buchty, Rainer; Heuveline, Vincent; Karl, Wolfgang; Weiss, Jan-Philipp (2012). "A survey on hardware-aware and heterogeneous computing on multicore processors and accelerators". Concurrency and Computation: Practice and Experience. 24 (7): 663–675. doi:10.1002/cpe.1904. ISSN 1532-0634.
- 1 2 3 4 5 6 7 8 Hou, Neng; Yan, Xiaohu; He, Fazhi (2019-06-01). "A survey on partitioning models, solution algorithms and algorithm parallelization for hardware/software co-design". Design Automation for Embedded Systems. 23 (1–2): 57–77. doi:10.1007/s10617-019-09220-7. ISSN 0929-5585.
- 1 2 Wolf, Marilyn (2012). Computers as components: principles of embedded computing system design (3rd ed.). Waltham, MA: Elsevier/Morgan Kaufmann. ISBN 978-0-12-388436-7.
- ↑ De Micheli, Giovanni (1994). Synthesis and optimization of digital circuits. McGraw-Hill series in electrical and computer engineering. New York: McGraw-Hill. ISBN 978-0-07-016333-1.
- 1 2 Compton, Katherine; Hauck, Scott (2002-06-01). "Reconfigurable computing: a survey of systems and software". ACM Computing Surveys. 34 (2): 171–210. doi:10.1145/508352.508353. ISSN 0360-0300.
- 1 2 3 Cong, Jason; Liu, Bin; Neuendorffer, Stephen; Noguera, Juanjo; Vissers, Kees; Zhang, Zhiru (2011-03-22). "High-Level Synthesis for FPGAs: From Prototyping to Deployment". IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. 30 (4): 473–491. Bibcode:2011ITCAD..30..473C. doi:10.1109/TCAD.2011.2110592. ISSN 0278-0070.
- 1 2 3 4 5 6 Jouppi, Norman P.; Young, Cliff; Patil, Nishant; Patterson, David; Agrawal, Gaurav; Bajwa, Raminder; Bates, Sarah; Bhatia, Suresh; Boden, Nan; Borchers, Al; Boyle, Rick; Cantin, Pierre-luc; Chao, Clifford; Clark, Chris; Coriell, Jeremy (2017-06-24). "In-Datacenter Performance Analysis of a Tensor Processing Unit". SIGARCH. ACM: 1–12. Bibcode:2017syca.conf....1J. doi:10.1145/3079856.3080246. ISBN 978-1-4503-4892-8.
{{cite journal}}: CS1 maint: periodical has ISBN (link) - 1 2 3 4 5 6 7 Hennessy, John L.; Patterson, David A. (2019). Computer architecture: a quantitative approach (Sixth ed.). Cambridge, MA: Morgan Kaufmann Publishers. ISBN 978-0-12-811905-1.
- ↑ Putnam, Andrew; Caulfield, Adrian M.; Chung, Eric S.; Chiou, Derek; Constantinides, Kypros; Demme, John; Esmaeilzadeh, Hadi; Fowers, Jeremy; Gopal, Gopi Prashanth; Gray, Jan; Haselman, Michael; Hauck, Scott; Heil, Stephen; Hormati, Amir; Kim, Joo-Young (2015-05-13). "A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services". IEEE Micro. 35 (3): 10–22. Bibcode:2015IMicr..35c..10P. doi:10.1109/MM.2015.42. ISSN 0272-1732.
- ↑ Khan, M. A.; Chiu, M.; Herbordt, M. C. (2013). "FPGA-Accelerated Molecular Dynamics". In Vanderbauwhede, Wim; Benkrid, Khaled (eds.). High-Performance Computing Using FPGAs. Springer New York. pp. 105–135. doi:10.1007/978-1-4614-1791-0_4. ISBN 978-1-4614-1790-3.
- 1 2 3 Gajski, Daniel D.; Abdi, Samar; Gerstlauer, Andreas; Schirner, Gunar (2009). Embedded System Design: Modeling, Synthesis and Verification. Boston, MA: Springer US. doi:10.1007/978-1-4419-0504-8. ISBN 978-1-4419-0503-1.
- 1 2 3 Sze, Vivienne; Chen, Yu-Hsin; Yang, Tien-Ju; Emer, Joel S. (2017-11-20). "Efficient Processing of Deep Neural Networks: A Tutorial and Survey". Proceedings of the IEEE. 105 (12): 2295–2329. doi:10.1109/JPROC.2017.2761740. ISSN 0018-9219.
- 1 2 Brown, Stephen D.; Vranesic, Zvonko G. (2014). Fundamentals of digital logic with Verilog design (3rd ed.). New York: McGraw-Hill Higher Education. ISBN 978-0-07-338054-4.
- ↑ Huang, Guyue; Hu, Jingbo; He, Yifan; Liu, Jialong; Ma, Mingyuan; Shen, Zhaoyang; Wu, Juejian; Xu, Yuanfan; Zhang, Hengrui; Zhong, Kai; Ning, Xuefei; Ma, Yuzhe; Yang, Haoyu; Yu, Bei; Yang, Huazhong (2021-09-30). "Machine Learning for Electronic Design Automation: A Survey". ACM Transactions on Design Automation of Electronic Systems. 26 (5): 1–46. doi:10.1145/3451179. ISSN 1084-4309.
- ↑ Mühl, Gero; Gesellschaft für Informatik, eds. (2012). ARCS workshops (ACRS), 2012: 28 - 29 Feb. 2012, München, Germany ; co-located with the Conference on Architecture of Computing Systems. Piscataway, NJ: IEEE. ISBN 978-1-4673-1913-3.
- ↑ Guo, Cong; Cheng, Feng; Du, Zhixu; Kiessling, James; Ku, Jonathan; Li, Shiyu; Li, Ziru; Ma, Mingyuan; Molom-Ochir, Tergel; Morris, Benjamin; Shan, Haoxuan; Sun, Jingwei; Wang, Yitu; Wei, Chiyue; Wu, Xueying (2025). "A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models". IEEE Circuits and Systems Magazine. 25 (1): 35–57. doi:10.1109/MCAS.2024.3476008. ISSN 1531-636X.
External links
[edit]- SystemC – SystemC language homepage
- Vitis HLS – AMD Vitis HLS introductory page
- Intel openAPI – Intel openAPI introductory page