2.1 Hardware Acceleration Methods for Embedded Systems (FPGA)

FPGAs are greatly flexible and customizable to meet the specific requirements of different applications. This flexibility allows FPGAs to be optimized either to perform high-performance parallel processing and data streaming in applications that require high throughput, or for fast response times in applications that require low latency, or finally for energy efficiency in applications that require low-power consumption.

The traditional development of FPGA-based applications has been mainly based on highly specialized register transfer level (RTL) designs [⁴⁴, ¹⁸, ⁴², ⁵, ¹³, ³⁴, ²⁰]. High-Level Synthesis (HLS) allows increasing design productivity and detaching the algorithm from architecture [¹⁶]. In our case, the vivado HLS tool is used to generate hardware accelerators from C language.

Ben Jmaa et al. [⁹] proposed an efficient hardware implementation for different sorts using high-level descriptions in a zynq-7000 platform. The authors compared the performance of the algorithms in terms of computational time, standard deviation and resource utilization. The results showed that the selection-sort was 1.01-1.23 times faster than other algorithms for less than 64 elements; otherwise, tim-sort was the best algorithm. Kobayashi et al. [²⁶, ²⁷] detailed a new approach to reducing FPGA programming costs while maintaining high levels of performance.

Their sorting library can use OpenCL for FPGA. This approach consumed at least twice the hardware resources of the merge-sort method restructured for the OpenCL programming model for FPGA. However, it operated at a frequency 1.08 times higher and had a sorting throughput three orders of magnitude greater than the baseline.

Chen et al. [¹⁴] proposed a sample-sort algorithm on a server with a PCIe-connected FPGA to sort large data sets. The prototype system was implemented using Verilog HDL on Amazon Web Services (AWS) FPGA instances equipped with Xilinx Virtex UltraScale+ FPGAs.

The authors demonstrated that this system can sort 230 key-value records 37.4 times faster than GNU parallel sort running on a CPU with 8 threads. However, their method assumed collaboration between the CPU and FPGA, and the sorting performance was ultimately limited by the PCIe bandwidth, which was 7.2 GB/s as reported in [¹⁴]. Moreover, their method was not suitable for implementing FPGA-centric applications that require sorting due to its structure.

Shinyamada et al. [³⁸] evaluated the impact of various sorts on high-level synthesized image processing hardware. The results showed that bubble-sort and odd-even-merge-sort were the fastest algorithms, as they were able to achieve pipeline processing. Conversely, selection-sort was not able to achieve ideal pipeline processing, and its performance was not as good as the former two algorithms. Concludingly, optimizing sorts can have a significant impact on overall image processing performance.

Moghaddamfar et al. [³²] conducted a comparative analysis of OpenCL and RTL-based implementations of a heap-sort that merges sorted runs. The results showed that while both implementations required comparable development effort, their RTL implementations of critical primitives achieved four times better performance and used only half as much FPGA resources compared to the OpenCL implementation.

This highlighted the importance of carefully selecting the programming language and implementation approach when developing algorithms for FPGA-based systems. The study suggested that RTL-based implementations can provide significant performance benefits and resource efficiency compared to higher-level language implementations like OpenCL.

Chen et al. [¹⁵] proposed a novel hybrid pipelined sorting architecture based on a bitonic-sorter and several cascaded sorting units. This sorting architecture achieved a balanced trade-off between resource utilization and throughput, as well as between throughput and power consumption.

Specifically, the architecture was both resource and energy efficient in terms of the throughput-to-resource ratio and the throughput-to-power ratio. This study highlighted the importance of designing sorting architectures that maximize data parallelism to achieve increased throughput and reduced latency.

Montesdeoca et al. [³³] monitored by a network of 40 CO₂ sensors and performed real-time sorting of all the data via bubble-sort and insertion-sort on FPGA. The results showed that insertion-sort was faster than bubble-sort, but it consumed more hardware resources in the FPGA, illustrating the importance of the trade-off between speed and resource utilization when selecting a sort for real-time applications.

In [³], the authors evaluated multithreaded sorts on a 32-core reconfigurable architecture with embedded real-time Linux support. The architecture consisted of NIOS II/f soft cores and was implemented on an FPGA. The authors proposed a new approach for performance evaluation of a soft multithreaded multicore architecture conducted in real-time.

This approach was based on the recursive generation and execution of sorts such as merge-sort and quick-sort. The architecture was capable of achieving high parallelism and throughput while maintaining low latency. The architecture outperformed the others in terms of speed and scalability. In [¹], the authors focused on hardware implementing bubble-sort, selection-sort, insertion-sort, merge-sort, bitonic-sort and odd-even-merge-sort using FPGA in synchronous and pipelined architectures. The authors compared these implementations in terms of computational time and area.

They showed that non-pipelined bitonic-sort and non-pipelined odd-even-merge-sort had the best performance in terms of computational time, while the non-pipelined selection-sort and non-pipelined insertion-sort had the lowest area of synchronous architecture.

For pipelined architectures, bitonic-sort and odd-even-merge-sort had much lower computational time when implemented in hardware. Additionally, odd-even-merge-sort was found to be the smallest in terms of area. While bitonic-merge-sort was slightly larger in area and slower in execution than odd-even-merge-sort.

Lobo et al. [³¹] compared five merge-sorts (serial-merge-sort, parallel-merge-sort, bitonic-merge-sort, odd-even-merge-sort and the modified-merge-sort) in terms of resource utilization, delay and area on FPGA. The results showed that the serial and parallel merge use the highest amount of resource utilization compared to bitonic-merge, odd-even-merge and modified-merge.

Also, the parallel-merge algorithm was much faster than a serial-merge algorithm. In addition, the odd-even and modified-merge had a very close value of the area used while bitonic-merge had as slightly higher value.

Abdelrasoul et al. [²] proposed an index and sort algorithm (IaSA) based on an FPGA (vertex-5 series) in a pipelined sequential structure using Verilog HDL. The results showed that, for various data set sizes, IaSA performed best in terms of computational time.

In [²⁴], the authors presented a column-sort, mapped on an HBM (High Bandwidth Memory)-enabled FPGA. The approach utilized computational pipelines, hardware-efficient interconnection networks and several optimizations to achieve high-throughput sorting. The results showed that their optimized design yields 14.8×, 4.73× and 2.18× speedup compared with state-of-the-art implementations on CPU.

CI(s) is compared with those of previously ranked sorts and used to assess whether a partial or a total ranking of the sorts is possible when inserting and ranking s in S′.

2.2 Software Acceleration Methods for Embedded Systems (ARM)

ARM processors offer advantages in terms of flexibility and ease of reconfigurable integration technology for a wide range of applications, from embedded systems to high-performance computing.

Compared to classical processors, soft-core processors like ARM allow for greater customization and adaptability because they can be easily programmed and reconfigured to meet the specific needs of a given application.

Additionally, soft-core processors can be integrated into a variety of reconfigurable technologies, such as FPGAs, allowing for even greater flexibility and performance optimization.

In [¹¹, ¹⁰], the authors proposed a software implementation for insertion-sort, quick-sort, heap-sort, shell-sort, merge-sort, and tim-sort on an ARM Cortex A9. They compared the performance of these algorithms in terms of average and standard deviation of computational times, energy consumption, and temporal stability.

The results demonstrated that shell-sort was the best algorithm, being 42.1% faster and even reaching up to 72% faster when the number of elements to be sorted is greater than 64.

However, when the number of elements is smaller than 64, tim-sort was the best algorithm. Additionally, shell-sort was the best algorithm in terms of standard deviation of computational times and energy consumption.

In [⁷], the authors proposed a hardware heap-sort implementation using FPGA of a wavelet based image coder. Their architecture provided up to 20.9% power reduction on the memories compared to the baseline implementation. Moreover, their architecture provided 13x speedup compared to ARM Cortex A9.

In [⁸], the authors introduced an adaptive heap-sort that was designed for an image coding implementation on FPGA with high throughput and scalable sorting. The authors compared its performance to an embedded ARM Cortex A9 running at 666 MHz.

Their architecture, running at 100 MHz, provided around 13 times the speedup while consuming 242 mW of average core dynamic power. In [¹²], the authors adapted a hybrid sort based on quick-sort and bitonic-sort. They employed bitonic-sort to handle small partitions/arrays with a vectorized partitioning implementation to divide these partitions.

Their approach required only an array of size O(log n) for recursive calls in the partitioning phase. They evaluated the performance on an ARM v8.2 (A64FX) and assessed their implementation by sorting/partitioning integers, double floating-point numbers, and key/value pairs of integers. The results showed an average speedup factor of four compared to the GNU C++ sort algorithm.

In [²⁵], the performance and energy efficiency of hardware and software implementations of the heap-sort are compared. The results showed that the hardware implementation (Digilent Basys 3 Artix-7 FPGA) was more energy efficient, but slower than software implementation (ARM Cortex A72) due to a low clock frequency.

2.3 Software Acceleration Methods for Non Embedded CPU

In “standard” workstations, Central Processing Units (CPUs) can drastically increase the number of instructions processed per second, allowing the computer to perform more complex tasks or run more programs simultaneously. Indeed, recent CPUs have a potential for higher performance thanks to higher clock speeds, more cores, and improved SIMD instructions. However, these processors consume much more power than their embedded counterparts.

In [³⁰], the authors presented optimized serial and parallel counting-sorts. They compared this sort to others such as bucket-sort and merge-sort, implementing both counting-sort and merge-sort on CPU and GPU. The results showed that the optimized counting-sort took only 6 ms to sort 100 million integers, being 23 times faster than the previous version.

In [²¹], the authors provided an analysis of current host-GPU data transfer mechanisms and explored methods for mitigating performance bottlenecks. They developed a heterogeneous CPU/GPU sort and demonstrated that while out-of-place GPU sorting achieved the best performance, an in-place sort further reduced some host-side bottlenecks.

In [⁴], the authors compared the grouping-comparison-sort (GCS) to selection-sort, quick-sort, insertion-sort, merge-sort and bubble-sort, using random input sequences. On an Intel Core 2 Duo E8400 @ 3.00 GHz (2 CPUs), the result revealed that for small input sizes, the performance of all six algorithms was almost comparable.

However, for larger input, quick-sort proved to be the fastest, while selection-sort was the slowest. GCS ranked as the third fastest for small input size (10000 elements) and the fifth fastest for large input size (30000 elements). In [²⁸], the authors introduced a distribution sorting method that utilized a trained model of the empirical Cumulative Distribution Function of the data.

Additionally, they applied a deterministic sort that performed well on almost sorted arrays, such as insertion-sort. The performance was measured on an Intel Xeon Gold 6150 @ 2.70 GHz using up to one billion double-precision keys following a normal distribution.

Their approach achieved an average performance improvement of 3.38 times compared to the C++ STL-sort, which is an optimized hybrid of quick-sort, a 1.49 times improvement over sequential radix-sort, and a 5.54 times improvement over a C++ implementation of tim-sort, which is the default sorting function for Java and Python.

2.4 Synthesis of Related Work

Table 1 summarizes a literature review focusing on the use of several sorts on different platforms (FPGA, Intel CPU, ARM). This summary shows that the authors generally use sorts on FPGA to improve performance in terms of computational time and resource usage. Moreover, a majority of authors do not use HLS except for [²⁶, ²⁷, ³⁸, ⁹, ¹¹, ¹⁰]. In addition, the number of elements to sort is usually much larger than thousands of items. Contrary to these studies, our work is based on different implementations of sorts using HLS and compare the sorts on FPGA and Intel i7 in terms of computational time, resources utilization and temporal stability.

It is worth noting that while many authors are interested in the first two criteria, the number of papers concerning the stability of algorithms is much lower. Moreover, since an avionics application is targeted, deterministic sorting is required and the use recursive functions or dynamic memory allocations are forbidden contrary to many applications in the literature.

Similarly, parallel versions of sorts are not allowed because it is not possible to certify such algorithms on standard multicore architectures in our target avionics application.

A finding from this synthesis is that beyond its inherent complexity, the “best sort” depends on the number of elements to be sorted, the target architecture, the parallelization mode and the considered key performance indicators. In the following sections, we compare software and hardware implementations of sorts on Intel i7 and FPGA.

3.1 Design of Experiments

In order to evaluate the performances of sorts, the average and standard-deviation of their computational times are studied using n=8 to 4096 elements n∈N= {8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096}, via P=47 permutations of n integers generated using Lehmer’s method [¹⁷]. For each value of n, this set of permutations Pn is used to characterize the temporal variations due to a sort s in itself, i.e. its temporal stability.

For each permutation πn of size n, R=1000 replications are used to identify the external variations coming from the “computational environment” (e.g., transmission error, operating system noise, I/O buffering). To further evaluate the stability of sorts, the Relative Standard Deviation (RSD) is calculated by dividing the standard deviation σ by the average of computational times μ and expressed as a percentage:

It is a relative measure of the dispersion of data around the average. Ultimately, this ratio is used to compare the degree of variation from one sample to another, even if the means are different. For a sufficiently large number of items in a population (empirically ≥ 30), an RSD of less than 1% is considered “excellent” to make the average representative.

From a practical point of view, an RSD of 5% is generally considered acceptable. However, if RSD is greater than 5%, it is advisable to use statistical tests, possibly supplemented by graphical representations such as boxplots. Our study relies on two measures of RSD:

For each permutation πn of size n, RSDR(s,πn) is computed over the set of R replications of the same computational time measurement (i.e. for a given sort s, sorting the same permutation πn).

Then, for each sort s, RSDP(s) is computed over the set Pn of P permutations, considering – for each permutation πn – the average computational time μR(s,πn) over R replications of the same computational time measurement. To reduce RSDR(s,πn), the operating system (Debian Linux 12.4 with “processor affinity for RT-tasks” based on kernel version 6.1.0-17) has been configured to avoid most of the OS’s noises, a disruption may however occur due to potential non-maskable interruptions and unavoidable waiting times for external events or resource availability.

The values of RSDR(s,πn) are not detailed for the sake of conciseness. R has been chosen so that RSDR(s,πn)≤5% in “almost all cases” (i.e. for the large majority of the sorts and values of n). However, due to above-mentioned external variations, this target is not reached for all sorts with n=8 on i7 and FPGA as well as for quick-sort and tim-sort with n=16 on i7.

Nonetheless, it should be noted that the average RSDR(s,πn) over all sorts s and all permutations πn is approximately equal to 1.2% which is far less than the expected limit of 5%. In our design of experiments, RSDP(s) is computed for each sort s to assess the representativeness of its average computational time.

Actually, the stability of sorts in terms of computational time is assumed significant if RSDP(s)≤5%, otherwise statistical tests are performed. Moreover, in order to rank the sorts, confidence intervals CI(s) are computed as follows:

In addition to the focus on a single sort, boxplots [⁴¹] are useful tools for visualizing and comparing distributions of computational time measurements on the same scale. Specifically, boxplots allow us to compare and analyze the stability of different algorithms.

Standard boxplots are usually based on five values that summarize the data (Q0 (min), Q1 (first quartile), Q2 (median), Q3 (third quartile), and Q4 (max)) for the studied population (here, 47 permutations). As usual, IQR is also defined as the difference between the third and first quartiles. All observations above Q3+1.5×IQR or below Q1−1.5×IQR are considered as outliers.

On the basis of the number of outliers # outliers, this allows us to compute the percentage of outliers as follows:

Upper outliers (denoted by #⁺) are defined as all observations above Q3+1.5×IQR. Similarly, %⁺outliers is the percentage of upper outliers. Each set of P experiments (on P distinct permutations) of a sort s may be represented by a boxplot and summarized by Q0(s) to Q4(s).

Supplemented by the outliers, σP(s), μP(s) and RSDP(s), it is possible to obtain a precise view of the temporal stability of a sort. RSDP(s)<5% and/or a small value of IQR(s) suggest(s) that the sort s is stable, this is usually confirmed by a small value of %outliers(s).

On the contrary, RSDP(s)>5% and/or a large value of IQR(s) and/or a significant difference between Q2(s) and μP(s) suggest(s) that the sort is not stable. In the context of real-time applications, particular attention should be payed to the percentage of upper outliers (%⁺outliers) since it provides information on worst-case computational times of a given sort.

In other words, depending on input data this sort may reach computational times that are not compatible with the targeted time-constraints.

Moreover, by grouping several boxplots (one per sort) in the same plot, it is possible to visually compare the performance and temporal stability of the sorts. In addition to μP(s) and RSDP(s), a significant variation of Q2(s) between different distributions (i.e. sets of P permutations, one set per sort s), suggests that the sorting times vary and it is possible to rank the sorts.

In contrast, if the medians are quite similar across several distributions, σP(s), μP(s) and/or boxplots are not sufficient to rank the sorts and statistical tests are required. Consequently, it should have been interesting to present boxplots and statistical tests for all sorts on all target architectures, however for the sake of conciseness, numerical results have been reduced to μP(s), σP(s), RSDP(s) and %outliers(s) when/where sufficient. The results of statistical tests are also summarized when needed.

In our design of experiments, the following statistical tests are performed: First of all, to assess the normality of the data distributions, the Shapiro-Wilk test is conducted on each set of P experiments and each value of n. As a result of these tests, it appears that the data deviate significantly from a normal distribution, consequently nonparametric tests must be used.

So, in a second step, the Kruskal-Wallis test is employed to examine the overall differences in computational times among the five sorts on each platform. Subsequently, to refine the results and rank the sorts as far as statistically possible, pairwise comparisons using Wilcoxon tests are conducted to identify specific algorithm pairs that exhibited significant differences.

For a given architecture, if all tests are performed on the η=|N| values of n, this leads to η×Σ Shapiro-Wilk tests, η Kruskal-Wallis tests and η×(Σ×(Σ−1))/2 Wilcoxon tests. This makes a total of – at most – 160 statistical tests (with η=10 and Σ=5) performed via several scripts written in R language [²³].

The effective number of pairwise comparisons can be reduced if it is guided by the ranking of sorts by μP. All p-values are adjusted using the Bonferroni correction method so as to obtain an overall α level set to 5%.

For a given architecture (i7 or FPGA) and a value of n, our DOE follows several steps described in a simplified way as a pseudo-code in Algorithm 1, supplemented by some comments about main lines of the algorithm.

Algorithm 1 Main steps of our DOE 

As shown in previous paragraphs, some steps of Algorithm 1 are dedicated to check the stability of the computational environment while others are used to assess the temporal stability of each sort and the others deal with the total or partial ranking of the sorts on the basis of their computational times. Finally, it is important to note that the resulting ranking in S′ is not necessarily the same when n varies.

3.2 Performances Study of Sorting Algorithms on i7

This section illustrates our DOE to compare and analyze the sorts on an Intel i7-9850H @ 2.60 GHz. All codes are written in C language and compiled via gcc version 12.2.0-14 (on Linux Debian 12.4) with the O3 optimization flag turned on. The sorts are evaluated in terms of computational times. They are then rated on the basis of their temporal stability.

In more details, the first step of our DOE computes RSDR(s) to check for external variations from the computational environment (lines 5 to 8 of Algorithm 1). Then, subsection 3.2.1 illustrates the second step of Algorithm 1 (lines 10 to 12). In a similar way, subsection 3.2.2 follows the third step of Algorithm 1 (lines 13 to 17). Finally, subsection 3.2.3 is dedicated on the last step of Algorithm 1 (lines 18 to 20).

We detail each of the steps 2 to 4 of Algorithm 1 in a separate subsection to illustrate how it works and “its” results in several tables, but in reality the steps follow one another in the algorithm to lead if necessary to statistical tests and temporal stability analysis.

3.3 Performances Study of Sorting Algorithms on FPGA

As for i7 in section 3.2, this section illustrates our DOE to compare and analyze the sorts on FPGA. All codes are written in C language and the optimized hardware implementation is generated using HLS directives (loop unrolling, loop pipelining, input/output interface). Vivado is used for synthesis and running the VHDL architecture. The sorts are evaluated in terms of computational times and temporal stability.

On FPGA our DOE follows the steps of Algorithm 1: The first step of our DOE computes RSDR(s) to check for external variations from the computational environment (lines 5 to 8 of Algorithm 1). Then, subsection 3.3.1 illustrates the second step of Algorithm 1 (lines 10 to 12).

Contrary to i7, the values of RSDP(s) on FPGA are such that there is no need to use statistical tests and the third step of Algorithm 1 is “skipped”. Finally, subsection 3.3.2 is dedicated to the last step of Algorithm 1 (lines 18 to 20). In the following subsections, steps 2 and 4 of Algorithm 1 are detailed to illustrate how it works and “its” results in several tables.

3.4 Comparison of the Computational Platforms

Despite a much lower frequency, computational times on FPGA are “respectable” compared to those on i7 because Xilink’s tool is able to extract the parallelism of the algorithms by means of the optimizations introduced via HLS directives. However, a decrease in terms of average and standard deviation of computational times leads to an increase in resource utilization on FPGA.

Additionally, it is worth noting that the temporal stability of the hardware implementation on FPGA is much better than that on i7, when considering the relative variations given by RSDP(s) or when considering CI(s).

For conclusion, FPGA provides a better temporal stability than Intel i7 but sorts on FPGA are slower than on i7, even if FPGA offers high performance in terms of parallelism. On the contrary, i7 leads to worse performance than FPGA in terms of temporal stability, when considering RSDP(s).

However, the sorts run much faster on i7 than on FPGA and it is possible to act on worst-case computational times and limit the time variations by reducing the upper outliers (#⁺outliers(s)) thanks to dedicated configuration of the operating system, based on “processor affinity for RT-tasks”.

The choice of the “best sort” from the points of view of computational time and temporal stability is clear on FPGA and tim-sort appears to be the ranked first. The same choice on i7 is not so easy and depends on n but also on the criterion to be minimized, i.e. either the computational time or the temporal stability. It appears that the best choice considering both criteria for n≥512 is the heap-sort.

For 64≤n≤256, the choice is not so easy whereas for n≤64 the computational times and the relating variations are so small that any sort is acceptable on i7 when considering the maximum time-constraints of our target application.