Cross-Layer Hardware/Software Assessment of the Open-Source NVDLA Configurable Deep Learning Accelerator

The NVIDIA Deep Learning Accelerator (NVDLA) is a free and open architecture that aims at promoting a standard way of designing deep neural network (DNN) inference engines. The analogy between open-source software and hardware points to FPGAS as ideal implementation platforms for open hardware accelerators. However, the instantiation flexibility enabled by reconfigurable logic should be correlated to the capacity of cost-effective devices. This paper explores the resource utilization-performance trade-offs spanned by the main precompiled NVDLA accelerator configurations on top of the mainstream Zynq UltraScale+ MPSoC. For the sake of comprehensive end-to-end performance characterization, the inference rate of the software stack is matched to that of the accelerator hardware, thus identifying current bottlenecks and promising optimization directions.