Embedded Neural Network Processing System for General Purpose Assistive Technology

Abstract

Assistive technology (AT) is invaluable to people with special educational needs and disabilities, enabling them to function more efficiently. Designed with low power consumption and a GPU capable of running AI frameworks, NVIDIA's Jetson Nano is an affordable embedded system device. With this system, users will be mobile and connected to any computer with an appropriate interface, and able to interact with that computer. Furthermore, Jetson Nano has modularity, which could solve other issues related to AT in the future. For this study, the FCOS and TOOD models were used for the cursor object detection along with the SAHI algorithm. This study combines a Speech-to-Intent module and a Mouse Cursor Detection module to move the computer mouse using speech commands. A USB Gadget API can be developed in the future in order to extend the assistive technology device to provide assistance to other USB-equipped devices using the Linux operating system in the future, as well as using powerful embedded devices, like the Jetson Nano or Xavier, that are able to combine additional deep learning models to enhance their functionality