Subscribe to EDOM TECH Newsletter
2023/12/11
Small but powerful – NVIDIA Jetson-powered embedded systems for a smarter future
Embedded systems have revolutionized numerous industries, powering everything from autonomous vehicles and drones to intelligent surveillance systems and medical devices. As the demand for intelligent and AI-driven applications continues to rise, the need for powerful and efficient embedded computing solutions has become increasingly critical.
The recent performance surge in AI-augmented accelerated computing further drives said trend to new highs with the main player in the scene, NVIDIA, and its embedded systems family Jetson, contributing the most by creating dedicated, cutting-edge solutions for mobility-focused AI applications.
Everything a system needs within a small package
NVIDIA Jetson products are basically running a System-on-Module (SoM) design that bears strong similarity to the SoC, or System-on-Chip, silicon brains living inside your smartphones and tablets. This approach basically contains the CPU, GPU, RAM, and storage in one big component that provides a smaller physical footprint and closer electrical traces for better performance while the other small components are installed and existed outside of the SoM package. With this approach, NVIDIA is capable of providing AI inferencing and deep learning capabilities while maintaining power efficiency.
Popular NVIDIA Jetson Offerings
With the lesson on SoC done, let's go through some of the most popular NVIDIA Jetson products currently available in the market.
1. Jetson Orin Nano
Primed as one of the valuable boards with the best entry barrier, the Jetson Orin Nano packs a 6-core ARM Cortex-A78AE CPU with 1024 CUDA cores and 32 Tensor Cores powered by the famous Ampere architecture. The small form factor allows it to be the main processor behind solutions like intelligent surveillance cameras, home automation systems, and even small-scale robotics.
One of the examples that you can find in Malaysia and perhaps the closest to you are license plate-and automobile identifying cameras in toll stations.
Provided by Tapway who joined the NVIDIA Metropolis program recently, it solves the main issue of different classes of vehicles requiring a different fee and is often followed by manual human checking. As such, Tapway's VehicleTrack software is capable of reading license plates and detecting the vehicle class, make, and color in just 50ms. It can do so even if the vehicle is approaching from a direction at 40km/h. Trained with NVIDIA Triton Inference Server, the backend is capable of handling 50 video streams simultaneously coming directly from an edge device which can be powered by a Jetson Nano class product.
Designed mainly for edge computing cases, the Jetson TX2i goes even further on the small and efficient spectrum with lower power consumption rate and form factor than “Nano” class boards.
Running the older Pascal architecture, it hosts 8GB of LPDDR4 RAM and 32GB eMMC storage together with ARM-A57 CPU plus 256 CUDA cores. As such, those who want to utilize their budget on a higher number of deployed units with lower but matching performance requirements will find the Jetson TX2i feasible. Places like smart factories would harness Jetson TX2i for intelligent vision systems and industrial automation.
Plucking a real-life example is from the Japan-based Musashi Seimitsu Industry which focuses on manufacturing automobiles and associated parts. The automation process is delivered in the form of optical inspections for defects in gears and welding which prior to this, are all done manually and account for up to 20% of the entire manufacturing time.
They first utilizes NVIDIA DGX Station to train an Inception v2 Keras network algorithm which is then processed using NVIDIA Jetpack and TensorRT. Lastly, the deployable model is then operated within a Jetson TX2 model preinstalled with software and peripheral support, allowing Musashi to quickly build and deploy Neural Cube (The entire optical inspection solution as they called it) with ease.
When operated, an image of a freshly produced part is scanned and analyzed for defects. Once found, the robot arm will then take out any faulty parts and then replaced them with a good one when possible. The current software is capable of achieving 99% accuracy in finding 1mm defects on gears within 5 seconds and 0.5mm defects in welding within 7.5 seconds without any human supervision.
Need something with more power but don't want to go too overboard? Then the Jetson Xavier NX should do the trick.
Although not the latest and greatest, the 16GB version (Both RAM and storage) is powered by a Volta-based GPU with 384 CUDA cores and 48 Tensor cores alongside NVIDIA’s Carmel ARM v8.2 CPU capable of delivering 21 TOPS of AI performance. All of these can be powered as little as 10W running through complex and modern neutral networks in parallel while accepting data inputs from multiple high-resolution sensors.
Examples of systems utilizing it are drones and medical imaging devices alongside smart city equipment.
Saildrone's implementation of Jetson solutions is quite unique as the company specializes in developing autonomous uncrewed surface vehicles (USVs) equipped with tons of sensors. Those sensors group is capable of capturing tons of data which can be processed on Jetson modules for AI at the edge and then passed into the DeepStream SDK for intelligent video analytics.
The main goal is to make operations of ocean intelligence collection as cost-effective as possible while packing all the real-time data for research and industrial purposes such as in the fields of science, fisheries, weather forecasting, ocean mapping, and maritime security. With weather and climate changes occurring more than desired in recent years, Saildrone’s mission to constantly be aware of the ocean’s state are only achievable with equipment that is both powerful, compact, and portable that allows processing data at all times, locally or in the cloud.
4. Jetson AGX Orin Industrial
For organizations that seek the very best without worrying about budget, the Jetson AGX Orin Industrial is by far the strongest performer out of the Jetson family.
Equipped with a 12-core NVIDIA Arm Cortex A78AE V8.2 CPU, the main highlight comes from the 2048 CUDA core + 64 Tensor core combo powered by NVIDIA Ampere plus 64GB LPDDR5 RAM and 64GB eMMC storage. Such a combination lands its AI performance at 248 TOPS – more than 10 times more than the Jetson Xavier NX 16GB. Let's not forget the Jetson family's core spirit which is efficient computing, landing it at the range of 15W to 75W of power consumption.
Recent breakthroughs in autonomous mobile robots capable of handling hazardous materials while transporting them from place to place requires sophisticated processing only achievable with something in the tier of this particular model.
Since the chip is used by leading industry players, one of the brands who utilize it is F&B giant PepsiCo relying on NVIDIA Metropolis partner KoiReader Technologies’ advanced machine vision solution.
The system enables superior efficiency in reading warehouse labels and barcodes, no matter their size, condition, and angle, all thanks to powerful algorithms capable of identifying them. Not only they are scanning shelf cartons but also those on the conveyor belts and forklifts, making a 100% item traceable warehouse plausible. As for the data generated, it includes things like how long it takes for a worker to pack a box, how much they are packing, or the other way around in unpacking.
The NVIDIA Jetson AGX Orin modules are not the only ones responsible for all the backend processing as things as NVIDIA’s A series professional GPU and Triton Inference Server plus DeepStream SDK and TensorRT software all make up the necessary parts for a truly smart and powerful machine vision solution.
Conclusion
In conclusion, NVIDIA Jetson is the go-to solution for developers in the embedded systems domain that has the potential to take AI and deep learning to the next level.
Aside from the hardware, NVIDIA is also providing the same level of software stack access across all tiers and generations of offerings, meaning that the only difference here is about scalability and perhaps requirements matching.
For the industry folks who would like to give NVIDIA Jetson a try, you can take a look at EDOM Technology and get in touch with them.
The Taiwan-based organization offers a wide range of solutions from AI computing and automotive to IoT and mobile-related topics.
And yes, they also carried the aforementioned NVIDIA Jetson family of products too.
Follow the links below to learn more about EDOM Technology and NVIDIA Jetson.
The recent performance surge in AI-augmented accelerated computing further drives said trend to new highs with the main player in the scene, NVIDIA, and its embedded systems family Jetson, contributing the most by creating dedicated, cutting-edge solutions for mobility-focused AI applications.
Everything a system needs within a small package
NVIDIA Jetson products are basically running a System-on-Module (SoM) design that bears strong similarity to the SoC, or System-on-Chip, silicon brains living inside your smartphones and tablets. This approach basically contains the CPU, GPU, RAM, and storage in one big component that provides a smaller physical footprint and closer electrical traces for better performance while the other small components are installed and existed outside of the SoM package. With this approach, NVIDIA is capable of providing AI inferencing and deep learning capabilities while maintaining power efficiency.
Popular NVIDIA Jetson Offerings
1. Jetson Orin Nano
Primed as one of the valuable boards with the best entry barrier, the Jetson Orin Nano packs a 6-core ARM Cortex-A78AE CPU with 1024 CUDA cores and 32 Tensor Cores powered by the famous Ampere architecture. The small form factor allows it to be the main processor behind solutions like intelligent surveillance cameras, home automation systems, and even small-scale robotics.
One of the examples that you can find in Malaysia and perhaps the closest to you are license plate-and automobile identifying cameras in toll stations.
Provided by Tapway who joined the NVIDIA Metropolis program recently, it solves the main issue of different classes of vehicles requiring a different fee and is often followed by manual human checking. As such, Tapway's VehicleTrack software is capable of reading license plates and detecting the vehicle class, make, and color in just 50ms. It can do so even if the vehicle is approaching from a direction at 40km/h. Trained with NVIDIA Triton Inference Server, the backend is capable of handling 50 video streams simultaneously coming directly from an edge device which can be powered by a Jetson Nano class product.
2. Jetson TX2i
Designed mainly for edge computing cases, the Jetson TX2i goes even further on the small and efficient spectrum with lower power consumption rate and form factor than “Nano” class boards.
Running the older Pascal architecture, it hosts 8GB of LPDDR4 RAM and 32GB eMMC storage together with ARM-A57 CPU plus 256 CUDA cores. As such, those who want to utilize their budget on a higher number of deployed units with lower but matching performance requirements will find the Jetson TX2i feasible. Places like smart factories would harness Jetson TX2i for intelligent vision systems and industrial automation.
Plucking a real-life example is from the Japan-based Musashi Seimitsu Industry which focuses on manufacturing automobiles and associated parts. The automation process is delivered in the form of optical inspections for defects in gears and welding which prior to this, are all done manually and account for up to 20% of the entire manufacturing time.
They first utilizes NVIDIA DGX Station to train an Inception v2 Keras network algorithm which is then processed using NVIDIA Jetpack and TensorRT. Lastly, the deployable model is then operated within a Jetson TX2 model preinstalled with software and peripheral support, allowing Musashi to quickly build and deploy Neural Cube (The entire optical inspection solution as they called it) with ease.
When operated, an image of a freshly produced part is scanned and analyzed for defects. Once found, the robot arm will then take out any faulty parts and then replaced them with a good one when possible. The current software is capable of achieving 99% accuracy in finding 1mm defects on gears within 5 seconds and 0.5mm defects in welding within 7.5 seconds without any human supervision.
3. Jetson Xavier NX
Need something with more power but don't want to go too overboard? Then the Jetson Xavier NX should do the trick.
Although not the latest and greatest, the 16GB version (Both RAM and storage) is powered by a Volta-based GPU with 384 CUDA cores and 48 Tensor cores alongside NVIDIA’s Carmel ARM v8.2 CPU capable of delivering 21 TOPS of AI performance. All of these can be powered as little as 10W running through complex and modern neutral networks in parallel while accepting data inputs from multiple high-resolution sensors.
Examples of systems utilizing it are drones and medical imaging devices alongside smart city equipment.
Saildrone's implementation of Jetson solutions is quite unique as the company specializes in developing autonomous uncrewed surface vehicles (USVs) equipped with tons of sensors. Those sensors group is capable of capturing tons of data which can be processed on Jetson modules for AI at the edge and then passed into the DeepStream SDK for intelligent video analytics.
The main goal is to make operations of ocean intelligence collection as cost-effective as possible while packing all the real-time data for research and industrial purposes such as in the fields of science, fisheries, weather forecasting, ocean mapping, and maritime security. With weather and climate changes occurring more than desired in recent years, Saildrone’s mission to constantly be aware of the ocean’s state are only achievable with equipment that is both powerful, compact, and portable that allows processing data at all times, locally or in the cloud.
4. Jetson AGX Orin Industrial
For organizations that seek the very best without worrying about budget, the Jetson AGX Orin Industrial is by far the strongest performer out of the Jetson family.
Equipped with a 12-core NVIDIA Arm Cortex A78AE V8.2 CPU, the main highlight comes from the 2048 CUDA core + 64 Tensor core combo powered by NVIDIA Ampere plus 64GB LPDDR5 RAM and 64GB eMMC storage. Such a combination lands its AI performance at 248 TOPS – more than 10 times more than the Jetson Xavier NX 16GB. Let's not forget the Jetson family's core spirit which is efficient computing, landing it at the range of 15W to 75W of power consumption.
Recent breakthroughs in autonomous mobile robots capable of handling hazardous materials while transporting them from place to place requires sophisticated processing only achievable with something in the tier of this particular model.
Since the chip is used by leading industry players, one of the brands who utilize it is F&B giant PepsiCo relying on NVIDIA Metropolis partner KoiReader Technologies’ advanced machine vision solution.
The system enables superior efficiency in reading warehouse labels and barcodes, no matter their size, condition, and angle, all thanks to powerful algorithms capable of identifying them. Not only they are scanning shelf cartons but also those on the conveyor belts and forklifts, making a 100% item traceable warehouse plausible. As for the data generated, it includes things like how long it takes for a worker to pack a box, how much they are packing, or the other way around in unpacking.
The NVIDIA Jetson AGX Orin modules are not the only ones responsible for all the backend processing as things as NVIDIA’s A series professional GPU and Triton Inference Server plus DeepStream SDK and TensorRT software all make up the necessary parts for a truly smart and powerful machine vision solution.
Conclusion
Aside from the hardware, NVIDIA is also providing the same level of software stack access across all tiers and generations of offerings, meaning that the only difference here is about scalability and perhaps requirements matching.
For the industry folks who would like to give NVIDIA Jetson a try, you can take a look at EDOM Technology and get in touch with them.
The Taiwan-based organization offers a wide range of solutions from AI computing and automotive to IoT and mobile-related topics.
And yes, they also carried the aforementioned NVIDIA Jetson family of products too.
Follow the links below to learn more about EDOM Technology and NVIDIA Jetson.
