Presented by:

Harley Lara

from Rhine-Waal University of Applied Sciences - EOLab

I like to build stuff 😁

Ilgar Rasulov

from Rhine-Waal University of Applied Sciences (Hochschule Rhein-Waal, HSRW)

Ilgar is a master student at Rhine-Waal University of Applied Sciences(HSRW).

He is busy with research tasks in the field of Data Science, Machine Learning and Artificial Intelligence. Before joining HSRW, he worked for several years as an ERP programmer and database manager for corporate clients.

At HSRW, he is working with mini drones and Jetson computers. There are finished projects of Computer Vision applications, drone control app as well as the versions, that utilise the Jetson Nano/Xavier computer for Computer Vision tasks. He was also in the SNAP! projects, especially in SNAP! - drone, SNAP! - Jetson, and SNAP! - Jetson - drone integraion projects.

He is currently interested in robotic applications, drone software and Computer Vision tasks.

Ilgar Rasulov contributes at 1 Event: Let's plAIy!. at Snap!Con 2022

Rolf Becker

from Rhine-Waal University of Applied Sciences (HSRW.eu)

Harley, Ilgar, and Ali are my students who are the creators of our contributions to SNAP!Con 2022. We are presenting together!

Rolf has been a professor for environmental physics at Rhine-Waal University of Applied Sciences (HSRW) since 2010.

He received his diploma in physics from the University of Bonn and his PhD in hydrology from the Karlsruhe Institute of Technology.

At HSRW, he leads the IoT Lab, the Drone Lab, and the Earth Observation Lab (EOLab). In addition to his research and teaching in environmental monitoring in bachelor's and master's programs, he is heavily involved in STEM education. He regularly conducts workshops with schools.

His current interest is combining SNAP! with AI on NVIDIA Jetsons in robotic applications.

Volunteer Hosts
Thanks for helping with Snap!Con 2022!

Hands-on Workshop on Combining Snap! with the NVIDIA Jetson AI Embedded Systems

Together we will create a Snap! game based on fruits and vegetables detection (image classification).
Local and remote attendance possible!

Trainers: Harley, Ilgar, Ali and Rolf (in Heidelberg)

We will use the NVIDIA Jetson Nano computers we brought to the Heidelberg conference location. The reservation links for the Jetsons are further down.

We are able to perform object detection ('DetectNet') and segmantic image segmentation, too, but for this workshop we decided to concentrate on image classification for the sake of simplicity.

Snap! and AI

Reserve your NVIDIA Jetson Nano for the Let's plAIy! Workshop!

In the Let's plAIy! workshop at Snap!Con 2022 we will use AI in Snap! to create simple games based on image classification.
The image classification (an AI algorithm) is running on a NVIDIA Jetson.
We have 14 Jetsons available! First come, first served!

Participants in Heidelberg

  • We have 7 Jetsons in total for attendees in HD
  • Register under section Heidelberg to reserve one.
  • You will connect through the local Wifi to the Jetson allocated to you.
  • You will get your personal Jetson hostname after registration in the workshop.

Participants Abroad / Remote

  • We have 7 Jetsons in total for attendees not in HD
  • Register under section Abroad to reserve one.
  • You will connect through VPN to the Jetson allocated to you.
  • This requires to install VPN. See our VPN installation instructions
  • You need additional personal username / password. We will send them by E-Mail to you after registration.

Installation Instructions

Please follow the INSTALLATION INSTRUCTIONS ON EOLAB WIKI

Motivation

AI at Schools

Artificial intelligence is a prevailing technology that poses opportunities and risks for society and the environment. Building knowledge and skills in schools is critical to enable students early on to master the technology instead of being mastered, to demystify AI, to acquire critical reflection competencies, to assess the opportunities and limitations of AI for problem solving. The usability of many AI toolkits continues to improve. Less and less expert knowledge is required to use them. This Simplification can be further advanced.

Combining Snap! with AI on NVIDIA Jetson

Artificial intelligence (AI) and machine learning (ML) are being integrated more and more into school curricula. This raises the interesting question of how AI and ML can be combined with Snap! Several groups are working on such integrations. Our approach presented and practiced in this workshop is using the AI power of NVIDIA Jetson Nano computers for image classification in Snap! The software is open source and no cloud is needed. Jetsons running Ubuntu are interesting for schools, since you can do many more things with them, e.g. build your own mobile robots (aka JetBots). Snap! could even run on the Jetsons but we decided to demonstrate a distributed system architecture.

Technical Background: Snap! and NVIDIA Jetson

We use SNAP! to communicate with image classifiers and object detectors based on convolutional neural networks (CNN). For many activities we are using NVIDIA Jetson Embedded AI computers with GPU. The Jetsons are extremely versatile and NVIDIA provides very good AI software optimized for the 'little' machines.

Snap! is used to take video frames from the local user PC webcam. These frames (pictures) showing up on the Snap! stage are transmitted via a JavaScript websocket to the AI server.

Image Classification

We use ImageNet. To be continued ...

Object Detection

In the current setup we utilize the object detection software DetectNet running on a NVIDIA Jetson Nano embedded computer as a backend server.

This backend object detector reports the bounding box, the object type, and the confidence level for each identified object on the image back to Snap! This information is overlaid with the picture in Snap! This object metadata can be used to control Snap! programs such as games.

Duration:
1 h
Room:
Room 1
Conference:
Snap!Con 2022
Type:
Workshop