No materials for the event yet, sorry!

Pose2Blocks is software that changes the way we interact with Snap! programming by incorporating body movements, gestures, and even dancing(!) as control inputs. Building on Kinect2Scratch(1), Pose2Blocks utilizes machine learning models to estimate skeletal joint positions from a simple consumer webcam, making the technology more accessible and compatible with Windows, Mac, and Linux operating systems.

Kinect2Scratch was first developed and released in 2011, and was used by students, teachers, and researchers to develop games and applications. Some surprising uses included ehealth games for children with cerebral palsy(2), therapy for chronic stroke patients(3), and games to teach independent living skills (4).

This talk will include a demo of Pose2Blocks, demonstrating how sprites can react to gestures and follow the movement of your hand. We will then explore the possibilities of making your Snap! games and apps respond to unique 'dance' moves, by building a app that draws your 'skeleton' that responds as you move.

References

1. Howell, S. (2012). Install Scratch and Kinect2Scratch. In J. St. Jean (Ed.), Kinect Hacks: Tips and Tools for Motion and Pattern Detection (1st ed., pp. 184 - 206). Sebastopol, CA: O'Reilly.
2. Hung, J.-W., Chang, Y.-J., Chou, C.-X., Wu, W.-C., Howell, S. and Lu, W.-P. (2018) ‘Developing a Suite of Motion-Controlled Games for Upper Extremity Training in Children with Cerebral Palsy: A Proof-of-Concept Study’, Games for Health Journal. Mary Ann Liebert, Inc., publishers, 7(5), pp. 327–334. doi: 10.1089/g4h.2017.0141.
3. Hung, J.-W., Chou, C.-X., Chang, Y.-J., Wu, C.-Y., Chang, K.-C., Wu, W.-C. and Howell, S. (2019) ‘Comparison of Kinect2Scratch game-based training and therapist-based training for the improvement of upper extremity functions of patients with chronic stroke: a randomized controlled single-blinded trial’, European Journal of Physical and Rehabilitation Medicine, 55(5), pp. 542–550. doi: 10.23736/S1973-9087.19.05598-9.
4. Kang, Y.-S., Chang, Y.-J. and Howell, S. R. (2020) ‘Using a kinect-based game to teach oral hygiene in four elementary students with intellectual disabilities’, Journal of applied research in intellectual disabilities: JARID. doi: 10.1111/jar.12828.