ezgif-5-cb8409af273f.gif
ezgif-5-66f705ea1329.gif
 

Conversation Garden

Overview

As schools transitions into a online space, we find classrooms experiencing a common problem of a lack of conversation. Therefore, using Google's Teachable Machine, we wanted to create a Zoom function that would motivate student to make more engagement. 

Timeline

November - December 2020

Tools Used

HTML/CSS/JS/p5js, Teachable Machine

Collaborators

Alice Fang, Charmaine Qiu, Mimi Jiao, Sebastian Carpenter 

 
Problem

In order to encourage more conversation in online classrooms, we wanted to some sort of engagement that would reward students for more contribution but also not penalize those who don't talk. This is so that we could incentivize students to talk rather than making people feel uncomfortable if they choose not to show their screen.

Taking care of your own engagement flower

We brainstormed multiple options in visuals that we could do to achieve this such as how students can earn coins to then "purchase" Zoom accessories for their videos. But this could be distracting to the class environment, therefore, we pursuing the idea where each student is responsible for growing their own conversation flower.

Screen Shot 2020-12-22 at 11.01.06 AM.pn

Amplifying gestures in real-life conversations

As part of our system, I felt that including features that could help the professor further evaluate the tone of the classroom and amplify natural gestures that one would make in conversation would further add complexity and meaning to our project. This resulted to the team exploring how to incorporate gesture recognition with the voice recognition from Teachable Machine.


 

ezgif-5-d4063fcb33c3.gif
Process
 

Face tracking

I tested using PoseNet for our gesture, however, this program only works well in regards to the whole body. Therefore, we used BRF (Beyond Race Reality) that only tracks the face instead. We also explored how visuals would respond such as expand in size (seen on the leftmost image) as someone talks for longer periods of time.

ezgif-5-58deb251a8ea.gif

👎

👍

🕐

🔇

🔈

🔊

Portraying key gestures

We decided on certain gestures to implement into the flower characters such as thumbs up/down, hand raise, and timeout (away). We decided on poses that were static because this would be captured by the BFL software more accurately than a pose that changed over time (such as clapping). 

 

ezgif-5-f4f7bb98d2e7.gif

Connecting with OBS camera with Zoom

With our animations implemented into our code with Teachable Machine and BFR, we then used OBS camera to use with Zoom. This is a snapshot of a conversation taking place on Zoom with our program.

ezgif-5-66f705ea1329.gif

Expanding our system with customization & Future steps

For our futures steps, we would have wanted to create a website where users are able to create their own flower character and use Teachable Machine to match with their own faces for accurate gesture recognition. 

ezgif-5-cb8409af273f.gif
 
Final Product