Conversation Garden

Timeline
Tools Used

November - December 2020

As schools transitions into a online space, we find classrooms experiencing a common problem of a lack of conversation. Therefore, using Google's Teachable Machine, we wanted to create a Zoom function that would motivate student to make more engagement. 

HTML/CSS/JS/p5js

Teachable Machine

Collaborators

Alice Fang, Charmaine Qiu, Mimi Jiao, Sebastian Carpenter 

Customize your own conversation flower

ezgif-5-cb8409af273f.gif

See your flower accompany your calls
& see it grow as you talk

ezgif-5-66f705ea1329.gif
zoom-Logo-Vector-01.png
raise.gif

Balanced conversations,
increase virtual engagement,
and always have fun!

 
The Challenge

In order to encourage more conversation in online classrooms, we wanted to some sort of engagement that would reward students for more contribution but also not penalize those who don't talk. This is so that we could incentivize students to talk rather than making people feel uncomfortable if they choose not to show their screen.

 
Process
Screen Shot 2020-12-22 at 11.01.06 AM.pn
ezgif-5-d4063fcb33c3.gif

Taking care of your own engagement flower

We brainstormed multiple options in visuals that we could do to achieve this such as how students can earn coins to then "purchase" Zoom accessories for their videos. But this could be distracting to the class environment, therefore, we pursuing the idea where each student is responsible for growing their own conversation flower.

Amplifying gestures in real-life conversations

As part of our system, I felt that including features that could help the professor further evaluate the tone of the classroom and amplify natural gestures that one would make in conversation would further add complexity and meaning to our project. This resulted to the team exploring how to incorporate gesture recognition with the voice recognition from Teachable Machine.


 

Face tracking

I tested using PoseNet for our gesture, however, this program only works well in regards to the whole body. Therefore, we used BRF (Beyond Race Reality) that only tracks the face instead. We also explored how visuals would respond such as expand in size (seen on the leftmost image) as someone talks for longer periods of time.

ezgif-5-58deb251a8ea.gif

👎

👍

🕐

🔇

🔈

🔊

Portraying key gestures

We decided on certain gestures to implement into the flower characters such as thumbs up/down, hand raise, and timeout (away). We decided on poses that were static because this would be captured by the BFL software more accurately than a pose that changed over time (such as clapping). 

 

ezgif-5-f4f7bb98d2e7.gif

Connecting with OBS camera with Zoom

With our animations implemented into our code with Teachable Machine and BFR, we then used OBS camera to use with Zoom. This is a snapshot of a conversation taking place on Zoom with our program.

ezgif-5-66f705ea1329.gif

Expanding our system with customization & Future steps

For our futures steps, we would have wanted to create a website where users are able to create their own flower character and use Teachable Machine to match with their own faces for accurate gesture recognition. 

ezgif-5-cb8409af273f.gif
 
Final Product