ABOUT         PROJECTS         DOODLES         RESUME 


GNITO

3rd place winner




Telling it all, no matter how small.

check out the live prototype here!

WHAT IS GNITO?


GNITO harnesses Agora’s various RTE capabilities to expose aggressors and protect the identities of victims of police brutality, discrimination, and assault. We offer instant multi-app live streaming with blur facial masking to anonymize select video subjects, while providing enhanced accessibility features, such as translatable closed captions.

PROBLEM


Protestors, activists, witnesses, and victims should feel safer when sharing video evidence of abusive or discriminatory behavior in order to expose the perpetuator. Often times, police use protest videos meant to share information to make unlawful arrests on peaceful protestors.

team
Jeanette Andrews

Moe Hay Mar Kaung


ROLE
Team Lead
UX/UI Design

UX Research

TOOLS USED

Adobe Xd
Figma

DURATION

15 hours, February 2021

PROJECT



Project: DubsTech Protothon Hackathon, 2021
Track: Enterprise (other tracks: Valentine’s Day for long distance couples & redesign of Frantically Speaking website)
Parameters: Your task is to think outside of the box and design an enterprise Real Time Engagement mobile or web application that showcases how Agora’s video conferencing/voice/streaming/messaging solutions can be used.


Agora


Agora provides the SDKs and building blocks to enable a wide range of real-time engagement possibilities. Agora features we used are bolded:

  • Voice Call
  • Video Call
  • Live Interactive Audio Streaming
  • Live Interactive Video Streaming
  • 3D Spatial Audio
  • 3D Face Mask Features
  • Object Detection
  • Sign Language Recognition
  • License Plate Recognition
  • Augmented Reality
  • Machine Learning

BACKGROUND


Given the opportunity to take Agora’s RTE capabilities to the next level, in a way that had not been done before, our team wanted to think outside the box and figure out a way to use multiple Agora services for social good. We were intrigued by its abilities to integrate Facial Masking, Object Detection, Live Interactive Video Streaming, and Sign Language Recognition. We thought about Live Video Streaming and thought about its current uses, and where it improvement is needed. Who were the main users of live streaming? Celebrities, enterprises with large social media followings, and educational public figures. We decided to focus on public figures leveraging social media to reach the masses to inform and educate and thought about current gaps in technology for them.

RESEARCH


We immediately thought of the 2020 Black Lives Matters protests that had been live streamed more than ever before (New York Times, 2020), opening up a new branch to activism. However, facial recognition software had been weaponized to make unlawful arrests on protestors using videos of protests shared on social media (Washington Post, 2020). We recognized that there was an obvious gap: social media and live streaming were powerful and necessary, but protestors, journalists, photographers, and organizers needed identity protection or else these videos would continue to decrease.

PAIN POINTS



USER PERSONAS


USER FLOW



FEATURES




KEY DESIGN DECISIONS


We identified 3 details that guided our design decisions:

1. Protestors are in highly stimulated environments and are focused more on the in-person situation over their phone screens

2. Abusive authority on site threatens their ability to ensure that the video they are recording will make it to social media once they have time to upload

3. Balancing recording the events of a protest and protecting the identity of fellow protestors was difficult without heavy video editing




1. QUICK ACCESS TO RECORD
GNITO provides access to live streaming on multiple social media platforms, simultaneously. Because streamers were more concerned with their own safety and presence at the protest, we
  • made the record button the first and spotlighted button on the opening page
  • placed it in the thumb zone for easy access
  • allowed users to not keep their thumb on the record button to continue recording

2. UPLOAD BUTTON SECONDARY
The upload button is less visible than the “Go Live” button because we wanted to decrease the time that a steamer had to spend figuring out how to go live when in a stressful environment.

3. UNIVERSALLY RECOGNIZABLE RECORD BUTTON
A black circle with red center is a “pre-digital” icon recognizable by many users of different ages, tech background, and languages.

4. “GO LIVE” WARNING
The words “Go Live” are displayed in large text above the record button rather than “Record” to mitigate confusion for the user who may not realize that they would automatically begin streaming.

notes:
• There would be a 5-second countdown before streaming to give users time to make streaming decision or stop recording
• First time users would have a sign in page where they permit access to their social media pages and would choose which platforms to be set to default. We based this decision on the assumption that most users would download the app when they have more time to set up rather than during filmable events, where they would not think that downloading a new app was a priority.
• If a user wants to explore the app without streaming, perhaps to familiarize themselves with its functionality, they could disable streaming on every social media platform. In this case, the recording with blur and identification features would continue and save to camera roll

OPTIMIZATION: Make it less complicated for users to test the app without live streaming by offering this option from the home page.
5. FULL SCREEN
The video preview is mostly full screen, rather than as a mini player because there are very minimal features to navigate and so that the user can record as they are used to

6. DEFAULT BLUR
Rather than clicking on which faces to blur out, as there could be too many to deal with while live streaming, blur is on default for all detected faces. We considered allowing users to pre-determine which faces were default blur or default visible, but chose the safer option that eliminates technological biases or the compromisation of some identities.

The unblur option is for revealing the identity of chosen persons, such as perpetrators.

7. ICON-LEAD NAVIGATION
We have heavy use of icons to simplify navigation.



8. STOP RECORDING NOTIFICATION
When a streamer stops recording or accidentally stops recording, they are warned beforehand.



9. AUTO SAVE
Once recordings are finished, they are automatically saved to camera roll and the user’s cloud account (if set up). This is a solution for protestors and bystanders whose phones are nonconsensually taken away and recordings are deleted.

10. AUDIO STREAMING OPTIONS
There are three audio streaming options: no audio + no captions, background noise cancellation + captions, and default audio + captions. This allows streamers to choose how to protect identities via audio.



OPTIMIZATION: Allow recording with audio streaming but disable captions.

11. SOCIAL MEDIA STREAMING OPTIONS
While recording, users can decide which apps to stream to and when. Streamed videos can have different start and end times for different social media apps without going into individual apps or editing videos.



12. QUICK VIEW FOR CAPTION TRANSCRIPTION
Users can quickly view the captions being automatically transcribed while recording. While they cannot edit these, we allow them to see transcriptions so they may decide to turn them off and on. For example, if Agora cannot accurately transcribe, a user may decide to disable captions. This is available in multiple languages. Non-English transcriptions are saved as plain text and can be exported as text.



OPTIMIZATION: Make text more legible by not using glass background

13. TATTOO IDENTIFICATION Based on our research, tattoos are also commonly used to identify persons in videos. We allow users to self-identify tattoos and blur them. We keep this option in the “more” section because tattoos are not on default blur, as they are not as easily detectable with Agora’s AI features.

14. BADGE & ID CARD IDENTIFICATION
Like tattoos, users can identify badges and ID cards. Agora may scan these using License Plate Recognition and all info will be automatically saved to Video Details


15. ACCURACY DISCLAIMER
An accuracy disclaimer is shown with badge and ID card identification in the case that Agora misreads information about persons in the video.



INTERACTION DESIGN


1D, WORDS
The words “Go Live” are displayed in large text above the record button rather than “Record” to mitigate confusion for the user who may not realize that they would automatically begin streaming.


2D, VISUAL RESPRESENTATIONS
A black circle with red center is a “pre-digital” icon recognizable by many users of different ages, tech background, and languages.

Inside the app, every button uses icons rather than titles to simplify navigation.

3D, PHYSICAL OBJECTS AND SPACE
The upload button is less visible than the “Go Live” button because we wanted to decrease the time that a steamer had to spend figuring out how to go live when in a stressful environment.

The video preview is mostly full screen, rather than as a mini player because there are very minimal features to navigate and so that the user can record as they are used to.

The app is completely vertical for easy single-hand recording and accurate streaming on social media apps where live broadcasts are mostly vertical.

4D, TIME
Once recordings are finished, they are automatically saved to camera roll and the user’s cloud account (if set up). This is a solution for protestors and bystanders whose phones are nonconsensually taken away and recordings are deleted.

When a streamer stops recording or accidentally stops recording, they are warned beforehand.

5D, BEHAVIOUR
Gnito is inherently connected to social media, so users receive feedback on use of the app from social media connections who observe the outcomes of the app’s blurring, transcription and translation services, and badge/ID identification.


FUTURE OPTIMIZATION


1. DISABLE CAPTIONS
Allow recording with audio streaming but disable captions.

2. NON-LIVE APP USAGE
Give users easier, low-risk access to exploring the app’s functionality without live streaming.

3. REAL-TIME ASL RECOGNITION
Connect disabled activists with the greater public by translating American Sign Language to other global forms of dialogue.

4. MACHINE LEARNING
Optimize Agora's machine learning capabilities as app further familiarizes itself with user data.