Facecard: help discover unfamiliar names in small groups meeting with name retrieval system

Facecard ISWC Paper

Facecard

With distributed work teams and widespread social circles, friends’ and colleagues’ faces may remain unfamiliar even when communicating on a routine basis. In a business meeting, it is embarrassing to ask for a colleague’s name.

Scenario

This video demonstrated the problem Facecard is trying to solve.

There are unfamiliar faces and people often call out the wrong name, and they use the voice recognition system to say Talyor is here. Later in the video it shows the interruption of all kind of technologies in the meeting, people pull up the phone to check sms, receiving emails, looking for chargers. People keep forgetting where they left over.

Current Solutions

Most current real-time face identification systems are awkward to use. For example, Nametag claimed to have the best facial recognition algorithms on Glass, but still uses facial recognition towards a paperboard. In the practice, most real-time systems are awkward to use, it requires the user to adjust his head angle to align the college's face in the center of the focus and try to get a good view and lighting. nametag, glasshole.

Definition of Facecard

Facecard, a mobile Android application/glassware, provides a list of names and faces for colleagues expected to attend a meeting with the user. We aim to create an application that is efficient, intuitive to use, causes less interruption to the meeting.

Data and System Architecture

Facecard with Bluetooth LTE

I started out by building the first round of MVP (Minimum Viable Product) with the use of bluetooth LTE. Each phone connected with a pair of Glass, the phone calls the server fetch these data from Google plus api. After another person enter the lte range of 10m, it will show up in the Facecard list and request that person's information from the local database stored on the phone, glass serve as the head-up-display to keep updating the cards within the range.

Facecard with Google Calendar

The another trial i did was using Google Calendar. Since in the business meeting, we can read the invitation list and fetch calendar profile. The problem here was what if there are 20 people in the list, who should appear on the first screen. We decided to use the TF-IDF (term frequency–inverse document frequency) to rank the people based on likelihood they are unfamiliar faces with exchanges of emails over the past months.

User Interface on Facecard - Glass part

For the interface on Glass, I used Glass's design standard, live-card region standard and started with several rounds of wireframing and design iterations and came out with these three-layout cards design with the balance of overview and details on the interface. We'll give users a 8-cards layout first, you can swipe forward and backward to select or unselect the card, if you want to see more information, you can click on the pad to jump to 4-cards layout with more information exposed.

User Interface on Facecard - mobile part

For the user flow on mobile part, we started out on Google Calendar, on the meeting page, i added the unfamiliar faces read from the invitation list, after you click onto them, it comes to the main screen of facecard with different layouts. If clicked on a single card, notes and exchanged emails will be shown out there.

Facecard demos on Glass and mobile phone

I built the Facecard for Google Glass and Android phone with Native Android Development Environment. Here's a quick demo of how it looks on both platforms.

Facecard Demo on Google Glass

When users first landed on the Facecard, they can swipe forward to locate the 4-cards layout which give them an overview of most people in the meeting. If the users want to find more details of a specific person, they can click on the touchpad to go deeper level and locate on the single card layout.

Facecard Demo on Android mobile phone

Facecard on Android follows the similar design pattern - material design. For each interface, it uses a dynamic resizing card which accommodates the amount of information on that card. Users could swipe to check out more face cards.

Usability Testing 
Evaluation about Information Density and layout on Glass and mobile phone

Method

let participants perform a face-searching task on Facecard and fill out a post-survey in terms of four metrics with likest-scale on 1-card layout, 4-cards layout, 8-cards layout.

Questions

  1. I could find the required person's facecard quickly.
  2. It gives me enough amount of information.
  3. It is very easy to find required person's facecard on Glass.
  4. In terms of overall experience, finding a required person's Facecard on Glass is satisfying.

Metrics

  1. Identification Speed
  2. Amount of information per card
  3. Ease of Use
  4. Satisfaction

User Study
Evaluation of Social Appropriateness from a bystander's perspective

Comparison between Facecard and Facial Recognition on both Google Glass and mobile phone from bystander's perspective. Evaluating the Social appropriateness of four name retrieval systems from a bystander's perspective.

Facecard on Glass
Facial Recognition on Glass
Facecard on mobile phone
Facial Recognition on mobile phone

Video

A/B Testing
Evaluation of Interruption and System Preference

Comparison between Facecard and Facial Recognition on both Google Glass and mobile phone from bystander's perspective. Evaluating the Social appropriateness of four name retrieval systems from a bystander's perspective.

Experiment Setup

Scripting the notifications with automated emails

Python + SMTP Gmail Server

I wrote the scripting program with Python and calls SMTP Gmail Server every minute.  As illustrated in the diagram below,  short messages sent to participants are marked yellow, four identification tasks are marked green. Check out the code.

Surveys

Survey 1: Assessment of preferences on devices to receive short messages during a meeting

Survey 2: Assessment of interruptions due to the use of technologies

Survey 3: Evaluation of preferences toward four name identification systems

Results

Interruption Per Technology

Figure on the left shows the bystanders’ perception of the amount of interruption each name identification technique had on the meeting when reviewing each interaction on video. When doing these ratings, the bystanders were still unaware of the true nature of the experimenter’s tasks. Thus, we obtain a naive measure of how interruptive the name identification techniques were. We hypothesized that Facecard on Glass would prove less interruptive than any of the other techniques. However, the mean ratings achieved a significant difference when compared to Facecard on phone and face recognition on phone and not when comparing Facecard to face recognition on Glass.

Device Preferences

Participants preferred to receive short messages on mobile phone as rated 5.75/7.00 compared with the rating for getting short messages on the head-up dis- play which was 3.75/7.00. However, people felt less ignored when seeing others receiving short messages on the HUD 3.00/7.00 while they felt more ignored when others were receiving the short messages on mobile phone 5.50/7.00. These differences were statistically significant. 

System Preferences

After the deception was revealed, overall preference for each name identification method was examined with average ratings from the 7-point Likert-scale questions. Facecard on Phone and Facecard on Glass were liked by participants over- all with the highest average ratings of 4.83 and 4.33. How- ever, facial recognition on Phone proved to be disliked by most participants and received the lowest average rating of 1.58. While our hypothesis was that Facecard on Glass would be preferred to all other methods, only the mean ratings com- pared to facial recognition on phone achieved statistical significance.

Reflection

We attempted to build a name retrieval system on wearable head-up-display like Google Glass and compared it with another technology Facial Recognition using different metrics. The challenge was to pack the hardware of a small computer into a head-mounted device that would be intuitive to use and social appropriate. But the uncanny relationship between users and technology, devices, also requires social and behavioral changes that were too radical. In terms of the project, it's good that we tried to explore how to design an application, a device from multiple perspectives and tried to find answers in related to privacy, interruption and social appropriateness.