What are My Friends Watching? Social Information on 10-Foot Interfaces

  1. Introduction
  2. Incorporating Social Content
  3. Users
    1. Media Viewers
    2. Gamers
  4. The System
    1. Requirements
    2. Design Space
    3. Design
      1. The Card
      2. Feeds
      3. Content
      4. Content Creation
      5. (Social) Engagement
    4. Ethics
  5. Study and Measurements
    1. The Prototype
    2. Evaluation
    3. Results
      1. Participants
      2. System Usability Scale
      3. Media Selections
      4. Qualitative Data
    4. Discussion
      1. Friends Versus Reviews
      2. Sharing and Social Features
      3. The Mental Model
      4. Issues
      5. Reflection
  6. Conclusion
    1. Limitations
    2. Future Work
    3. Findings and Recommendations
  7. Acknowledgments
  8. Citations
  9. Appendices
    1. Appendix A — Protocol
    2. Appendix B — System Usability Scale

A note on citations: When possible, I have included links to the original sources cited in this project. They were freely available at the time that this page went online in 2015.

For T —

who talked me down when I needed it (a lot), supported me when the going got rough (often), and was confident for me when I wasn't (thank you).

—A

Introduction

TV, movies, and increasingly video games, are all highly social activities. There are huge online activities entirely dedicated to a single series or game1. These forms of media consumption involve different levels of social interaction, ranging from one or two colocated viewers to family units to entire viewing audiences (Cesar & Geerts 2011). A common form of media-based social interaction is sharing insights and opinions on social media after the completion of a viewing or interaction session (Chorianopoulous & Lekakos 2008). But despite the common social mediation that media provides, very few modern TV interfaces provide social features.

Within the last five years, the rise of smart TVs and attached streaming boxes like the Roku and Apple TV2 have allowed users to connect their TVs to the internet and access a limitless amount of movies and television shows. At the same time, there has been adoption of consoles with easily-accessible media stores and streaming applications (The Pew Research Center found that 40% of all adults and 56% aged 18-29 had a console in 2015, and NPD found that 33% of all internet-connected households had a streaming device). Finally, the recent adoption of smart televisions with built-in media applications has meant that more users have access to internet-based media than ever before. These 10-foot interfaces3 present a unique design challenge, with solutions to problems running deeper than just interface design.

Internet-enabled console platforms like Xbox Live and the Playstation Network have allowed users to play video games online with their friends, but beyond simply connecting players, little has been done to leverage this social information. Consoles may have community tabs, but these are often filled with long streams of repetitive content that, at best, provides limited value to players, and streaming boxes lack any sort of social features at all. This is not because users do not want this sort of social interaction — huge communities of users have been built around games and TV on social media platforms like Reddit and Facebook. Instead, the challenges of privacy and social recommendation coupled with limitations of input devices have led to stagnation in this area.

The problem is twofold: first, there is a lack of information available to users about what their friends are engaging with and what is popular in their immediate social network. Second, despite huge communities and a wide range of media available about TV shows, movies, and games, what information is often hard to access or provides little value to users of 10-foot interfaces. This project deals more with social recommendations and implementation, though suggestions are provided for effectively integrating community-generated content into the platform for further value enhancements.

Incorporating Social Content

The addition of social media to a 10-foot interface can quickly veer too far in the wrong direction. Interruptions, intrusive alerts, or simply information that is uninteresting to users can quickly doom a social platform. Girgensohn and Lee (2002) proposed four components required for effective social communication: common ground, awareness, interaction enablers, and a location to communicate and share. Existing 10-foot interfaces fail on a number of these requirements: the Apple TV and Roku provide none of these components; the Xbox provides a location to communicate and share, but impairs awareness of it by hiding it away in a separate panel of the interface that has to be explicitly accessed. This is a particular problem because users of these platforms are often not there for the purpose of interacting socially: they want to play a game or watch something (Chorianopoulous & Lekakos 2008; Geerts & De Grooff 2009; Ducheneaut, Moore, Oehlberg, Thornton, & Nickell 2008) and don't want to be interrupted in that task to do something else. Nobody wants to visit Facebook on their TV (Buffone 2012).

Users

While gamers and watchers may have different end goals — interactivity versus passivity, for example — interviews with users have shown that they make decisions about what to consume in similar ways. Users of a single 10-foot interface may switch between the two roles over the course of an interaction or over the lifespan of the product. For these reasons, these two roles are treated as having identical habits for the purposes of media selection.

The nature of 10-foot interfaces mean they almost always involve the user sitting on a couch or chair in a private place. Sometimes there is a group of viewers, but more commonly, there are between one and four who watch together. Input devices can vary widely, from the simple Apple TV and Roku remotes to the much more complicated TV remotes with numerous, single-purpose buttons, to programmable remotes with touchscreens. This variability means designing solutions can be challenging.

Users were interviewed in person, on the phone, and over instant messaging.

Media Viewers

In talking with users, it became clear that the variety of devices and context-awareness for use blurs the lines between interfaces. Media viewers tend to use two devices at once — typically a TV and a mobile device — when watching TV or a movie, often currently or within close reach of social media. A user who primarily watches media said that she did not want to share clips or screenshots from her TV, but that if her mobile device could allow her to post clips that are on her TV now, she would be more willing to engage with social media.

The user described a clear difference between watching TV shows and watching movies — TV shows are generally a constant source of new material and something she regularly engages with socially, whereas movies tend to be more experiential and not something discussed with others4. The user described TV show viewing taking two different forms: social and experiential viewing. Social viewing often involves simultaneous or proximate social interaction related to the TV show, like sharing show-related posts on social media during the show and texting with friends afterwards. Experiential shows are largely about the narrative of the show. While social media posts may still be made about them, this is not the primary purpose of watching the show.

Privacy and embarrassment can be a concern with TV watching. The user mentioned that part of her reticence to post on social media about TV shows was the limited ability to direct that content to people who also watch that show. Such posts are essentially public to a social network, and so the user was not interested in discussing guilty pleasure shows there — but did want to share discussions and posts with select users on social platforms.

The primary interaction with the 10-foot interface is in the selection of media to watch. This can take the form of the user sitting down, already knowing what they want to watch or browsing through available media — usually a movie — to determine it. Typically, TV shows are not browsed for; the newest unwatched episode is the default choice and is immediately selected. Time permitting, the next episode will be played — often automatically. The goal is to both conclude the narrative and "clear the queue" so that the user can watch another show5. New TV shows are selected from recommendations on social media — often from those that are popular or being discussed regularly.

When browsing for media, the user said that she'll "be in the mood for something" but not necessarily a specific movie. She may seek out a favorite go-to movie, or browse the catalog or specific categories to find what she is looking for. She will use ratings and her own memory of previously recommended movies to decide what to watch, but often ignores the poor built-in recommendations from streaming services.

Gamers

For gamers, there are a wide range of sources they go to for social content, but they rarely seek it out on their gaming platform. One user regularly goes to social platforms like Reddit and Facebook for game-related content, but almost never uses his TV-based interface for it. However, this is not to say that he doesn't engage socially on that interface — inline, community-generated content like news is read when it appears interesting, but it is not sought out. This user also regularly uses the in-game messaging features to communicate with friends.

Both users were interested in social applications for their games, including friend ratings, which friends were playing what games, and what their friends were doing in them. Currently, it is easy to identify which friends are playing what games, but finding out what is happening in those games is more challenging and requires multiple steps. Depending on the platform, this can involve seeking out the friend's profile directly and navigating through menus or going to a separate section of the interface to see it. Neither user regularly did this, but one who did on occasion said that he did it "to see where I stack up against someone."

There is a widespread culture of sharing screenshots and video clips of interesting parts of games, especially those that are highly competitive and skill-based. While numerous public spaces exist for this, one user was not interested in sharing in these spaces because he felt like his personal bests would compare poorly against more skilled players. He liked the idea of being able to share with his friends and compete against them, but he did not think the platform currently supports it6. Some systems will auto-generate reports about achievements or in-game moments, but one user said he never pays attention to them because they provide no context about the achievement, and most achievements are easy to get and thus the same for all players.

Both users fell into the "multitasking"-type group7, where they make progress in a number of games over a long period of time, much like reading several books at the same time. One estimated he had five going at the same time, and another guessed around seven. There are several reasons cited for this, including variations in genres to suit certain moods (like the moods of the movie watcher above), the novelty of the game, and the number of friends in the user's social network who play the game. Both users said they will play one game for several hours (not consecutively) before switching to another game. This continues until they return back to the first game, and eventually stop playing one entirely8 (according to CNN, only 10-20% of gamers actually finish the games they start).

When selecting new games, users had similar approaches. They read several game-related news sites and social platforms and received information about upcoming games there; these would be tracked mentally or added to a list when possible. Frequently, new games were shared with friends or heard about from others: one user believed that a third of the games he played were either introduced to him by a friend or purchased based on the recommendation of a friend. Users also wanted to play recent games that "everyone was playing," to be a part of the conversation about them. This was further reinforced by the pleasure users found in sharing game-related content with their friends.

Currently, console platforms do not provide these users with the kind of social content they like, provide them the tools to generate it, or give them valuable content that is generated on the platform. They actively seek out community content on websites like Reddit, but do not have access to them on the console interface. Like media viewers, they are not interested in going to these websites on their interface, but would enjoy the content and connections implemented in a usable way on the platform.

The System

The system described here attempts to generally begin to solve the problems of limited social integration, compromised interface choices, and poor social content by improving awareness and interaction enablers in passive ways, demonstrating common ground between both friends and similar users, and offering an easy-to-access location to share9. By passively showing important decision-driving information, users can compare media, see what is popular, and make decisions based on that popularity without needing to seek it out.

Specifically, information about what their friends engaged with, and more nuanced information about how their friends engaged with the material is an important and powerful driver of selection behavior (Geerts & De Grooff 2009; Harper, Sen, & Frankowski 2007) but one that is rarely incorporated into first-level designs. However, users may not always want recommendations from their friends — they may not have the same tastes or might want to find things on their own. Utilizing tracing and personalization algorithms to provide both recommendations (Harper et al. 2007; Wei, Yang, Hseih, & Estrin 2016) and information about similar users is a powerful addition to a social interface. Simply telling users what others with similar taste interact with has been a powerful motivator in other interfaces (Wiebe, Geiskkovitch, & Bunt 2016). Here, it not only gives an extra layer of information for making decisions, it also provides evidence of common ground with other users and a sense of belonging to an in-group10.

Requirements

  1. The system shall allow users to engage with interesting social content in an easy, natural way.
  2. The system shall provide information about a user's friends' media activity.
  3. The system shall allow users to determine what new media they would like to engage with based on their friend's activities and informal recommendations.
  4. The system shall allow users to determine what existing media they would like to engage with based on their friend's activities and informal recommendations.
  5. The system shall allow users to share social content with a specific social network, derived from the media they own.
  6. The system shall allow users to easily and transparently decide what information is shared, who they would like to share it with, and when it is shared, in order to maintain their privacy.

Design Space

Though the social implementation for video and games are similar, certain compromises must be made for the same interactions to be applied to both. Video games are inherently interactive and based on player performance while TV shows and movies are linear and do not require interaction, and so while social recommendations and community content can be shared equally, there are more specific interactions that could be created. For example, games cannot guarantee that all players will interact with the same elements at the same time in the same way, whereas all video media is identical for each user. Social content could therefore be temporally integrated with video elements based on the timestamp or even based on image recognition; in a video game, such an implementation would require tight integration with the developer based on the state of the game.

Additionally, variations in input devices between game-focused devices like consoles and video-focused devices like streaming boxes and smart TVs mean that a range of potential solutions could be effectively implemented. This design document treats both game- and TV-and-movie-focused devices as if they have the same requirements: a limited input device like a remote and a range of content to engage with. However, focusing on one or the other could allow for some problems to be more easily solved for that type of media than for others.

The limitations of the input device is the most challenging requirement to support. A 10-foot interface's on-screen user interface can be largely solved by managing information density, but inputting information is fundamentally limited by the number and style of inputs allowed to the user. Text entry is thus very challenging — assuming no numbers, capital letters, punctuation, or special characters, 26 inputs must be mapped to just a few buttons. The increasingly ubiquity of mobile devices may help alleviate this problem, however, as discussed below in Content Creation.

Design

The Card

The fundamental interactive element is the card. Cards are representations of data objects like social media posts in a feed. From Khoi Vinh (2014), a populizer of the card as user interface, describes it this way:

A card is a single unit of content or functionality, presented in a concise visual package. More advanced cards use that form to surface content or functionality from other apps, and allow users to interact with that content or functionality directly in the context of where a user encounters the card.

Cards

Cards allow varied but similar content to be displayed in a regular manner with clear and consistent interactive elements. There are particularly valuable in social media applications, where content can take many forms but needs to be presented in an understandable and efficient manner. For this interface, the highly shareable nature of content means that cards make an ideal user interface.

All cards in the interface are one of two kinds: interaction cards, which allow the user to launch something, and media cards, which allow the user to like something. They are similar enough in flexibility that they can fit into the same parts of the interface, like a feed, but visually distinct enough that users can rapidly distinguish them from one another without experimentation. All cards have three primary interactions: an expand function, which allows for more detailed information and complex activities to be taken with the card; a context-specific action, which varies by the type of card and the media item it is about; and an activation action, which brings the user to a page specifically about that piece of media.

Social networks typically involve interactions via text, often in the form of status updates and comments. However, a 10-foot interface makes reading large amounts of text challenging, and the input limitations of a controller impair the ability to enter text on its own. Despite these limitations, social features must allow users to express themselves easily and flexibly.

The recent popularity of emoji may provide a solution. Rapid, easy access to emoji as a first-level replacement for an on-screen keyboard allows users to express their own feelings and understand the feelings and responses of others without the usage of text. This quick response-style communication has been used effectively in other limited input systems. Rocket League, for example, offers 16 quick responses that can each be accessed with two presses on the directional pad, but based on the context of the game, communications from joy to sarcasm can be expressed. Phantasy Star Online allowed users to combine icons and emoticons into "sentences" that expressed meaning without words, allowing for language-independent communication. A similar solution is stickers — essentially, large emoji designed to be sent individually, without any additional text. Stickers could be implemented on a per-user and per-media basis, with the potential for media creators to sell sticker packs based on their specific content.

However, while pictographic communication may be effective for many simple responses, more complicated thoughts will necessitate the use of text. Providing a keyboard as a second-level choice, with users strongly encouraged by the UI to prefer emoji responses, may be required in a 10-foot interface. An effective method for typing with a controller is challenging to find; presently, the majority of interfaces rely on a hunt-and-peck style interface with the user moving a cursor over each key on a keyboard and pressing a button to append it to a text string. While simple, this is slow and only uses two of the inputs on a controller. Steam's recently-abandoned Daisy Wheel interface was an interesting experiment in better controller typing: 16 "petals" were mapped to a directional stick, and tilting it in a certain direction would display letters that mapped to the ABXY buttons on the right side of the controller.

Daisy Wheel

Cards should display thumbnails of the content they link to and direct access to the source the content is from. Screenshots should be miniaturized and shown, videos should autoplay on a loop, and achievement information prominently displayed. Enough information should be shown that the user is not required to open new viewport to see the content — instead, the size of the card should be adjusted based on some factor like popularity or importance. A user who sees the beginning of a muted video in their feed and finds it interesting should be able to watch the entire thing at a reasonable size with relative ease.

Interactions should be accessible from the feed, not isolated in a new viewport. The ability to like and comment on a post should be mapped directly to buttons on the primarily input device, and would not be used for different functions in similar contexts. For example, if the X button allows a user to like a community post, it should not be used for adjusting options on an interaction card. Limiting the steps required to engage with the community is fundamental in ensuring that users are interested in doing so. Requiring three steps to see, open, and like a post is frustrating — allowing the user to do it in a single button press in its original context is far easier.

The number of steps the user must take from turning the system on to engaging in their intended activity must be as few as possible. Presenting recently used applications and regularly used applications on the home screen is essential. Users have become tolerant of launch screens, but these should be carefully applied only to applications where outside information can enhance the user's experience. While knowing how many friends are playing a game is valuable information, knowing how many friends have installed Netflix or YouTube applications is not, and ubiquitous applications should not include launch screens.

Feeds

In existing systems, users rarely engage with community and social content, often because it is uninteresting or adds no value to their time with the system. Improving engagement with community posts involves improving the signal-to-noise ratio and increasing the number of engagement points (Kietzmann, Hermkens, McCarthy, & Silvestre 2011). By providing content that users are interested in, they are more likely to interact with it and create their own, and by reducing the friction involved in engaging with the material, users are more likely to consistently and regularly use these features. This can be done by eliminating posts with limited value, enhancing the value of existing posts, and allowing for user-generated content to be shared easily, while simultaneously making it easy to interact with this material.

Of key importance is the understanding that a console is not a social network — users are not using it to engage with their friends or the community as a whole. They are here for a specific media- or interaction-related task like watching a movie or playing a game. For this reason, isolating the community elements will mean they are rarely ever used. Instead, they must be included in the homepage where they are obvious, easy to use, and actually enhance the value of the game or media — but also be easy to ignore and not get in the way of allowing the user to do what they came to do.

The proposed solution to this problem is to integrate social content with non-social tasks, like management, metadata, and navigational activities into a series of feeds. Similar to the News Feed on Facebook, a feed contains information relevant to a specific area like friends or games. While a feed can be an entire portion of the interface, they are likely more valuable and useful when they are attached to other parts of the interface and supplement functions or information already there. For example, a game screen needs to provide the user functions like launching the game or checking for updates, so a feed acts as supplementary information to that.

Two different but related feeds are designed. The media feed contains information related to the specific media — a movie, TV show, or video game. All information in this feed is specific to that piece of media, with an emphasis placed on social posts within the user's friends list and social network but also containing information from the larger community including blogs and social networks. The home feed contains a combination of feeds from all of the user's recent- and regularly-used media, sorted by importance.

Content

No matter how diverse the content on a feed, if it fails to be interesting to the user it becomes irrelevant at best and annoying at worst. Content should be subtly shaped to appeal to the user's needs and desires (Kietzmann et al. 2011). If they spend the majority of their time gaming, tune it towards that, with resolution of shaping as fine-grained as possible. Favorite and regularly enjoyed genres, publishers, developers, directors, actors, similarity with other users and friends — all of this information should be used to ensure that content displayed to the user is of interest and value to them. If many of a user's friends are engaging with a piece of content, show it to the user — but this rule does not necessarily apply to content that is popular within the entire network. If the user rarely views that kind of content, it should be phased out of the feed.

Community posts must provide added value for the user. Because auto-generated information about achievements or screenshots don't take into account the metacommunity surrounding the game or the user's interests, they are ignored and are rarely interacted with. Successful game- and media-related communities on Facebook, Reddit, and Steam frequently have conversations and content about the topic, as opposed to from it. In instances where screenshots or videos are shared, they are unusual, exciting, or otherwise interesting media that most users are unlikely to see in any other context. For example, in a game, special achievements, high-level character abilities, or videos demonstrating highly-skilled actions are very popular. It is essential that metaposts about a game or media source like image macros, animated gifs — or gif-like videos11 —, guides, conversations, and questions must be easily created and shared.

One likely popular avenue for community post consumption is when the user has no access to the console but still wishes to engage with game- or media-related material. This may take the form of reading articles about the content, watching videos, or discussing it with others, and the community posts can act as a hub for this information, available both on and off the console. Web or mobile applications can provide access to the information the user wants, flavoring it towards their preferences and friendgroups, all still within the console's ecosystem.

Content Creation

The relative difficulty of creating visually complex and text-based content using a limited-input device like a remote or a gamepad presents a challenge for community-created posts. While effective for simple tasks that do not require a great deal of precision, limited-input devices become much more difficult to use in graphics editing and typing. Community posts must fall within the community's norms, but also be made consistent so they stand out from both each other and other items in a feed. I believe there are three useful solutions to this problem: create a limited-input device-focused interface that negates as many of the downsides as possible, replace the limited-input device with a mobile device for use with an on-console interface, or completely offload the content creation to non-console devices.

Creating content on the console with a limited-input device ensures that all users will have equal access and availability to the tools, but presents significant trade-offs for effective implementation. The tools must be simple and intuitive enough that they are not challenging or too time-consuming to use regularly, but robust and powerful enough to provide both rudimentary video and image editing. Text needs to be placed and colorized with specific (and often community-codified12) fonts, videos trimmed, and the ability to draw upon images added. By limiting the flexibility of these tools, they become easier to use for some tasks but hard for highly creative applications.

Replacing or augmenting the limited-input device with a mobile device allows for more direct interaction and finer-grained control — users can touch and drag elements around on a screen, for example, and move through menus more quickly — but results in the limitations of the mobile touchscreen platform being applied to the console. It also limits the pool of users to those who have a mobile device compatible with the system, have installed the app, and are able to effectively synchronize it with the console13.

Some content creation can be automated. Achievements are already created and posted to community feeds, but all achievements are posted despite varying levels of difficulty involved in completing them. A simple algorithm that decides which achievements to post based on how many friends or worldwide users have completed a challenge can act as a basic filter to remove irrelevant and uninteresting achievement posts from a feed. Achievements then go from being a stream of the common to the unique and impressive among friends — essentially turning them into bragging rights, which are important for gamers. Challenging achievements often require particularly skillful play, and utilizing existing recording features to provide a brief video leading up to and immediately following the achievement would further enhance the value of these achievement posts. Users can not only demonstrate their abilities through an achievement, they can also prove it by showing video evidence of their skills.

Further filtering of achievements can lead to implicit competition between friends, which can enhance engagement with media (Yoo & Alavi, 2001). The system can help with comparisons between the user's and their friends' achievements by displaying those achievements that many of their friends have but that they do not. Merely suggesting that the user is lacking the same achievements with a call to action — e.g. "13 of your friends have this achievement, can you get it too?" — may make users more likely to intentionally carry out action to receive that achievement. These can be dynamically adjusted to a goal just within the user's reach, making it something challenging but still attainable, tuned the the user's progress and skill level.

Better information about game states and data can be leveraged for similar competitions. Displaying a friend's high score or statistic while comparing it to the user's provides another form of competition. For example, in a soccer game, users can be engaged by creating posts that say how many goals a friend is from a milestone: "twist4 is 36 goals away from 10,000 — go beat him to it" or "Häagen-Dazs-Bioroid just beat your record of 500 epic saves." These types of post serve a dual purpose of providing a target for a user to aim for while simultaneously giving them context of their place within a larger community.

(Social) Engagement

Ultimately, community and social posts need to make a user feel like they are sharing an experience with a larger network of people (Hogan 2010; Kietzmann et al. 2011), whether that is within their friends, a group, or the entire userbase. Frequently, a good community post becomes popular because it is something that many users can sympathize with — a "me too" feeling that suggests their opinions and experiences are those of the in-group.

Popular games, media, or events can provide another avenue towards the in-group feeling. When a large proportion of a user's friends are doing similar things, like watching the same show in a streaming application or playing a newly-released and popular game, the interface should present this to the user and make it very easy to join in with these actions. If several friends are involved in the same activity at the same time as the user, automatically providing a group chat platform further enhances the sense of membership.

Friends lists differentiate between users that are online — often with some sort of further status like away or idle — and those that are offline. Frequently, offline users are listed with when they were last online, but with no additional information. This could be improved by simply saying what the user was last doing before they want offline — what show they were watching, what game they were playing, etc. Additional metadata can be provided, like specific actions that were taken or interesting statistics about the user's activity.

This can be further extended to information about individual pieces of media. Oftentimes, launch pages show data about which friends are playing games but often fail to indicate how long or when those users were engaging with it. Providing this information allows users to develop a sense of whether that media is still popular amongst their friends and can even act as a form of community-based reviews: for example, a game that has been played for a short amount of time by most friends suggests that it is either not popular or not well-made — and either way may not be of interest to the user. However, a user may know that their friend shares similar tastes, and seeing that a friend has recently put significant time into a game indicates that the user may also enjoy the game. This can be applied to other forms of media like TV shows and movies, perhaps by replacing how long the friend has played with information such as how many episodes the friend has watched or how many times they've rewatched a movie. These data should be displayed on game launch pages, but can also be shown on store pages to buy games. Knowing that a number of the user's friends have purchased a game is likely to make the user interested in it as well.

Ethics

The nature of entertainment platforms has a limited potential for ethical issues. A common concern with increasingly connected devices and social media is privacy. It is essential that users have clear privacy settings that allow complete control over their information. Users should be able to easily control who sees what content, both by default and on a per-post basis. It should be easy to tell who is going to see a user's information, and in a worst-case situation, easily and permanently remove information.

Study and Measurements

The Prototype

View the prototype here.

Using a medium-fidelity computer prototype that matches the limitations of the final product is in line with the literature, which has found no relationship between fidelity and usability (Landay, Takayama, & Walker 2002). It is essential that users have to deal with the same limitations in the prototype as they would in the real world which rules out most low-fidelity techniques. Any interactions the user finds easy with the mouse cannot be compared to interactions with a remote or gamepad. However, using a medium fidelity prototype made things more complicated — there aren't any prototyping or wireframe platforms that allow for keyboard inputs, so a new had to be written.

A content-filled wireframe was created that users could interact with using just a few buttons. Specifically, they could use the arrow keys and three buttons — two different kinds of interaction buttons and one to go back. The content was made up TV shows, games, and movies, showing the different numbers of friends and percentages of similar users had interacted with it.

The prototype displays cards arranged in a number of feeds. This prototype focused on the home feed, which contains social information from all the user's media, friends, and interests, and the media-page feed, which contains social information related to that specific piece of media, be it game, movie, or TV show. There are also media and store pages, where the user can view media they "own" and media they can "buy." This design provides multiple routes to engaging with new and existing media based on what is interesting to the user's friends. Users will likely take different approaches to each task, whether that search starts through a card on the home feed, through the store, or somewhere else.

This prototype is unable to allow for launching any media, which presents a problem for the third task — sharing media. The system as described requires that users can customize what they share — whether that's screenshots or video of their current media — and that requires a much larger backend infrastructure.

A challenge of designing this system is that a large part of its effectiveness comes from the content displayed. It needs to be engaging enough to draw the user into opening the card, and then the media that card is tied to, and the only way to do that is for it to be populated with things that are interesting to the user. Tastes are very different between users, so what one user finds interesting and want to open is different than what someone else will. Since this system is designed to support more community and social engagement, a better prototype would tie into existing social information to create things that are actually targeted to the user. Unfortunately, a key element of the system is new and better types of social information which does not exist and cannot exist without an entire system infrastructure.

Evaluation

Participants were given an initial description of the goals of the system, the prototype, and how it worked. They were encouraged to spend time interacting with it in order to understand its hierarchy and navigational elements. Because the prototype only works with a keyboard, it was important that users understand what each key does. If this is too complicated to quickly understand, it is likely that a final implementation will also be too challenging for most users.

Next, participants were asked to complete two tasks: one to select media to interact with that they already owned, and then to buy new media. No information was given to participants about how they should make this decision, and neutral, fictional media types were created to avoid recognition and familiarity with existing properties. Participants could view information about how many friends had interacted with something, how long, and when, as well as information about users with similar tastes and aggregated review scores. Content was split into four groups:

  1. High friends, high similar users
  2. High friends, low similar users
  3. Low friends, high similar users
  4. Low friends, low similar users

It was expected that users would gravitate towards media that fell into the first category regardless of personal preference. They would choose the second or third category based on whether their tastes aligned with their friends' or not; and few users would select media in the fourth category.

In the first task, users were instructed to select the best piece of already-owned media they could. In the second task, users were given the same instruction but told that they had $100 to spend; after every "purchase," they were told how much of their money remained. Participants were allowed to purchase as much media as they wanted as long as they remained within this budget. This was intended to provide extra pressure towards making the best decision they could within the second task.

Here, users cannot make errors, they can only make decisions that are less optimal than others in terms of enjoyment. This, coupled with an inability to show them the media they selected, precluded the ability to rank their decisions as successful or not. Additionally, because users are not trying to complete tasks as quickly as possible, it was not necessary to record how long it took them to complete a task. Instead, it was more important to focus on what users thought was best, why they thought that, and how to provide them the information they need to make the best decision.

A think-aloud use test (Nielsen 1993) was used to determine users' mental model regarding their selection process. This allowed me to ask about what information they were using to make their decisions, what they weren't using, and importantly, ask why they used that information. The think-aloud task can be challenging for users who can accidentally begin describing their actions and not necessarily their thoughts; in order to prevent this from occurring, users were encouraged to explain what they were thinking before each task and were verbally prompted for their thoughts and rationales when they fell quiet, shifted their activity, or began describing their actions.

After using the prototype, an in-depth, open-ended interview was conducted to determine their feelings, thoughts, and preferences about including social information on a 10-foot interface in general, and social selection and media sharing in particular (Appendix A). Users were asked about what information they used and why, how they made their selections, and how they felt about sharing information. Interview questions were developed to be non-leading and open-ended in order to allow probing of the user's mental model, in accordance with the Pathfinder recommendations (Boyce & Neale 2006). Open-ended interviews have long been used as a powerful method of extracting information from users (Berry 1999).

Finally, participants were given the system usability scale (SUS), a tool for determining users' opinions about an interface[^single-score]. Described as "quick and dirty," (Brooke 1996) the system is nonetheless highly reliable and widespread (Usability.gov) even with small numbers of participants (Sauro 2011). Because the questions are the same from test to test, it allows comparisons to be easily made. During the SUS administration, participants were encouraged to offer their thoughts as prompted by each question.

Results

Participants

The average age of participants was 25, consisting of two women and four men ages 24 to 28. Users were selected based on their experience with existing 10-foot interfaces and their familiarity with socially-derived information. All users currently owned or regularly interacted with a 10-foot interface, with an average estimation of 5.2 times per week. Two participants said that they used their 10-foot interface twice a day, every day.

Two users interacted with the prototype in a living room environment from 10 feet away. Four were interviewed remotely and did not use the prototype on a TV. Instead, they interacted with it on their own personal computers.

Table 1. Participant demographics.

Participant Age Gender Interactions with 10-foot interface per week
P01 24 F 7 (Twice daily)
P02 25 M 5
P03 26 M 3
P04 28 M 7 (Twice daily)
P05 24 M 2
P06 24 F 7

Living room interface. Figure 1. The setup for testing in the living room.

System Usability Scale

The average SUS score of 83.5 (SD=14.20) was calculated for five of the six participants (one did not answer the questionnaire). The lowest recorded score was 57.5 and the highest was 97.5. The SUS requires that answers to the odd numbered questions have one subtracted from them, and answers to even numbered questions are subtracted from five. This provides scores ranging from 0 to 4, which are then summed and multiplied by 2.5 to obtain a score from 0 to 100.

Table 2. System usability scale answers; higher scores for odd questions and lower scores for even questions indicate a more usable system. Participant #2 did not answer the system usability scale.

Participant Number SUS1 SUS2 SUS3 SUS4 SUS5 SUS6 SUS7 SUS8 SUS9 SUS10 Calculated Score
P01 5 1 4 1 4 1 5 1 5 2 97.5
P03 4 2 5 1 3 2 5 2 4 1 82.5
P04 3 1 4 1 4 1 5 2 5 2 85
P05 1 1 4 1 2 4 5 3 4 1 57.5
P06 5 1 4 1 5 1 5 1 4 1 95
Avg 83.5
SD 14.20

The average system usability score of 83.5 places the prototype interface in the highest 10 percent of all interfaces for usability. Scores above 80 fall in the range for which users would recommend an interface to their friends (Sauro 2010); all but one user individually rated the interface to this level. Every user thought the system was easy to use and easy to learn, with all participants rating 5/5 for the SUS prompt "I would imagine that most people would learn to use this system very quickly."

Media Selections

Media was presented with information about similar users' behavior and the number of friends who had interacted with it. Users were asked to select media they already owned and new media they would like to buy. In the first task, users were asked to select a single piece of media. In the second task, participants were allowed to select more than one piece of media.

Table 3. Selection results for each type of media, by task.

Selecting Owned Media Selecting New Media
High friends/High similar 2 6
High friends/Low similar 1 2
Low friends/High similar 3 4
Low friends/Low similar 0 3

This data was used in conjunction with interview statements to determine what sources of data participants used to make their media selections. Users who selected media with high friend and similar user ratings preferred to get their information from those sources unless they explicitly stated otherwise during the interview. All participants acknowledged the existence if this information, and two stated in the interviews that they made the selections they did because of the friend data displayed.

Table 4. Participant media selection sources. High/medium/low social selections are the total number of media selections participants made, broken down by the ratings described above.

Participant Preferred Selection Source High Social Selections Medium Social Selections Low Social Selections
P01 Friends 3 0 1
P02 Reviews 0 2 2
P03 Reviews 2 0 0
P04 Friends 2 3 0
P05 Reviews 0 2 0
P06 Friends 2 3 0

During the think-aloud task, P01 said they were searching for social information and used it to make their decision. The one low-social choice they made was to buy something that looked interesting but was unrelated to their friends. P03 explicitly stated that they were ignoring the social information and made selections based on the review score. P06 stated that they were "more interested in what similar users watched" than what their friends did; all of their medium-social selections were of media with low friend counts but high user similarity scores.

In general, review-based users demonstrated use of the lexicographic decision making strategy. This strategy "selects the option with the best value on the most important attribute" (Riedl, Brandst├Ątter, & Roithmayr 2008) and uses less-important attributes to break ties. Participants looked for media that had the highest review score from their preferred source — either IMDB or Rotten Tomatoes — and if two sources were tied, they used the alternative source. One user said that they felt "trained to look for 80s and 90s [percentage numbers], because I'm a very Rotten Tomatoes-based person." When deciding between two similarly-rated movies, they used the IMDB score to break the tie.

Friends-based users preferred the elimination-by-aspects strategy, which removes options that do not fall above a minimum value on the most important attribute. Though not explicitly stated, users said they were looking for media with either high friend values or high similar users, and ignored media that did not fall above a certain threshold. One user explicitly called this out when making a selection; when trying to decide between two options, they noticed that the score of one was "too low" and selected the other.

No users selected the least variance heuristic, which selects options that have the least variation between different attributes. Here, that would be the least variation between the number of friends who watched the similar users rank, between the different review scores, and between both groups. Instead, users focused on a primary source of information to make their decisions.

A strong driver for some friends-based users was information about what friends were doing now; one said they were less interested in what their friends had been doing and much more interested in what was going on at this moment. One user said that if friends were online, they were more likely to play the same game as their friends than another one. Another said there are some games they would only play with friends, and would switch between multiplayer games and single player games based on who was playing a game and their current environment. One user was driven by reviews so strongly that they ignored the social information entirely.

Qualitative Data

These data are selected quotes from both the think-aloud task and the post-task interview.

Selection Information

Users established themselves as relying largely on reviews or largely on their friends, and preferred that kind of information displayed on the main pages. Three users relied heavily on social information, and three on reviews. With information about friends and similar users, participants typically expected more information immediately available without needing to enter a second screen.

  • "I like the 'your friends did this' [section]."
  • "I'm trained to look for 80s and 90s [percentage numbers], because I'm a very Rotten Tomatoes-based person."
  • "I wish IMDB and Rotten Tomatoes were swapped with friends [on the main pages]."
  • "The combination of friends and users like me is important."
  • "I saw my friends were playing now and I just jumped in without checking anything else."
  • "I might trust [my friends'] taste," but I trust the [users like me] more."
  • "[This is a] hot game for right now — I should be getting up into it."

Home Feed

Some users primarily navigated using the home feed. They liked the display of social content mixed with promoted media, especially information about what their friends were doing now.

  • "If I clicked on something and it said 'buy this game,' that would be a huge bummer." In reference to promoted items on the homepage.
  • "Maybe I'd play Trainrealm to try to catch up."
  • "[I'm] mindlessly scrolling through until something interesting comes up."
  • "Don't make me think and then we're good."

Sharing and External Content

Participants were generally interested in sharing and seeing what their friends shared, but only if what they were sharing was very interesting or unique. Most participants wanted to only share things with their friends, but didn't mind seeing content from other sources like Reddit. Some (both review- and friends-based) actually relied on this information for selecting media — they looked for information about games from outside sources like screenshots and videos to help make their choices.

Achievements

Participants regularly explained their dislike of achievements. As expected, this was largely due to the high volume of low-importance information.

  • "I hate achievements." Later explained that this was because they were far too common and uninteresting.
  • "I would rather see incidentally funny things, really amazing things that you've done."
  • "Everyone has the same ones, I don't need to see a huge list of that."

Discussion

Friends Versus Reviews

Perhaps the most important finding was that users fell into two different groups when it came to deciding on media: those who relied heavily on information about their friends and users like them, and those that relied heavily on reviews. Some users were uninterested in what their friends were engaging with, while others said they didn't trust their friends' taste. Those who liked the social information said they wanted to see what their friends were doing so they could talk about it with them and be in the loop. Both groups wanted to see their preferred source of information on the main pages so they could compare items without needing to dive into the interface hierarchy.

This presents a challenge for an interface. Users have exclusive preferences for what information they want, but there is limited real estate for presenting it. Allowing users to customize what information they see is a potential solution; perhaps with information gleaned from an interview during the initial setup to ask from whom they want to get their recommendations. Certainly, more information needs to be displayed — one user suggested that when the cursor moves over an item, it expands to present more information without entering a new screen. This additional space would allow both review and social information to be displayed simultaneously.

A surprising finding was how powerful information about friends doing something now was for friends-based users. Just seeing that a friend was currently playing a game or watching a movie overrode the user's selection strategy in favor of that media. This effect should have ramifications for designing a future interface: provide clear information about the current activity of friends, and make it easy for users to engage with the same media.

The users like you ratings were a potential source of displeasure for some users, who were concerned about the implications for tracking this involved. One user said they would be comfortable if the information came from within the system but not if it involved tracking behavior from places like Facebook or Google. It is essential that users can easily change their privacy settings, with clear information about how changes to settings will impact their shares, social footprint, and inclusion in anonymized statistics. This should not have a default opt-in or opt-out state; users should set up their own privacy settings during initial setup.

Sharing and Social Features

The biggest limitation of this study was the inability to explore sharing behaviors with the interface. The complexity of implementing a backend for sharing and viewing shares was outside the scope of this study. Instead, contextual interviews about sharing behaviors, preferences, and desires (Appendix A) were conducted, with specific questions about potential implementations.

Most users reacted positively to both sharing and seeing the shared content of others, as long as it was easy to access and interesting. Users were particularly interested in sharing content they thought was entertaining to their friends — funny moments from TV shows and movies, animated gifs, and screenshots and videos from games they've played. One user said that he wasn't interested in sharing now, but he would be if it was easy to do — and that he would be very interested in seeing what his friends had shared. This was a common refrain amongst all users. They wanted it to be very easy to share content when they wanted, especially in games where shareable, interesting moments can be highly unpredictable. One user suggested that content be recorded passively in the background and then shared with a command.

Users were interested in seeing their friends' content, but only if they didn't have to look for it. Isolating content in a community tab — the way current systems like the Xbox One are implemented — would be a dealbreaker for most users. One went so far as to say that they would never look at any social content if it wasn't displayed on the main page. Overall, users like the idea of content on the home feed so they could easily browse through it, but could also easily ignore it and immediately perform the task they wanted to do. One user phrased their preference as "don't make me think and then we're good."

Achievements were a constant source of frustrations. One user adamantly objected to their inclusion, with one exception: those that were "exceptionally interesting." This was echoed by a number of game-focused users who were frustrated by the amount of boring, same-y content that failed to present any new information about their friends and activities. Participants wanted to see when their friends did something particularly brag-worthy or exciting — and wanted to share similar material. Automatically displaying friends' achievements that were rare or unusual was a popular potential implementation.

Users were split on the idea of including external content. Some users were interested, while others felt that they could better interact with that content on another platform. One pointed out that their primary method of interacting with a 10-foot interface was relaxing on the couch; reading or interacting with long-form content was not something they wanted to do. Several users mentioned the idea of short-form videos or images ("like Snapchat!"), which could be imported from external websites. The primary lesson drawn was that users are more interested in their friends' content, but external content in the right form could also enhance their experience.

The Mental Model

When users are searching for media to watch, they are looking for specific, preferred information to support their decision making. In this study, users tended to prefer either social information — what their friends and users with similar preferences were interacting with — and review sources. The best choice can be predicted by their decision strategy, which is based on the information available and their preferred source. Users do not decide on what source they prefer at the moment of selection, nor does it change between selections or over a short period of time. Instead, it is based on previous experiences demonstrating that their preferences either align with their friends' preferences or with reviews.

Issues

There were only a few issues users brought up; most were related to issues with the prototype specifically.

Table 5. Usability severity issues with the prototype.

Severity Problem Solution
Cosmetic Not enough visual appeal; More thumbnails, icons in interface This is solved by using actual media for content. One user wanted to see icons in place of labels for users, friends, and reviews. Thumbnails would allow for easier gestalt recognition and browsing.
Cosmetic Mis-aligned splash screen buttons The "Watch/Play" and "Share" buttons on media splash screens were misaligned for some users and covered feed content. This needs to be corrected in the final version as it obscured important information.
Minor Confusion over similar users label Some users misunderstood the "users similar to you" label; one thought that it was the average user review score and did not realize it said "users similar to you" until mid-way through the test. Other users did not understand what the label meant without some explanation.

Reflection

Usability testing and evaluation is an essential part of the design process. Designing without it is akin to working in a vacuum. Without context, expertise, and experience a designer is incapable of seeing potential problems which severely limits their ability to fully understand someone else's point of view. In order to create highly usable and effective interfaces, designers must work with users to make sure their needs are being met. What makes sense for the designer may not make sense for users — and what makes sense for one user may not make sense for them all. Limitations to sample sizes mean that designers must take the feedback they receive from users and then use their own expertise and knowledge to improve their interface.

For this project, it was surprising to find how neatly users fell into two groups and how strongly they wanted their preferred source of information presented on-screen. This result was not anticipated before the study and only reinforces why testing and evaluating is critical for a successful interface. One round of testing is rarely enough; if this interface was being taken public, it would require multiple rounds of evaluation and testing to make sure that the system was successful for users.

Conclusion

Limitations

Perhaps the largest limitation of this study is that it could not test sharing behaviors using the prototype interface — the system simply did not support actually *engaging* with that media. Interviewing participants after using the prototype revealed information about how they say they would share, but oftentimes what users say they would and and what they actually do are very different.

Future Work

Some users mentioned that they spent a lot of their social interactions with a 10-foot interface with co-located friends. They will take turns sharing media on the screen, either by casting from their mobile device or by taking turns with the remote. This is a unique use-case that with different requirements than the above, but is still a social usage of a 10-foot interface.

An extremely common TV- and movie-watching practice is to use another device simultaneously while watching. The second screen could allow for more complex social information to be displayed and interacted with, like creating content on the second screen device for displaying on other users' 10-foot interfaces. How these devices are being used and how they can be effectively integrated into a 10-foot interface-based social workflow is a rich avenue to explore.

Finally, a number of users engage with media on non-10-foot interfaces, but many of the same issues with social content on 10-foot interfaces remain on these small screens. Are users of these devices more amenable to interruptions and multitasking? Can social information be displayed alongside the media?

Findings and Recommendations

Currently, no media streaming platforms like the Roku and Apple TV incorporate any kind of social information, and other 10-foot interfaces that do often do it incorrectly. Social content is often corralled into a separate section of the interface that requires users to go out of their way to find it — and even if they do, the information there is boring, repetitive, and irrelevant. Remedying this problem means providing information users want to look at, like rare achievements and relevant, external social posts, in ways where they can passively access it. It also means adding passively-available information to help users make selections using the sources they want.

This is especially important because not all users rely on the same sources of information to make their decisions. Users typically rely either on social sources, like what their friends or similar users are engaging with, or on reviews from their favorite sources. Interfaces need to cater to both types of users, providing them what the need and not showing them what they don't; this has implications for interface design and customization.

Users are particularly driven by belonging to a group and what their friends are doing now. Showing users what their friends are playing or watching at *this moment* can cause them to choose that option to engage with over anything else, regardless of other social information or reviews available. Additionally, users want to know what their friends are watching so they can watch it. This is true for gamers as well, but they want also want to play in order to compete with their friends. This can be catered to by providing calls to action, which show comparisons of key metrics like points in games between users and their friends.

Social media is everywhere, and there is enormous demand for social content about video games, movies, and TV shows. Currently, a hugely common way of engaging with media — the 10-foot interface — doesn't cater to these desires and in some cases actually makes it *harder* to do. The first interface that gets this right will quickly become widely adopted: people just want to talk.

Acknowledgments

T — the above and more.

C, T, M&D — the motivation to get started and stick with it.

T, D, C, E, C, A — letting me inflict my study on you.

T & S — incredibly thorough edits.

Citations

Anderson, M. (2015). Technology device ownership: 2015. Pew Research Center. Retrieved from http://www.pewinternet.org/files/2015/10/PI2015-10-29device-ownership_FINAL.pdf

Altavilla, D. (2008). Inside the tech of the Netflix player with Roku. Hot Hardware. Retrived from http://hothardware.com/news/inside-the-tech-of-the-netflix-player-with-roku1.

Apple Inc. (2007). Apple TV now shipping [Press release]. Retrieved from http://www.apple.com/pr/library/2007/03/21Apple-TV-Now-Shipping.html

Brooke, J. (1996). SUS — a quick and dirty usability scale. In Jordan, P. W., Thomas, B., Weerdmeester, B. A., & McClelland, I. L. (Eds.), Usability Evaluation in Industry. Boca Raton, Florida: CRC Press.

Buffone, J. (2012). Internet connected TVs are used to watch TV, and that's about all. The NPD Group Blog [Web log]. Retrieved from https://www.npdgroupblog.com/internet-connected-tvs-are-used-to-watch-tv-and-thats-about-all/#.VvahtDZ57e0

Cesar, P. & Geerts, D. (2011). Past, present, and future of social TV: A categorization. 3rd IEEE International Workshop on Social TV — The Next Wave.

Chorianopoulous, K. & Lekakos, G. (2008). Introduction to social TV: Enhancing the shared experience with interactive TV. International Journal of Human-Computer Interaction, 24(2), 113-120.

Ducheneaut, N., Moore, R. J., Oehlberg, L., Thornton, J. D., & Nickell, E. (2008). Social TV: Designing for distributed, socialable television viewing. International Journal of Human-Computer Interaction, 24(2), 136-154.

Geerts, D. & De Grooff, D. (2009). Supporting the social uses of television: Sociability heuristics for social TV. CHI 2009.

Girgensohn, A., & Lee, A. (2002). Making web sites be places for social interaction. Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work, 136–145.

Harper, F. M., Sen, S., & Frankowski, D. (2007). Supporting social recommendations with activity-based clustering. RecSys'07.

Hogan, B. (2010). The Presentation of Self in the Age of Social Media: Distinguishing Performances and Exhibitions Online. Bulletin of Science Technology and Society, 30(377).

Kietzmann, J. H., Hermkens, K., McCarthy, I. P., & Silvestre, B. S. (2011). Social media? Get serious! Understanding the functional building blocks of social media. Business Horizons, 54, 241-251.

Landay, J. A., Takayama, L., & Walker, M. (2002). High-fidelity or low-fidelity, paper or computer? Choosing attributes when testing web prototypes. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 46(5), 661-665.

Nielsen, J. (1993). Usability engineering. Boston: Academic Press.

NPD. (2015). Streaming media player penetration to reach 40 percent of U.S. internet homes by 2017, according to NPD. Retrieved from https://www.npd.com/wps/portal/npd/us/news/press-releases/2015/streaming-media-player-penetration-to-reach-40-percent-of-us-internet-homes-by-2017/

Sauro, J. (2010). Does better usability increase customer loyalty? MeasuringU [Web log]. Retrieved from https://web.archive.org/web/20160323174528/http://www.measuringu.com/usability-loyalty.php

Sauro, J. (2011). Measuring usability with the system usability scale. MeasuringU [Web log]. Retrieved from https://web.archive.org/web/20160324234839/http://www.measuringu.com/sus.php

Sherif, M. (1936). The psychology of social norms. Oxford, England: Harper.

Snow, B. (2011). Why most people don't finish video games. CNN. Retrieved from http://www.cnn.com/2011/TECH/gaming.gadgets/08/17/finishing.videogames.snow/

Statista. (2015). Most popular television series in 2015, sorted by average number of tweets (in 1,000s). Retrieved from http://www.statista.com/statistics/304363/us-twitter-tv-series-audience-buzz/

Vinh, K. (2014). What is a card? Subtraction.com [Web log]. Retrived from https://www.subtraction.com/2014/08/26/what-is-a-card/

Wei, H., Yang, L., Hseih, C-K., & Estrin, D. (2016). GroupLink: Group event recommendations using personal digital traces. CSCW'16 Companion.

Wiebe, M., Geiskkovitch, D. Y., & Bunt, A. (2016). Exploring user attitudes towards different approaches to command recommendations in feature-rich software. IUI'16, 43-47.

Usability.gov. System Usability Scale (SUS). Retrived from http://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html

Yoo, Y. & Alavi, M. (2001). Media and Group Cohesion: Relative Influences on Social Presence, Task Participation, and Group Consensus. MIS Quarterly, 25(3), 371-390.

Appendices

Appendix A — Protocol

Not all questions were asked of all users; instead, questions were asked that attempted to elucidate the following items:

  • The information they use
  • How they compare items
  • What they decide is worth their time and money
  • How their friends affect their decisions
  • How other users affect their decisions
  • How the visual availability affects their decisions
  • Which factors are most important and which are least important
  • Sharing behavior
  • Social media behavior

The questions listed below should be seen as a skeleton and not necessarily the exact phrasings.

Today I'm asking for your feedback on a prototype interface designed for TVs. You're already familiar with a console or streaming box like an Apple TV; this is designed to enhance that experience by providing information about what your friends and others are doing.

Spend some time getting accustomed to the interface. Most of the keys are on-screen, but you can also press escape to go back. Your primary interaction button is Z. Let me know when you're ready.

---

I want you to complete a couple of tasks today involving selecting media. The actual buy and play buttons don't work, you just need to tell me when you use that button if it actually did something.

That you already own:
    Pick the best game you want to play.
    Pick the best TV show or movie you want to watch.

    How do you know it's the best?
    How do you know you'll enjoy it?
    What made you pick it?

New:
    You have $100 to spend. I want you to pick the best combination of movies, TV shows, and games that you want to own. I'll keep track of how much money you have left.

    How do you know it's the best?
    How do you know you'll enjoy it?
    What made you pick it?
    Why did you decide to spend your money on that?

Would you be or more or less likely to play ____? How come?

---

Now I have some questions for you that will help guide development in the future.

Would you share screenshots, clips, or videos of what you play and watch with your friends? Why or why not?
    When would you do that?
    What would you want to share?
    What wouldn't you want to share?
    What would stop you from using it?

Would you want to see your friends' shares?
    What would you want to see?
    What wouldn't you want to see?
    What would be annoying?

Do you own a game console?
    How much do you interact with the social features now?
        What do you do with them?
        What do you like seeing? What don't you like seeing?

How often do you talk about media with your friends?
    Do you buy things because your friends have? Have you ever?

How often do you interact with a 10-foot interface?

Appendix B — System Usability Scale

I'm now going to read you a series of statements. I want you to tell me how much you agree or disagree with them, ranking them from 1 to 5 with 5 being "strongly agree."

I think that I would like to use this system frequently.
I found the system unnecessarily complex.
I thought the system was easy to use.
I think that I would need the support of a technical person to be able to use this system.
I found the various functions in this system were well integrated.
I thought there was too much inconsistency in this system.
I would imagine that most people would learn to use this system very quickly.
I found the system very cumbersome to use.
I felt very confident using the system.
I needed to learn a lot of things before I could get going with this system.