Interactions

Authors:
Aaron Marcus, Eugene Chen

---

  Introduction

In the summer of 2000, the User-Interface Research and Development department of Samsung Electronics Corporation approached AM+A to develop conceptual user-interface designs for future wireless information devices (WIDs) evolving from cell phones and targeted for the year 2002. This project involved market research, user observation, brainstorming, scenario development, and prototyping over the course of four months. In this article, we explain our research process, some of the research themes, and the resulting design concepts.

Samsung gave AM+A a great amount of control over strategy and design. The entire user-experience was to be considered, from performance to pleasure, and the entire U.S. mobile market was our user-base to study. This freedom presented our design team with formidable challenges. We were to synthesize emerging technologies, but we knew from the outset that we wanted to avoid the trap of "technology in search of a problem."

The result from this project was the conceptual design for a "wireless information device" (WID) that we named "Mob-i." The "i" stands for both "information" and "identity." AM+A delivered to Samsung a "design concept catalog" of nearly one hundred different product directions and feature ideas. Some of these concept briefs are shown in the Design Concept examples. All of the designs shown are owned by Samsung with patents pending.

  Researching the Wireless Future, Distilling the Hope from the Hype

As difficult as it is to make small "baby-face" devices usable (i.e., devices with small displays), there is also the fundamental challenge of making them useful.

To achieve both goals, we found it critical to follow a user-centered design process.

We began by directing three of AM+A's designer/analysts to become "domain advocates" in three dimensions of research: cultural and market trends (particularly to learn from advanced product developments in Europe and Asia, which are the fastest growing markets), emerging mobile technologies, and advanced user-interface techniques. This approach allowed us to quickly acquire and maintain focused expertise. These three dimensions then informed the design through "checks and balances" of consumer desire, technical feasibility, and usability. A fourth dimension of research was direct observation of users (See Table 1).

In the beginning of the project even the device's main purposes was left to be defined. However, this approach provided a solid foundation for design decisions. We had to first identify the unmet needs and desires of users. From these we could derive needed functions, which could finally be designed into form. Each research dimension thus informed the design in a purposeful way:

  Observe Users To Discover Their Needs

To develop innovative products, one needs to identify the unmet needs and desires of users. We knew that focus groups would be unreliable, because users find it difficult to envision products or behaviors that they haven't encountered yet. We also knew that observation in the field would be necessary in order to fully understand people's mobile experience.

We targeted a wide variety of users for a contextual observation in order to witness as many situations and behaviors as possible. Desired subject types had diverging profiles, such as parent vs. teenager, technophile vs. technophobe, etc. Our eventual user set was made up of diverse people such as a minister who used many high-tech gadgets, a male college student, a female high-school student, a commuting professional, and a female, single-parent Silicon Valley entrepreneur.

Each user was shadowed for a day of his/her routine activities. The focus of the observation was "information use." Thus, we watched not only how the subjects used electronic resources like mobile phones and personal digital assistants, but also traditional resources like newspapers and sticky notes. We kept a running log in text and images, and we recorded unmet needs, inefficiencies, and information artifacts. Each user then continued a similar observation, recording activities in a self-log on paper or via tape recorder. Audio logging was particularly helpful in many situations in which paper logging is inconvenient (e.g., while walking) Finally, we brought back all users for a group interpretation session. The user-observation activities yielded many significant insights that later provided inspiration for our brainstorming workshops. Some examples are described below:

Figure 1 shows the personal organizer of a teenage girl. This artifact exhibits the remarkable amount of personalization that these physical personal information-managers undergo. The device is being used to organize intimate photos and memories in addition to raw information. The displays go beyond providing functionality to providing warmth.

Based on these observations, we developed concepts for an affective user interface that would make Mob-i feel more personal. (See Example 1: Affective Personality.)

Figure 2 provides evidence of a common problem. The subject is receiving information by phone, which he is manually transferring to a paper appointment book. This kind of use is very difficult on phone-personal digital assistant (PDA) combinations for which the device must be moved away from the ear for hand-and-eye activity. Observation of these basic needs was invaluable in grounding the design. (See Example 2: Detachable Ear-piece.)

  Wireless Device Usage Space: an Analytical Framework

Research on the project yielded many insights; the next task was to structure these insights to develop a system of solutions. Because wireless devices are now capable of many interrelated uses, making sense of such a complex design space is very challenging. We created an analytical framework to help organize the complexity of people's behavior and to form solutions that are systematic and not isolated. The framework in Figure 3 structures the many possible uses of wireless devices and proved helpful in scoping the product strategy and concepts.

Information. This space includes static reference information, such as weather reports, dictionaries, and entertainment listings. Organizational information includes all personal information management (PIM) functions. Business information supports specific professionals, such as a drug interaction reference for doctors.

Self-Enhancement. This space describes ways in which the device extends one's normal capabilities. For example, young people in Finland call cell-phones "Kanny" which implies "extension of the hand" [1]. Active Assistant functions include extending one's memory or reinforcing one's intentions. For instance, "FriendMinder," a project from Stanford Persuasive Technology Lab, encourages regular contact with loved ones [2]. Health and Safety describes the ability for mobile devices to enhance safety through contact with emergency response teams or family.

Relationships. Mobile devices will become especially successful where they extend existing social protocols. This space is founded around the core competence of phones to connect people. Products like Cybiko allow users to connect temporary communities of other Cybiko users [3]. For instance, they are capable of enhancing intimacy between two people who wish to be in more constant contact through "presence." (See Example 3: Relationship Room.)

Entertainment. This space includes enjoyment of portable media such as music, astrology readings, and jokes, or games involving multiple users and location-based elements. These contents become particularly desirable in captive situations or for short bursts of spontaneous stress-relief.

M-commerce. This space encapsulates a variety of uses including electronic money and e-coupons. The popularity of mobile devices will be catalyzed by viable commercial services. We designate this space as "they" because, unlike the other spaces, m-commerce involves services being provided from the business world to the individual.

  Wireless Devices: the First Truly "Personal" Computers

Identity. The five usage spaces mentioned above are centered around the identity of the user. Handheld devices are transported in the pockets or bags of their users. This makes them "nearly wearable" computers. These devices have a great potential to become knowledgable about their owners over time because of the "always-with-you, always-on" nature of handhelds. The "I" in Figure 3 represents the device's use as the seat of memory of the user.

The wireless information device should act as "cookies" do for Web surfing, providing persistent data and personalization. However, the WID will be able to do so over a broader spectrum of the user's life, because it will accompany the user wherever he/she goes. Memory of basic user information and more complex behavior patterns allows the device to reduce the amount of input needed. As a simple example of retaining basic user information, the problem of having to enter personal contact information more than once is exacerbated on a tiny device, but would be made unnecessary in many circumstances with a WID containing identity information.

As an example of more sophisticated capabilities, if the WID can be involved in the m-commerce transaction of renting a car, future car rentals can be shortened by remembering the user's vehicle preferences. Devices can ultimately implement "deep personalization" [4]. If the WID knows that you are scheduled to travel to Chicago next week, it can offer you the weather in Chicago, so that you do not have to browse for it. (See Example 4: My Yellow Pages.)

  Promise of Mobile Computing

As designers, we were excited about precisely exploiting the unique form factors of mobile hand-held devices. We first contrasted the inherent strengths and weaknesses of hand-helds in comparison to desktop and laptop computers (Table 2). Many of our designs then focused directly on utilizing the strengths and mitigating the weakenesses of hand-helds.

For instance, compared to laptop computers which are also shrinking in size, we noted that handheld computers are more portable to social settings because of their lower cost and greater durability. Overall, we concentrated our design on people-oriented tasks rather than data-oriented ones, like checking stocks or surfing the Web.

We also tried to focus Mob-i's purpose through segmenting the basic ways that people use information (See Table 3).

Mobile devices should excel at communication tasks; only the destination needs to be specified. They also may be made to perform adequately at retrieval tasks like email reading, Web surfing, and gathering tasks such as digital picture-taking. For instance, we emphasized information-gathering features because they present fewer input and output demands.

  Design Philosophies: The Information Appliance vs. the Swiss Army Knife

When designing products that hybridize many functions, two distinct philosophies come into conflict. The information-appliance philosophy stresses the value of devices that are consciously designed with a specialized application in mind. Conversely, the Swiss Army knife is often used as a metaphor for a philosophy that promotes designing a single device that is useful in many different situations.

This debate tends to polarize the question unnecessarily. We believe that the answer for which functions belong on a mobile device lies somewhere in between these two extremes. WIDs will be made valuable by providing access to high-value information and enabling mobility-based tasks. The challenge will be identifying what will be considered essential vs. what can wait until a user can access a desktop machine.

One technique we used to address this issue was to let our user subjects prioritize features themselves. We ran an exercise in which users listed each desirable feature on a card. Each user's cards were then taped together to form a long chain. This enabled everyone to visualize that the more functionality a device takes on, the larger and more complex it becomes.

Another way that we thought about the value-cost ratio of various features was through taking an inventory of the current mobile belongings of these subjects. We photographed and discussed the contents of their pockets, wallets, backpacks, and briefcases.

For example, one user exhibited a two-tiered bag system. A larger bag was used during the day along with more information tools (including textbooks, newspapers, various electronic devices, etc.). This user would switch to a smaller bag with essentials only for night-time mobility (cell phone and appointment book). In some situations even this smaller bag would be left behind, with only a wallet being needed. Even within bags, users showed calculated strategies for which devices (e.g., pens) would be most accessible in external pockets. The mobile device can be seen as a container of information and tools. Metaphorically, designers should decide whether they are intending to create a wallet, purse, backpack, or briefcase for their users.

  Input and Output Limitations

Hand-held devices are burdened by a "baby-face" display and tedious text input. Throughout the project, this "input and ouput issue" came up as a constant challenge. The tremendous constraints forced us to consider creative solutions, which are described below.

Contextual awareness. Mobile devices can use techniques to contextualize the user's input. By taking advantage of understanding the user's context, the device can automatically assist the user as much as possible. This saves the user's time and reduces the need for explicit input.

For instance, the phone can automatically default to dial a friend's work number during normal working hours in the friend's own time zone. On evenings and weekends the device can default to dial their home number. If the device can go further and indicate the "presence" of the person to be called (e.g. available, in-a-meeting, driving), then possibly a call won't even be necessary. (See Example 5: Activities During Calls.)

Location Awareness. Location awareness is enabled by the use of GPS and Bluetooth technologies. User-interface designers focusing on interaction will soon be able to utilize knowledge of a user's location as a way to aid usability. During observation, our users encountered many problems that could have been aided by location-based reminders. A helpful device can thus facilitate getting things done by reminding a user, at the appropriate time and place, when he/she can do a certain task. (See Example 6: Location-based Reminders.)

Time-shifting, Mob-i supports new modes of mobile work by affording a "start now, finish later" approach. For instance, users may simply mark each message as "Reply at Home," "Reply at Office," or "Print out at Office," etc. When they reach that location, they will be prompted to complete these partially finished tasks.

  Brainstorming and Prototyping

Following the research and strategy phase, we ran a series of structured brainstorming sessions. By analyzing these sessions and previous efforts, we provided Samsung with an illustrated catalog of conceptual product directions. We conveyed each concept through a descriptive scenario and illustrations.

Finally, we created an interactive prototype in Macromedia Flash to demonstrate certain scenarios selected by Samsung. This prototype focused on concepts that involved more complex interaction or animation. For example, the prototype shows what it is like to receive an e-coupon upon entering a store, or screen calls based on your location. We demonstrated solutions to challenging, but important situations, like creating appointments, using voice commands, and taking notes during a phone call.

  Conclusion

Because this project was open-ended and future-oriented, AM+A spent a greater-than-normal amount of project time in design research, product strategy, and user observation. The use of frameworks to organize and prioritize insights proved invaluable. The results of the project were varied, realistic, compelling, detailed depictions of coherent concepts for one or more products. Samsung engineers would determine, in the final analysis, which concepts to incorporate into specific manufactured products. Samsung's user-interface design group was able to observe and learn from a user-centered design team essential process components. Thus, Samsung felt that short-term and long-term goals were achieved.

  Acknowledgments

AM+A wishes to acknowledge the support of Ms. SunWha (Amy) Chung, Manager, User Interface Team, and Ms. Yoojin Hong, Designer, Software Laboratory, Corporate R+D Center, Samsung Electronics, Co., Ltd, Korea. All of the designs shown are owned by Samsung with patents pending. Among AM+A's staff, the authors wish to acknowledge the core team of Designer/Analysts Luke Ball and Junghwa Lee, who were essential to the project.

  References

1. Holstein, William J. Moving beyond the PC. U.S. News & World Report (December, 13, 1999). 50.

2. Fogg, B. J. et. al. http://hci.stanford.edu/captology. Stanford Persuasive Technology Lab.

3. Cybiko, Inc. www.cybiko.com.

4. Brand, Stewart. The Media Lab. (1987)

Abrams, M., C. Phanouriou, A.L.Batlongbacal, S. Williams, and J.E.Schuster. UIML: an appliance independent XML user interface language. Proceedings of the WWW8 Conference, May 11-14. p. 617-30. Toronto, Canada, 1999.

Bergman, Eric, Editor, Information Appliances and Beyond, Morgan Kaufmann, San Francisco, 2000.

D.Clark Preparing for a new generation of wireless data. IEEE Computer 32(8): 8-11.

A. Dillon, J.Richardson, and C.McKnight. The effect of display size and text splitting on reading lengthy text from the screen. Behavior and Information Technology, 9(3) 215-227. (1990).

R.L.Duchnicky and J.Kwahk. Readability of text scrolled on visual display terminals as a function of window size. Human Factors 25. 683-92.

S.H.Han and J.Kwahk. 1994. Design of a menu for small displays presenting a single item at a time. Proc. Human factors and Ergonomics Society 38th Annual Meeting. 360-364. (1994).

Jones, M., G. Marsden, N. Mohd-Nasir, K.Boone, and G. Buchanan. 1999. Improving Web interaction on small displays. Proceedings of the WWW8 Conference, May 11-14. 51-59. Toronto, Canada, 1999.

T.Kamba, S.Elson, T.harpold, T.Stamper, and N.Piyawadee. Using small screen space more efficiently. Proc. CHI'96, 383-390. (1996).

Marcus, Aaron. "Designing the User Interface for a Vehicle Navigation System: A Case Study," in Bergman, Eric, editor, Information Appliances and Beyond: Interaction Design for Consumer Products, Morgan Kaufmann, San Francisco, 2000, pp. 205-255.

Marcus, Aaron. Graphic Design for Electronic Documents and User Interfaces, Addison-Wesley, Reading MA, 1992.

Marcus, Aaron, "Chapter 19: Graphical User Interfaces," in Helander, M., Landauer, T.K., and P. Prabhu, P., Eds., Handbook of Human-Computer Interaction, Elsevier Science, B.V., The Hague, Netherlands, 1997, pp. 423-440.

J.F.Resiel and B. Shneiderman. Is bigger better? The effects of display size on program reading. In G.Salvendy (Ed.) Social, Ergonomic and Stress Aspects of Work with Computers. Elsevier, Amsterdam. 113-122. (1987).

S.J.Swierenga. Menuing and scrolling as alternative information access techniques for computer systems; interfacing with the user. Proc. Human Factors Society 34th Annual Meeting. 356-359. (1990).

Watters and Shepherd. Adaptive Medical Portals. ACM SIGIR Customized Information Delivery Workshop Proceedings. San Franscisco. August 19, 1999.

C. Watters and D. Comeau. Adaptive Portals with Wireless Components. Conference Proceedings, 1999.

Advanced Traveler Information Systems Committee...: www.sae.org/technicalcommittees/atishome.htm

American National Standards Institute: www.ansi.org/

AnyWhereYouGo: www.AnyWhereYouGo.com/
News and discussion on both technological and business trends in the wireless industry.

AngelBeat: www.angelbeat.com/
See Untethered: trends and definitions for wireless Web and mobile Internet by this solutions provider.

BlackBerry: www.blackberry.net/

Diversified Software Industries, Inc.: www.dsi-inc.net/

Handspring: makers of Visor: www.handspring.com/
Palm plus peripherals!

i-Appliance Association: www.i-appliance.org/

International Standards Organization homepage: www.iso.ch/

Kyocera Wireless Products: www.kyocera-wireless.com/

Nokia USA: www.nokia.com/main.html

OnHand Watch PC: www.onhandpc.com/
World's smallest computer, on the wrist. Interactive demo.

Palm accessories: www.palmgear.com/
One of the best sites to support Palm users. [NYT 0299]

Palm Design Philosophy: http://news.cnet.com/news/0-1006-200-921068.html?tag=st.ne.1002.bgif?s

Samsung Electronics Products: mobile division: http://samsungelectronics.com/mobile_phone/mobile_phone_index.asp

The Standard's Wireless News: http://thestandard.com/wireless

Symbian: www.symbian.com/
World-wide wireless platform

Unstrung: www.unstrung.com/
Industry-specific news and views

Voice: IBM Voice Systems: www-4.ibm.com/software/speech
Info about voice and speech recognition. [Usability Interface, July 2000, p 13, Newsletter of STC Usability SIG]

Voice: Lernout and Haupsie: www.lhsl.com/
Info about voice and speech recognition. [Usability Interface, July 2000, p 13, Newsletter of STC Usability SIG]

Voice: Philips speech processing: www.speech.be.philips.com/
Info about voice and speech recognition. [Usability Interface, July 2000, p 13, Newsletter of STC Usability SIG]

WAP Forum: www.wapforum.org/
Information on the "wireless web" protocol

WSSN - World Standards Services Network: www.wssn.net/WSSN/

ZDNet: Handhelds Guide: www.zdnet.com/pcmag/filters/guide/0,10172,6001651,00.html

  Authors

Aaron Marcus
Aaron Marcus, President of Aaron Marcus and Associates, Inc. (AM+A), received a BA in physics from Princeton (1965) and a BFA and MFA in graphic design from Yale Art School (1968). He is an internationally recognized authority on designing user interfaces, multimedia, documents, and online services. He co-authored Human Factors and Typography for More Readable Programs (1990) and The Cross-GUI Handbook for Multiplatform User Interface Design (1994), and he authored Graphic Design for Electronic Documents and User Interfaces (1992), all published by Addison-Wesley. Mr. Marcus was the world's first professional graphic designer to work in computer graphics (1967), to program a desktop publishing system (for the AT&T Picturephone, 1969-71), to design virtual realities (1971-73), to establish a computer-based graphic design firm (1982), and to receive the NCGA Industry Achievement Award for his contributions to computer graphics (1992). As President of AM+A, with offices in Emeryville, CA, and New York City, NY, he and his staff work with Fortune 500 companies and start-ups as planners, consultants, designers, and programmers. Mr. Marcus programmed and designed his first user interface in 1969 and wrote his first user-interface design guidelines document in 1981. In the last 20 years, he and his staff have designed or helped to design at least 300 user interfaces. Mr. Marcus has given userinterface design tutorials at CHI since 1990 and at SIGGRAPH since 1980. At CHI-98 he organized the first baby-face panel. AM+A has consulted on the Nokia Communicator 9000 and worked with Cisco/InfoGear on its Web-phone products. In 2000, Mr. Marcus published a case study of the user interface AM+A designed for Motorola's vehicle-navigation system, in Bergman, Information Appliances and Beyond, and his firm designed concepts for advanced mobile phones with PDA characteristics for Samsung, Korea, detailed in this article.

Eugene Chen
Eugene Chen, Director of Design/Analysis and of the California office of Aaron Marcus and Associates, Inc. (AM+A), received his BS degree in Computer Science and Engineering with a minor in Design of the Environment from the University of Pennsylvania in 1987. He directs the practice of user-interface design and analysis at AM+A. His favorite design experience is researching user behavior and developing new methodologies. At AM+A, he has led dozens of successful projects such as the re-design of The Getty Museum Web site for the Getty Trust, and creation of future smartphone concepts for Samsung Electronics.
He brings an integrated sense of design and usability to AM+A's work, applied on projects for clients such as Cisco, AC Nielson, Gigabeat.com, the U.S. Treasury Department, and Visa. His research and lecture interests include mobile device design, social interfaces, and Web applications. He has spoken on these topics for Web 2001, BayCHI, Forrester's Research, and ID Magazine.
Before joining AM+A, Mr. Chen helped launch many software products and was involved in development activies from product strategy to software engineering. Other products he has developed include Key Curriculum Press's Fathom, an educational environment for data exploration and visualization, Buckminster Fuller's World Game™ interactive atlas, G-Vox™ MIDI based guitar lessons, and Picture Network International—the Internet's first online stock photography source. He has also worked as a professional composer and sound designer, creating audio branding, original music and sound installations for such clients as Eidos and Levi Strauss.

  Figures

Figure 1. Personal organizer of a teenage girl. Note the degree of ornamentation and personalization.

Figure 2. Observation subject using phone and appointment book simultaneously.

Figure 3. Usage Spaces of Wireless Devices

  Tables

Table 1. Research areas and output

Table 2. Strengths and weaknesses of hand-held computing

Table 3. Taxonomy of basic information usage

  Sidebar: Design Concepts

Example 1. Affective Personality

Mob-i has an engaging and expressive personality, shown through an expressive animated character. This character resides on the title bar of every screen, lets the user know what is happening on the device in a constant, noticeable, but non-obtrusive manner. Indicators include signal strength, memory, battery life, reminders, etc. Moreover, Mob-i's personality is intended to enable users to feel more attachment to their personal pocket assistants. Users can download new personalities, which load new images, phrases, and speech voices into the device.

Example 2: Detachable Ear-piece

Mob-i's main body communicates to a wireless headphone-microphone via Bluetooth. This soft ear-bud, which pops out of the back of the device, can be worn comfortably in the ear like a piece of jewelry. Separating the ear attachment allows the user to perform visual tasks, such as taking notes during calls— without using a speaker phone which compromises privacy, and without the inconvenience of trailing wires. Wearing the ear-bud regularly creates a more discrete experience; users are notified of calls and reminders privately and silently by voice output from the device. Text reminders are read through text-to-speech and voice recordings are played back.

Example 3: Relationship Room

Each person in your address book has an easy-to-enter Relationship Room. This room is a one-to-one chat, scribble, and photo area. You can exchange personal messages with this person in real-time, or leave him/her love notes for later. The rooms are kept over time and eventually form a mutual scrapbook of memories and/or a data/transaction archive.

Example 4: My Yellow Pages

Jack is getting some repairs done by a carpenter. He will need to call back in a few days to see if the work is done and then will need to see the map again to remember how to get there. Mob-i maintains a history of regular businesses that Jack uses. Jack saves businesses to the "My Yellow Pages" list (e.g., the veterinarian for the pet), separate from his personal address book.

Example 5: Activities During Calls

The What-can-I-do button allows the user to perform common tasks during a phone call. For instance, if the user is talking to Jack, the menu is adapted to include shortcuts for making an appointment with Jack and looking up Jack's address.

Example 6: Location-based Reminders

Because Mob-i knows where Jack is, Mob-i can help Jack to remember to do something when he is at a good place to do it. This capability allows Jack to separate his to-do's for work, home, and out-running-errands. Jack used to leave himself messages on his home answering machine, but now he leaves messages for himself on Mob-i and Mob-i reminds him of these things. Examples include the following:

• Remind me to take out the garbage when I get home on Thursdays.
• Check oil next time I get in the car. Pick up jumper cables at the mall.
• Get my CDs back when I'm at Jill's house.
• Take home insurance forms when I leave the office.
• Talk to Bill as soon as he gets back in the office, using Bill's phone-identifier.
• Your spouse wants you to stop by the bank on the way home, so you are reminded when you get in the car.

 

©2002 ACM  1072-5220/02/0100  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2002 ACM, Inc.