Week 6: Test Results of the Think Aloud Method

A picture and description of the Think Aloud user test setup can be found in the previously blog post. The user tests took in average around ten minutes and took place in a seminar room of the TU Berlin. As recording technique we choose a piece of paper to document notes such as the behaviours of the test person.

The tasks we performed in our user test can be described as following:
1. Find a predefined picture out of the gallery
The following three tasks are designed to test the intuitional understanding of the navigation. We put alot of pictures into the gallery, more then the user can see on the screen. Thereby the test user had to use the navigation to find the requested picture.
2. Zoom into a predefined picture
In this tasks the user had to go forward to zoom into a picture
3. Select the picture right to the predefined picture
This task is designed to test, if the user is adapting the navigation of the slide-gallery (going left-/rightwards to) also to the fixed modus.
4. Select the picture draw modus
If the user is in the fixed mode, a pop up appears and the user can switch into the drawing mode by putting his hand into the pop up.
5. Draw a smiley
This task measures if the user is enjoying the drawing modus.

User 1: Male, 25 years old, computer science student, high experience with natural user interfaces, no experience with public display applications.
1. Find a predefined picture out the gallery:
It took a while to find the requested picture, because in the beginning the navigation was not clear to him. But without any hint he found out how to slide the gallery.
2. Zoom into a predefined picture
While doing the first task he found out how to zoom into a picture. So this task was pretty easy to him. But it looks that the navigation is quite clear to him after a short understanding phase.
3. Select the picture right to the predefined picture
In theory it was easy for him to select the right picture, but due to the fact the navigation accuracy has still improvement potential it took a little time.
4. Select the picture draw modus
The selecting of the drawing modus took a time, because he left the fixed mode, which is necessary to join the drawing mode. After a hint to go into the fixed/zoomed mode it also took a little time to understand how to join the drawing mode. Also it was not easy to him to hit the pop up, because it appears on a hard to reach area.
5.Draw a smiley
Drawing a smiley was not easy for him, because first he was not able to understand were he currently is painting (the position of the brush / his hand).

User 2: Female, 23 years old, bwl student, medium experience with natural user interfaces, no experience with public display applications.
1. Find a predefined picture out the gallery:
It took some time to find out how to handle the navigation. But after some time she find out how to handle the navigation and the task was successfully finished.
2. Zoom into a predefined picture
Well done after found out how to handle the navigation (task 1).
3. Select the picture right to the predefined picture
Worked
4. Select the picture draw modus
Could not join the drawing modus without hint.
5.Draw a smiley
Drawing a smile worked good for her, but she said it would be nice to have some more features like having new colors.

Major issues and solutions:
Huge learn phase -> Having a help function
Navigation accuracy -> More work on this task
Drawing Mode, where do i have to draw? -> Drawing hand
Drawing Mode, more colors.

Task 6: Test Setup

The test environment consists of a wide screen monitor with a kinect on top. The test subject will be:

• Find a predefined picture out of the gallery
• Zoom into a predefined picture
• Select the picture right to the predefined picture
• Select the picture draw modus
• Draw a smiley :)

Task 5: Heuristic Prototype Evaluation

The following blog post will show our heuristic evaluation result of our first software prototype. As the basic set of the evaluation we used the ten usability heuristics by Jakob Nielsen. Due to the fact the project time is very limited we focused on the gallery part for creating the heuristic evaluation.

Error prevention:

In the gallery mode we had to implement some error preventing arrangements, because the pictures stop sliding right- or backwards if the user is close to the kinect. We implement this feature in the previously prototype to provide the ability to watch a picture in a fix mode. But the problem was that it is not so easy for the user to control the gallery very accurate, which leads to the inconvenience that the user was in the fixed mode between two pictures.

PIC

To solve this error state, we improved the fixed modus. The application now detects the picture which fits best for showing in the fixed mode. This is the closed picture to the center of the public display screen. After detecting it, it slides to the horizontal center, so that the user got a perfect view on the selected picture.

PIC

We also implemented an error prevention to have a minimum size of the pictures in the gallery. In the old prototype the problem occurs that the user can be very far away from the kinect, which leads to an state that the pictures were very small.

Recognition rather than recall:

In our previous prototype it was not possible to switch between pictures if you are in the fixed gallery modus. The user had to go backward for leaving the fixed mode, go right- or leftwards to switch to another picture and go forward to get again in the fixed mode. We thought about different kind of interactions to switch pictures in the fixed mode, like a special gesture by hand. But we solved the problem by using the go right- or leftward interaction, which we also used to start the gallery slide. The reason for that is, that user already knows this kind of interaction. The issue that user usually do not have any experience with an interactive public display system leads to the point that the user always need a lot of time to understand how to control/interact with the system, which affected the usability because people have to spend a lot of time to understand how to interact. To avoid this effect, it is good to use consistently interaction techniques.

Severity Ratings for Usability Problems:

• The severity of a usability problem is a combination of three factors (Jakob Nielsen):
• The frequency with which the problem occurs: Is it common or rare?
• The impact of the problem if it occurs: Will it be easy or difficult for the users to overcome?

The persistence of the problem: Is it a one-time problem that users can overcome once they know about it or will users repeatedly be bothered by the problem?

Both problems which are described previously occur very often because they are part of the main navigation through the gallery. So it got a four on the frequency rating scale, which can be found here, and needed to be fixed quickly.

The impacts of both problems are huge, because they impact the navigation through the gallery. The user can overcome these problems by performing special interactions, like going backward; focus on a new picture and go forward to select a new picture. But these overcomes are not really user friendly and lead to a huge loss of usability. Due to these facts the problem got a three on the impact scale.

The persistence of the problems is also high and got a four on the persistence scale, because the user will be repeatedly be bothered by both problems.

Task 4: Gallery modus requirement update

Summarization of the gallery modus:

• The public display shows a gallery of pictures as a slide show
• If the user turns left/right, the slide show starts the move in the according direction
• If the user moves for-/backward, the pictures of the slide show are getting larger/smaller (zoom)
• If the user is very close to the public display, the slide effect stops and the pictures are in an idle modus

Requirement update:

We recognize in the lecture room that a solution for multiple users is necessary, because the room was full with people and the system was out of control if multiple users move in front of the kinect. Therefore we want to set the closed user into focus and provide an UI feature that shows the actual user.

Another requirement is the need of changing the picture if it is fixed (happens if the user is very close to a picture). Therefore we thought about a menu which pops ups if the user is in the “fix picture mode” or specific gesture.

Also the user should have to the possibility to change the velocity of the slide effect of the gallery, which is not yet implemented because the first prototype was more a kind of a horizontal prototype.

Task 4: Software prototype of the gallery

In the following three pictures you can see some pictures of our first prototype of the gallery function.

The three pictures show the gallery modus from a different range. The pictures of the gallery are represented as rectangles in the prototype. If you move forward you can zoom into the picture gallery and the pictures of the gallery are getting larger. If you move backward you can zoom out and the pictures will get smaller. By walking left- or rightwards the gallery starts to slide to the according direction. Is the distance between the picture and user very small, the slide of the image stops and the user can watch the specific image in an idle mode.

While developing the prototype and presenting it in the mobile interaction lesson, we specify some additional and modify some requirements to our design and functionalities. These changes will be discussed in more detail in the upcoming blog post.

Task 3: Summary of the observations and user feedback

In the following we will summarize the user feedback and observations of our first paper prototype we made (see previously blog post).

The user feedback of our first basic prototype showed up, that the user interface is not as intuitive as we expected first. Especially the painting mode needs some adjustments. So it was not intuitive understandable how to start and setup the drawing settings, like changing colors and brush size. Therefore we need to focus even more on help functions for a clear understandable user interface. An additional reason for that is, that public displays systems are totally new to users and they have no clue how to interact with them, because they don’t have any experience with them.

Another difficulty while working with the prototype occurred. For example it was not possible to simulate our gallery and computer aided interaction, because they don’t come with classical user interface elements that are visible to the user.

For the next Phase of the project, we focus on the gallery feature, because we think it is very interesting to have a gallery navigation based on user movement and not like the conventional way. Also we think that this is a part of the project with a lot of potential, were we can gather a lot user feedback to continue the project successfully.

Have a nice day!

Task 3: Paper Prototype

The following pictures show our first first paper prototype we used to gather some user feedback.

Computer aided / driven-by art interaction:

Draw Modus:

Gallery Modus:

The results of our paper prototype user studies can be found in the upcoming blog posts.

Improved project description

Due to the fact that we think our project description is not quite clear to everybody, we decided to describe it in more detail. Therefore we use the help of a more detailed story board and some additional comments. These adjustments also contain the user feedback we gather from the interviews of task 2.

Drive-by computer aided Interaction:

• The user walks in front of the public display
• The system detects him and draws some illustrations based on his movement
• If the user stands in front of the display, the accuracy gets better and user can also use his hands to draw illustrations
• 

Gallery mode:

• The gallery interacts with the movement of the user
• If the user walks to the left or right, the pictures starts moving parallel to him
• If the user moves for- or backwards, the picture will get bigger or smaller
• 

Painting mode:

• Classic painting mode
• See upcoming paper prototype on the next blog post

To change between the different modes, the user has to clap his hand. Therefore we need a help sign next to the public display or a window on the public display, which explains how to handle the system. Additionally there is a fixed timer that switches between the different modes.

Task 2: Design Principles

The following blog post contains the design principle that reflects essential design aspects of our application.

Like in the previous blog post mentioned: The user must always feel comfortable while interaction with the system. This effects indirect computer aided and direct art creation. For direct creation it means, that the user is willing to draw in public places in front of other peoples. And for indirect creation it is important not to affront the user with inappropriate computer aided art. Furthermore it is importent to priovide the user a complete and constant help functionality to offer him a comfortable feeling while using the application.

Task 2: Interview findings & Design modification

The following post is a summary of our main findings based on the results of our previously performed user interviews.

Although the amount of interviewed users is relatively small which a number of three, we found out that the interested for digital art is greater than previously expected.

Also we conclude that most of the users are interested in art, even if they don’t go often to a gallery. There are different reasons for it, like they don’t want to pay a fee or are not interested in classic exhibitions. But they like art in their everyday environment. Therefore we purpose that is a good idea to move the art out of the gallery into public places.

Our assumption that average people don’t like to create art in public, in front of other people, was approved by a small user test while doing the interview. Two of three interview partners were not willing to paint a small sketch in front of us. So we think it is important to focus on computer aided art creation and interaction.

Due to the fact that in some public places, like bus stations, the user got limited time to draw and the fact that continuing other paintings is a nice feature, which we found out of the interview, we conclude that this is also a must have feature for our project.

All in all we found out that most of the users were interested in this topic, but we have to modify our design to satisfy our users as good as possible. The design modifications can be summarized as following:

• It looks like the users would like the idea of moving the gallery to public places to beautify their everyday environment.
• Some users were averse of painting in public places, so it is important to find a good solution to give the user a comfortable feeling while painting. It is necessary to provide this feeling to make a system, which the user likes to use.
• Another modification based on the averse of paining in public places is to focus more on computer aided art creation and interaction, were the user creates art in a new and for him, interesting way.
• The user enjoys interact also with other users. This interaction can be simultaneously like painting together, but can also be consecutively (on after another).
• Due to alot of questions about how a system like that could work and the fact that these kind of systems are completly new to the users, it is necceary to provide an excelent and complete help function to the user.

Task 2: Interviewed users

The following blog post contains a short description of our interviewed users and gives an overview about the results of the interview. Based on this information we will provide our interview findings in the next blog post in more details.

User 1 Amélie:

Amélie is 24 years old, lives in Berlin and is an economic student with focus on marketing. She describes herself as a casual and sport interested person. In her free time she likes to play basketball, hanging out with friends, cooking and loves snowboarding. Because of her semesterticket and the fact that she has a lot of friends around Germany she travels a lot with public transports like bus and train. Therefore she already know what public displays are in generally and also likes the idea of it because of his course of studies (marketing). But right now she sees public displays more as an advertisement and not as an interactive place.

She is not really interested in classic art, but enjoys street art and photography. Galleries are not unknown to her and sometimes she is visiting some with her friends. But often she annoyed of high entrée fees and the artists. “I prefer street art, because the artists are not trying to make money with it and it is art from casual people with no academic background, so they don’t follow any rules or standards”, she says.

Her last “real” picture she draws in school. But she enjoys doodling while telephoning or when she is bored. She doesn’t through her doodling directly away after painting and is not angry if other persons see her sketches. Replay to the question, if she likes the idea of continues painting of other persons, she said yes and told us some stories of her WG, where she and her flat mates draw some funny things on their shared todo and shopping list.

But she doesn’t like the idea of digital art (art that is created with the help of computers) in generally, because for her it is not authentic and she can’t feel the colors and impressions on a digital display. She sees digital art as a creative work, but not as an art itself because a picture must be real handcrafted without any computer impact.

User 2 Doris:

Doris is a 56 years old, got two kids, housewife and an early pensioner. Before she got retired and got her second son, she worked as a language interpreter in Spain. Because of that she has one flat in Germany and Spain. She describes herself as an open-minded and curious person, that like to travel and enjoy culture events. Because of her residence in Spain and her family in Germany she travels a lot by airplane and train. Therefore she also knows what public displays are, after explaining it a little bit more detailed. But she thinks that public displays are very not very useful and cannot see any benefit of them. She often feels disturbed and overcharged with it.

She is really interested in art, especially in classical art (post-impressionism), and goes several times a year to an exhibition. Often she goes alone to a gallery, but would like to go with other persons to it to share thoughts and impressions and just discuss about paintings. As a replay to the question if she likes art that is shown outside of a gallery, she answered yes, because she loves to live in a colorful and dynamic environment.

Doris owns an iphone and knows how to handle a computer. But she has no clue what digital artwork could be. Because of that reason we showed her some desktop wallpapers, but it looks that she was not really interested in it, because she started to talk about his son, who is a good drawer and should make more out of this talent. So we asked her if he inherent this talent from her, but she said no, because she never really painted a picture. Our last question was if she can doodle a small picture for us, to see if she would draw something while other people can see it. But she said no.

User 3 Peter:

Peter is 25 years old, a computer science at the TU Berlin and describes himself as technical guru. Because we know him very well, we know that his hobbies are computer games and technical stuff. He travels everyday with public transports to the university. Peter likes the idea of public displays because he would like to live in the future, but “art is not very cool” he says. His last gallery visit was long time ago (school) and he would never go to a gallery on his own.

But digital art is not unknown to him and we would say that he is interested in it, because he said he changes his wallpaper twice a month and loves high end computer graphic games.

We asked him to draw a picture for us on a piece of paper, to check if he would draw pictures in front of other people. But he just laugh and said:” I hated the art class in school and I would never draw something in front of you” J. Never the less he is interesting in the further development of the project, because he is interested in public displays and a good friend of us. How knows, maybe he will be interested in public displays after the first release.

The notes of our interviews can be found here.

Storyboard

Interview Questions

• How old are you?
• Whats your current work activity?
• Please talk about your interests.
• Do you like Art?
• How often do you vistit art galleries?
• Have you seen any "Digital" Artwork?
• Are you artistically active?
• How do you spend your free time?
• Are you painting?
• Do you know any painter/artists ?
• Would you like to expirience a new interactive way of painting?

User Profile Persona

Chris Müller is a painter living in an art oriented place located in the Berlin distric Friedrichshain. He's unmarried and intends to leave a sign of his work in the world. His Nightmare is to loose his creations. He spends most of his free time visiting art galeries and is also very interested in digital artworks. Most people of his circle of acquaintances are also artists or involved in artistic projects.