Week 6: Test Results of the Think Aloud Method

A picture and description of the Think Aloud user test setup can be found in the previously blog post. The user tests took in average around ten minutes and took place in a seminar room of the TU Berlin. As recording technique we choose a piece of paper to document notes such as the behaviours of the test person.

The tasks we performed in our user test can be described as following:
1. Find a predefined picture out of the gallery
The following three tasks are designed to test the intuitional understanding of the navigation. We put alot of pictures into the gallery, more then the user can see on the screen. Thereby the test user had to use the navigation to find the requested picture.
2. Zoom into a predefined picture
In this tasks the user had to go forward to zoom into a picture
3. Select the picture right to the predefined picture
This task is designed to test, if the user is adapting the navigation of the slide-gallery (going left-/rightwards to) also to the fixed modus.
4. Select the picture draw modus
If the user is in the fixed mode, a pop up appears and the user can switch into the drawing mode by putting his hand into the pop up.
5. Draw a smiley
This task measures if the user is enjoying the drawing modus.

User 1: Male, 25 years old, computer science student, high experience with natural user interfaces, no experience with public display applications.
1. Find a predefined picture out the gallery:
It took a while to find the requested picture, because in the beginning the navigation was not clear to him. But without any hint he found out how to slide the gallery.
2. Zoom into a predefined picture
While doing the first task he found out how to zoom into a picture. So this task was pretty easy to him. But it looks that the navigation is quite clear to him after a short understanding phase.
3. Select the picture right to the predefined picture
In theory it was easy for him to select the right picture, but due to the fact the navigation accuracy has still improvement potential it took a little time.
4. Select the picture draw modus
The selecting of the drawing modus took a time, because he left the fixed mode, which is necessary to join the drawing mode. After a hint to go into the fixed/zoomed mode it also took a little time to understand how to join the drawing mode. Also it was not easy to him to hit the pop up, because it appears on a hard to reach area.
5.Draw a smiley
Drawing a smiley was not easy for him, because first he was not able to understand were he currently is painting (the position of the brush / his hand).

User 2: Female, 23 years old, bwl student, medium experience with natural user interfaces, no experience with public display applications.
1. Find a predefined picture out the gallery:
It took some time to find out how to handle the navigation. But after some time she find out how to handle the navigation and the task was successfully finished.
2. Zoom into a predefined picture
Well done after found out how to handle the navigation (task 1).
3. Select the picture right to the predefined picture
Worked
4. Select the picture draw modus
Could not join the drawing modus without hint.
5.Draw a smiley
Drawing a smile worked good for her, but she said it would be nice to have some more features like having new colors.

Major issues and solutions:
Huge learn phase -> Having a help function
Navigation accuracy -> More work on this task
Drawing Mode, where do i have to draw? -> Drawing hand
Drawing Mode, more colors.

Task 6: Test Setup

The test environment consists of a wide screen monitor with a kinect on top. The test subject will be:

• Find a predefined picture out of the gallery
• Zoom into a predefined picture
• Select the picture right to the predefined picture
• Select the picture draw modus
• Draw a smiley :)

Task 5: Heuristic Prototype Evaluation

The following blog post will show our heuristic evaluation result of our first software prototype. As the basic set of the evaluation we used the ten usability heuristics by Jakob Nielsen. Due to the fact the project time is very limited we focused on the gallery part for creating the heuristic evaluation.

Error prevention:

In the gallery mode we had to implement some error preventing arrangements, because the pictures stop sliding right- or backwards if the user is close to the kinect. We implement this feature in the previously prototype to provide the ability to watch a picture in a fix mode. But the problem was that it is not so easy for the user to control the gallery very accurate, which leads to the inconvenience that the user was in the fixed mode between two pictures.

PIC

To solve this error state, we improved the fixed modus. The application now detects the picture which fits best for showing in the fixed mode. This is the closed picture to the center of the public display screen. After detecting it, it slides to the horizontal center, so that the user got a perfect view on the selected picture.

PIC

We also implemented an error prevention to have a minimum size of the pictures in the gallery. In the old prototype the problem occurs that the user can be very far away from the kinect, which leads to an state that the pictures were very small.

Recognition rather than recall:

In our previous prototype it was not possible to switch between pictures if you are in the fixed gallery modus. The user had to go backward for leaving the fixed mode, go right- or leftwards to switch to another picture and go forward to get again in the fixed mode. We thought about different kind of interactions to switch pictures in the fixed mode, like a special gesture by hand. But we solved the problem by using the go right- or leftward interaction, which we also used to start the gallery slide. The reason for that is, that user already knows this kind of interaction. The issue that user usually do not have any experience with an interactive public display system leads to the point that the user always need a lot of time to understand how to control/interact with the system, which affected the usability because people have to spend a lot of time to understand how to interact. To avoid this effect, it is good to use consistently interaction techniques.

Severity Ratings for Usability Problems:

• The severity of a usability problem is a combination of three factors (Jakob Nielsen):
• The frequency with which the problem occurs: Is it common or rare?
• The impact of the problem if it occurs: Will it be easy or difficult for the users to overcome?

The persistence of the problem: Is it a one-time problem that users can overcome once they know about it or will users repeatedly be bothered by the problem?

Both problems which are described previously occur very often because they are part of the main navigation through the gallery. So it got a four on the frequency rating scale, which can be found here, and needed to be fixed quickly.

The impacts of both problems are huge, because they impact the navigation through the gallery. The user can overcome these problems by performing special interactions, like going backward; focus on a new picture and go forward to select a new picture. But these overcomes are not really user friendly and lead to a huge loss of usability. Due to these facts the problem got a three on the impact scale.

The persistence of the problems is also high and got a four on the persistence scale, because the user will be repeatedly be bothered by both problems.

Task 4: Gallery modus requirement update

Summarization of the gallery modus:

• The public display shows a gallery of pictures as a slide show
• If the user turns left/right, the slide show starts the move in the according direction
• If the user moves for-/backward, the pictures of the slide show are getting larger/smaller (zoom)
• If the user is very close to the public display, the slide effect stops and the pictures are in an idle modus

Requirement update:

We recognize in the lecture room that a solution for multiple users is necessary, because the room was full with people and the system was out of control if multiple users move in front of the kinect. Therefore we want to set the closed user into focus and provide an UI feature that shows the actual user.

Another requirement is the need of changing the picture if it is fixed (happens if the user is very close to a picture). Therefore we thought about a menu which pops ups if the user is in the “fix picture mode” or specific gesture.

Also the user should have to the possibility to change the velocity of the slide effect of the gallery, which is not yet implemented because the first prototype was more a kind of a horizontal prototype.

Task 4: Software prototype of the gallery

In the following three pictures you can see some pictures of our first prototype of the gallery function.

The three pictures show the gallery modus from a different range. The pictures of the gallery are represented as rectangles in the prototype. If you move forward you can zoom into the picture gallery and the pictures of the gallery are getting larger. If you move backward you can zoom out and the pictures will get smaller. By walking left- or rightwards the gallery starts to slide to the according direction. Is the distance between the picture and user very small, the slide of the image stops and the user can watch the specific image in an idle mode.

While developing the prototype and presenting it in the mobile interaction lesson, we specify some additional and modify some requirements to our design and functionalities. These changes will be discussed in more detail in the upcoming blog post.

Task 3: Summary of the observations and user feedback

In the following we will summarize the user feedback and observations of our first paper prototype we made (see previously blog post).

The user feedback of our first basic prototype showed up, that the user interface is not as intuitive as we expected first. Especially the painting mode needs some adjustments. So it was not intuitive understandable how to start and setup the drawing settings, like changing colors and brush size. Therefore we need to focus even more on help functions for a clear understandable user interface. An additional reason for that is, that public displays systems are totally new to users and they have no clue how to interact with them, because they don’t have any experience with them.

Another difficulty while working with the prototype occurred. For example it was not possible to simulate our gallery and computer aided interaction, because they don’t come with classical user interface elements that are visible to the user.

For the next Phase of the project, we focus on the gallery feature, because we think it is very interesting to have a gallery navigation based on user movement and not like the conventional way. Also we think that this is a part of the project with a lot of potential, were we can gather a lot user feedback to continue the project successfully.

Have a nice day!

Task 3: Paper Prototype

The following pictures show our first first paper prototype we used to gather some user feedback.

Computer aided / driven-by art interaction:

Draw Modus:

Gallery Modus:

The results of our paper prototype user studies can be found in the upcoming blog posts.