top of page

System Analysis

Performing a system analysis helps to identify pain points, and also helps the designer see what is working which can inform future design iterations. 

WOW AIRLINES

What is WOW? Wow airlines is a recently formed Icelandic airline that centers its business on cheap flights, tours, and travel information. Their website, Wowair.us, is, overall, a beautifully designed, clear, and concise website, but could stand to make a few changes. 

Shortcuts are a usability aspect that is evident in the design of the WOW airlines site design.

THE GOOD: Locus control is evident from the landing page. The user has the immediate ability to input flights times and dates through the flight search function. There is very little in the way of ads, and the experience is user-driven. The available actions laid out on the menu are easy to navigate because of font choices and layout. This makes the site easy to relearn the navigation. The writing style is clear which enables the user to figure out where to click for what purpose. The hierarchy is made uncomplicated through color and positioning of objects so the user is quickly able to find and perform tasks. The user is able to move through the booking system easily, in part due to the way the system offers informative feedback, the buttons/fields for dates and destination are clearly labeled from screen to screen. The use of a highly visible booking system also presents a shortcut for the user. They are able to perform the most basic task and move on quickly.

The change in the menu layout on one of the pages decreases the users learnability.

THE NOT-SO-GOOD: There are many aspects that could be improved on this system. The most glaringly obvious change is the difference between the main pages and the Tours page. The layout, fonts and logo position of the menu changes on the Tours page. This makes the Tours page inconsistent with the other pages, which affects both learnability and memorability for the user. In addition, being an Icelandic site, there are aspects that have not been addressed for the American user (establishing requirements). For example, some of the prices on child pages are in euros. Also it is evident that a separate US site was not built out as the main logo links back to the UK version of the site. This is jarring for the American user as not only are prices listed in pounds, the input of dates into the calendar is also day, month, year. The two sites look so similar that a user could conceivably move through the UK version  and only realize it after they try to input their flight specifications. On a more basic graphical level, there are slight issues. The “flights” section has a photo that overlays the input form, affecting the users ability to complete a task quickly. The color purple used is also not consistent within the same section and throughout the site. While that probably would not bother the average user, it would help if the elements were graphically united, so that the aesthetics helped to give a consistent feel to the user. 

 

IN CONCLUSION:In general, there are many things this site has going for it, but many things that could be improved upon. On the positive side, the locus control, hierarchy, writing-style, and navigation all are positive aspects of this site. There is room for improvement, though, especially in consistency of one page, which damages a users learnability and memorability. The different versions of the site (UK vs. US) could also frustrate a user quite quickly. On a smaller level, there are some issues with legibility and color consistency. 

MY ROLE

USER TESTING

Below I am using three different types of testing. Silent observer, think-aloud, and constructive interaction methods.

User Testing

SCENARIO: The reviews for Healthcare.gov have been mixed. The site launched on October 4th, 2013, and was met with mixed reviews. The goal of the website was created with two goals in mind: “[to] create  a marketplace with an array of choices and competitive prices. The other is whether it explains insurance so people understand it.[1]”. In addition to the reason mentioned above the site is meant to be a federal hub, where both individuals and businesses will be redirected to their state-specific website to try and get coverage. There has been a lot of press on how the system is difficult to use, and as a reaction the site is slated to receive a major upgrade in the next month[2]. The audience for use of this system is a wide age range, but in general toward the adult user, both male and female. As many found out this last tax cycle, the penalty for being uninsured is now 2% of ones income.The affordable Care Act makes the use of this site imperative for many people. Open enrollment begins November 1st,  with such a huge amount of impending users our goal is to evaluate healthcare.gov and see how it can be improved.

USER TEST:Three users were selected, two female and one male. Two of the subjects were in the 20-30 age range while the third subject was in her sixties. The Usability test was performed in Google chrome on a Macintosh computer. The tasks that were put forth and the reasonings for including these as tasks were the following:

  • Follow the process of getting coverage as an individual living in the state of Maryland, who earns $14,000 / yr. Stop once you are redirected to the state website. This tests the users ability to interact on the main page, and accomplish what most of what the traffic that visits the site is trying to accomplish.

  • What number should you call if you are having difficulty signing up? This task measures the users ability to find necessary information if they are unable to sign up.

  • Download a PDF copy of the actual Affordable Care Act Law. This is a secondary task. Legal information must be made public and therefore it is imperative that this information is somewhere that is fairly accessible to the user.  
  • Where is the appeal request form and the address to send it to? This task is meant to test the users ability to find information on a possible aspect of the sign up process.

  • Find out what the fee/penalty is for not being covered. This task is also meant to test the information architecture of the site.

  • Find information on how one can enroll in the Small Business Health Options Program (SHOP) marketplace This task tests the other type of sign up that is available to businesses. This will also test the navigation of the site, by switching from individual to business.

  • Find information on what options are available to you if you have a multi-state company. This task is a small follow-up to the prior task, and would help to give insight on whether the user can navigate within the businesses tab.

USER ONE

(SILENT OBSERVER): User is female, between 20-30. They rated their experience with computers as "I grew up with them." They also stated that they had used the macintosh operating system previously. The user has skimmed Healthcare.gov but had admittedly never done anything serious on the site.

ACTUAL USABILITY TEST: 6:17 mins

  • Though the subject uses the internet quite a bit, for the first task, the user did not realize that the menu at the top of the landing page was a menu.

  • Subject clicked on green button, that was labeled, “Get coverage” but actually led the user to information about being covered, instead of actually starting the process of getting coverage.

  • “I can’t tell there are two separate navigations here”—test subject, when trying to figure out how to initially get coverage.

  • For the second task the user went to the bottom of the page and the information was exactly where it was expected to be.

  • User used the search function and was able to complete this task fairly quickly.

  • The user attempted to use the search function to find the appeal request form, but it returned an empty search. The user then had to go through the site to find where this form may be located. After two tries, the user was able to find this form, and the address to send it to.

  • The user found the fee/penalty information in an expected place, once the user realized that part of the navigation was a sub menu, she was easily able to preform this task.

  • Finding SHOP information took the user seconds to accomplish, it seems the learnability of the site was fairly good. She was able to understand where parts of the site were and was able to accomplish the last to tasks fairly quickly.

POST-INTERVIEW

The user gave the website an overall average score, but pointed out how the navigation did not look like a menu bar. The user also felt there was too much text in the sub menus and she found that overwhelming. The user exclaimed that the “banner is silly,” she felt as though the photos served no purpose except to distract the user from performing the tasks. The search function was also lacking in the users opinion, as she was not able to search for appeal forms. As for good aspects of the site, she felt the overall site was clean and had a nice aesthetic.

USER TWO

(THINK-ALOUD METHOD): User is female, between 61-70. They rated their experience with computers as I am familiar. She also stated that they always use the Macintosh operating system. The user has never been on the site healthcare.gov.

ACTUAL USABILITY TEST: 25:05 mins

  • After about 5 minutes of searching for how to complete the first task the subject skipped the task and moved into searching for the Affordable Care Act

  • This happen mostly by accident as one of the pages that the user was on mentioned the Affordable Care Act.

  • It turned out to be very difficult for the user to find the PDF version of the affordable care act. The user exclaimed out of frustration, “I don’t understand! I have never seen a website like this.”

  • After 3 minutes of searching for the Affordable Care Act the user then gave up and moved back onto trying to get coverage.

  • The user clicked on the green main button and was taken to information on how to get coverage “this is obviously wrong,” she exclaimed, so she returned back to the main page using the back button.

  • She was eventually able to figure out where to begin to sign up for coverage, but became confused when they were asked to complete a “Marketplace Application,” before this task was being referred to as coverage. The user was then interrupted by a pop up window that asked about whether they would like updates sent to them.

  • The user closed the pop up window

  • After several hints (the user was demonstrably frustrated) I was able to direct them back to the home page so that they might sign up for coverage.

  • The coverage put the user on the state website, which confused the user. The state healthcare website looked fairly similar to Healthcare.gov.

  • The user re-attempted to find the affordable care act, and was able to find it via the main menu. “’About the Law’, oh, now I am used to going up to the top of the page to find things”

  • The user then attempted to find phone numbers at the bottom of the page, although the users found this to be an unexpected place to find the information they were looking for.

  • The user, after many minutes, was unable to find the Appeal Form. The tester attempted to give them a hint, but to no avail. The tester decided that this task should be skipped completely.

  • The user then attempted to find the fees and penalties for not having insurance within the actual act itself, and in the process was able to find the PDF of the Affordable Care Act.

  • The user wanted to read the entire act to try and get the fees and penalties. The tester attempted to steer them away from reading through the entire act. Within 3 minutes they were able to find the fee/penalties page.

  • For the last two tasks the user could not find the business tab, and was not aware of how the site navigation worked. After a hint from the tester, the user was able to figure out that there were two different sites. One for businesses and one for individuals.

  • The tasks were mostly completed.

POST-INTERVIEW

The user gave the website overall bad scores. When asked to rate if the website was easy to use the user gave it a 5 (very hard). The user also gave the website low marks when it came to rating if the navigation made sense, giving it a 5 (no, not at all). The user also felt that they could do no tasks quickly. On interviewing the subject, she expressed that the hierarchy of the site (the green buttons that are on the landing page), made it hard for the user. Mainly because the user wanted to do something simple initially (sign up for care) but the green button sent them to look at the law. The user felt the site was overall, too complicated. As for what the user liked about the site, she expressed that she didn’t like anything about it, she found the whole experience frustrating.

USER THREE

(CONSTRUCTIVE INTERACTION): The user is male, 20-30. The “helper” was user one. Being in the 20-30 age range the user also stated that they had grown up with computers, though admittedly, did not use Macintosh very often. The user had never been to healthcare.gov before.

ACTUAL USABILITY TEST: 6:24 mins

  • The user was quickly able to get coverage, mostly by talking aloud with the helper user. The helper user did not give anything away, but was trying to direct and talk about the tasks at hand.

  • Because the helper user had done these tasks before, she quickly talked to the novice user into using the search function for find the “Affordable Care Act law” however the novice user input “affordable care” and no results were returned. Once “Act” was added to the search than the user quickly completed this task.

  • The user went, as expected, to the footer, and found the contact information for the helpline.

  • The user did not attempt to use the search function because the prior user had warned him that it would not work. Instead he went to the main page and found it under the menu options.

  • The user went through the necessary steps to find fee/penalty information. The user seemed, with guidance from the helper, seemed fairly certain as to where this information can be found.

  • The last two tasks were completed very quickly.

POST-INTERVIEW

The Male user gave the website a fairly positive score (better than both the female users). He rated both the ease of use and the navigation as a “3” on the questionnaire. He did rate “being able to preform tasks quickly” as a “4”. The user did say that the experience could have been daunting had the helper not have been there. This user also felt there might be too many options under the main menu tab making it fairly confusing. When asked what he would change the user stated that there should be more menus on the top bar, and that the search function should be improved. When asked what the user liked about the site he did like the design and thought that the website was “easy to understand.”

CONCLUSION

SUGGESTED IMPROVEMENTS TO THE SYSTEM: In general, this site, while it could use some work, is attempting to take a fairly complex situation (signing up for healthcare) and make it user friendly. This site is especially challenging because it must reach a very wide audience. While the attempt is good there are numerous improvements that could be made. Every participant had issues with the main navigation, so that must be addressed.

  1. The main navigation bar should be made simpler by taking extraneous information and putting it into a separate tab. This would help avoid confusion when users are attempting to navigate to the most important task (getting healthcare).

  2. Make signing up for healthcare the main task on the landing page.

  3. There is only a tiny delineation between the employers healthcare page and the individual healthcare page (the color is slightly lighter). One of the participants expressed that this may confuse other users.

  4. Make it apparent when the user leaves Healthcare.gov to go to a state page. This would help the user return to the federal page as necessary.

  5. The navigation should appear to be clickable. Many users were unable to tell that the top most menu bar was, in fact, a menu bar.

bottom of page