User Evaluation of Technology

What is the course about?

This course presents a wide variety of methodologies for evaluating technological interfaces with users. It is aimed at providing engineering students with the necessary knowledge on working with users, statistical techniques for analyzing the information gathered from the users, and how to interpret the results from the user studies.

Logistics

When and where?

Porter Acad 246, Tuesdays and Thursdays 10:00-11:45AM

Who to contact?

Sri Kurniawan: Instructor

Assessments

  • Take home exam: 30% - due Nov 29, 2011.

This exam requires you to perform data snooping on the quantitative and qualitative data from this survey. The deliverable is a paper (no page limit, please use the ACM template) that has the following sections:

  1. Introduction: why you chose to investigate this particular case (e.g., why are you comparing data from older vs. younger people, or men vs. women, etc).
  2. Literature survey: Other studies that show that your choice of topic is interesting (e.g., there are differences in the way men and women perceive the usefulness of cell phones).
  3. Data analysis: quantitative and qualitative. I expect to see a bunch of tables from SPSS (or any other statistical software) and the content analysis mapping (use visio or mind mapping software, please)
  4. Discussion and conclusion. For discussion, I expect to see an analysis of your findings, relating them to other papers or other related data (e.g., explaining statistical significance of quantitative data with qualitative data).
  • One homework: 10%. Please note that in general I do not accept late homework. Any late submission will require evidence of a mitigating circumstance. The deliverables are:
    1. IRB protocol form (the full review, not the exemption)
    2. Test plan (see lecture notes from the first day of class)
    3. Recruitment material
    4. Survey, interview or observation guide
    5. Consent form. 

 

  • Student-led class presentation: 10%. In relevant weeks, students will be asked to read some chapters from the textbook and when appropriate some additional papers. The students may be asked to synthesize the chapters and papers; when appropriate collect/analyze some data as a proof of concept of the topic; and present the synthesis/data analysis in class. This presentation should be presented in lecture format, around 50 minutes long, followed by 15 minutes of class discussion. Please send me the ppt or pdf file the day before the presentation so that I can upload it to the class website.
  • Class discussion participation: 10%. In almost every week, I will assign paper readings that are relevant to the topics presented that week, and will randomly ask a student to lead the discussion. The lead will summarize the paper and critique it, and the rest of the class then can start the discussion/debate - you have 30 minutes for this exercise, it is up to you how you are going to use this time slot. The lead will not be the student presenting the chapters of that week. Of course, most likely you will need to read the Graziano chapters to understand the papers.
  • Individual project: 40%.
    For this project I need you to recruit an appropriate number of users to evaluate a technological interface of your own choosing. You need to collect both qualitative and quantitative data, analyze the data, interpret them and provide conclusions in terms of improvements to the technological interface to make it more user-friendly. 10% of this grade is for the presentations during the final exam slot, and 30% is for the report. More detail about the project will be given in Week 4 after the add/drop deadline.
    As a reference point, this is the sort of data gathering and analysis I am expecting you to do in your project: A paper on Older people and mobile phones: A multi-method investigation. The deliverables for the final exam are:
    1. A report in the format resembling the example paper above.
    2. 15 minutes presentation covering: motivation (why did you choose that particular interface), methodology (participants, procedures, measures), results, discussion and conclusion.
    3. 5 minutes Q&A.

Policies and accommodation

  • All students enrolling in this class are advised that Academic Integrity will be strictly enforced. Academic misconduct includes but is not limited to cheating, fabrication, plagiarism, or facilitating academic dishonesty or as further specified in campus regulations.
  • If you qualify for classroom accommodations because of a disability, please submit your Accommodation Authorization from the Disability Resource Center (DRC) to me in a timely manner, preferably within the first two weeks of the quarter. Contact DRC at 459-2089 V, 459-4806 TTY.

 

Readings/resources

  1. Textbook: Graziano, A. M., & Raulin, M. L. (2006). Research methods: A process of inquiry (you can get this used for $1-4 on Amazon).
  2. Lazar, J., Feng, J. & Hochheiser, H. (2010). Research Methods in HCI
  3. Cairns, P. & Cox, A.L. (2008). Research Methods for Human-Computer Interaction
  4. Usability Evaluation Methods
  5. Emerging research methods for understanding mobile technology use. In Proceedings of OZCHI, pp. 1-10.
  6. Various papers assigned in class.
  7. IRB.

Instructors and Assistants