Grading Criteria

1. Assignment 1: Project Topic Description
2. Assignment 2: System Requirements and Navigation
3. Assignment 3: Interaction (Dialogue) Styles and User Characterization
4. Assignment 4: Web Site Review Grading Criteria
5. Assignment 5: Project Status Review Grading Criteria
6. Extra Credit Assignment: Web site critique
7. Assignment 7: Individual Assigned Reading
8. Assignment 8: Usability
9. Assignment 9: Usability Study of an Existing System
10. Final Project Report (.pdf format)

Assignment #1: { Due @ September 11, 2001, 6:30pm PDT }

Back to Top

Assignment #2: System Requirements and Navigation.

     { Due @ September 18, 2001, 6:30pm PDT }

• When you generate your system requirements, use full sentences to describe them. Although you will use full sentences, keep them simple, readable, and grammatical. We only need the high level requirements. Make your requirements "testable". Try to avoid a broad description. The requirements are for your system, not for your users. Use precise and direct language to write your requirements, for example, use the word "shall" instead of vague or confusing words like "should"..
  
• For the site/navigation map, use one page to present your map. It should resemble a web site map, or a menu-tree-like hierarchical structure that shows how a user can move within the system that you plan to design. Include a few words of explanation to describe the main features of the site/navigation map.

• Both the requirements and the site map should change as you learn and discover more as your design matures. 

• v = both parts are completed satisfactorily. You are heading down the right path
  v- means resubmit this homework. 
  v+ = all possible requirements for your system are covered. The requirements are written in the specified format (specific English usage described above). The site map is clear and easy to follow. 
  
  Remember to write your names and team number, if you have received it, plus your location (campus or off-campus-location-name).
  

Back to Top

Assignment #3: Interaction (Dialogue) Styles and User Characterization

{ Due @ September 25, 2001, 6:30pm PDT }

There are two parts to be completed this week: select the Interaction (Dialogue) Styles and perform a User Characterization analysis.

• Part 1: When you evaluate your project's interaction styles, use Shneiderman's discussion on dialogue styles. In particular, see the class notes Pages 2-99 to 2-102. You are taking this table and modifying it for your project, filling in the elements in the table based on team agreement. You should evaluate based on the user psychology, knowledge and experience, and job and characteristics of your project. You have to present the evaluation in a tabular form. Also, explain the reason for choosing or not choosing certain dialogue style(s) for your project's UI. You have to figure out what the user psychology, knowledge and experience, and job and characteristics of your project are and explain why. 

• Part 2: In the second part, you are asked to perform a user characterization. See class notes, pages 3-7 to 3-11. You have to identify the following: user characteristics, user tasks, user workload, and user environmental considerations. Much of your analysis was already performed while completing Part 1. Also, complete and/or modify the "User Profile Checklist" (Notes page 3-10) for your project. 

• v = both parts are completed satisfactorily. You are heading down the right path. Dialogue Style Summary and User Profile Checklist pages are Xeroxed right out of the notes and edited with your project information
  v- means resubmit this homework due to errors or incompleteness. 
  v+ = all factors which will affect your decisions on the dialogue style(s) for your system are covered and explained clearly. All possible user characteristics (who is the user), user tasks (what tasks the user performs), user workload (for example life-critical systems vs. casual browsing and information gathering sessions), and user environmental considerations are analyzed and listed. Example environmental considerations are: working environment for your project system, e.g in a car, handheld computer, standing at the ATM, sitting at home, working alone or with others, etc.). V+ homework include some written explanation to clarify the decision made.
  
  Remember to write your names and team number, if you have received it, plus your location (campus or off-campus-location-name)
  

Back to Top

Assignment #4: Web Site Review Grading Criteria     

{ Due @ October 2, 2001, 6:30pm PDT }

This is a team assignment. Critique a web site design using what you have learned. Select a web site of a major corporation or government entity (a large city, state, etc., for example, cityofla.org) and comment on what you like or dislike about that web site. Base your critique on what was discussed in class, what is in your supplemental notes regarding web site design, and what is in your class textbook. Point out any comments you have with respect to the source, e.g. Shneiderman's text says---, Jakob Neilsen's www.useit.com web site suggests---, usableweb.com has a better example of ----, it was discussed in class that ---, to further justify your critique. Also, use the web design checklists on the Class Notes Pages 2-20, 2-21 and 2-22 to guide your discussion. You need not hand in those pages.

Limit your comments to two single-spaced, typed pages, maximum. Comment on amount of "eye-catching" visuals, advertisements, or other astonishing interfaces, as well as appropriateness for basic functions we have discussed such as search, help, site maps, navigation aids, etc. What about icon design, use of color, ability to print, frames? Supply a hardcopy of the home page of the web site also (black & white is OK).

v = one or two pages web site critique and provided the URL of the web site reviewed. The critique is subjective and does not point to any of the class references. 
v- = resubmit this homework. 
v+ = All important features of your selected web site are included in your critique. All comments are clear and hitting the point. You kept it to two pages. You used references, as appropriate, to substantiate your assessment of the web site. You made constructive suggestions on how the web site could be improved. You noticed any technology impacts of this site, e.g. too many unproven plug-ins or unreadable in older versions of Windows, etc. You remembered the hardcopy of the home page of the selected site. 

Remember to write your names and team number, if you have received it, plus your location (campus or off-campus-location-name).

Back to Top

Assignment #5:Project Status Review Grading Criteria
{ Due @ October 23, 2001, 6:30pm PDT }

Your team is required to submit one hard copy of your project status review. The approximate pages (slides) in presentation format for each part is listed on “Project Status Reviews I and II Agenda for October 23 and 30, 2001”. When you generate the slides, previous assignments will be the great references. Remember to consider the comments from our Grader, T.A. or Professor. Those comments will definitely help your team to improve your project. You can employ the content of your assignment 1, Project Description, in your Topic Explanation.

Address any assumptions you make for building your system. Also, refer to the grading criteria for System Requirements and Navigation Map again to insure these two are complete. For User Analysis Discussion, a Data Flow Diagram of tasks being performed by your system should be added this time. You only have to list what you’ve done so far.

Completion of the whole system is not required now. Use about 10 pages for your screen prototypes. Please write some explanation for the screen designs and navigation.  Sequence your slides following sample path(s) through your system. Try to create an example of how the system would work when the user is interacting with the system. Write comments, issues, complaints, assumptions,  tool(s) used and plans for the rest of the semester.

It is expected that you will be copying or repeating portions of your previously completed homework. The reason for the homework is an incremental development of your class project.

You must include one cover page in the hard copy. You have to list your team number, team member names, and your team location ( write “Campus” or off-site location name).

v+   Followed outline explicitly, addressing each item. It is taken into account who presents on 10/23/01 vs 10/30/01 in terms of preparation time allowed. Entire team presents project, each doing a portion. Team addresses screen designs factoring in issues and guidelines discussed in class. Teams illustrate corrections of design and decision made based on feedback of doing homework, class discussion, team discussion, etc. Written explanations of screen designs and navigation included in hardcopy handed in, not just screen dumps.

v  Interim capability demonstrated, but, a lot of more work to do. Team only follows general topic areas on outline, but does not address everything requested. Only 1 person of 2 or 3 person team presents team project.

v-  Team presents screens drawn using drawing tools in PowerPoint or handwritten (low fidelity prototype), with only a small subset of the eventual design. Sections missing from the outline. Lack of team cooperation apparent.

Back to Top

Extra Credit (Individual) homework

{ due 10/23/01 }

Visit http://www.cityofla.org/

Regarding Midterm Question #1
Review the web site above and state any assumptions you may have made while taking the midterm, and now that you have had a chance to review the web site with a test case, e.g. obtaining a dog license, how would your critique of the web site change, if at all.

Suggested length: two paragraphs, typed. Certainly less than one page.

v+  Recall any assumptions you made and state whether they were correct or how you would have changed your midterm answer now that you have had a chance to review the site. State how easy or difficult it was to figure out how to obtain a dog license. Perform this assignment as an
individual. About two paragraphs in length, less than one page.

v   Worked this as a team. State subjective preferences of the web site, positive or negative. Just one or two sentences.

No one gets v- on this extra credit.
 

Back to Top

Assignment #6:Individual Assigned Reading Assignment

{ due 11/06/01 }

Select a technical journal article (not trade journal) or textbook from the references to read in depth. If you have questions about whether the article or textbook contains sufficient technical detail, e.g. an article that is not from a refereed journal, PLEASE contact Professor Jacobs.

If you want to review a "classic" article (i.e. more than or equal to five years old), contact Professor Jacobs for its worthiness. Otherwise, assume the article you review must be less than five years old.

If it is related to your project, state how and why.

Hand in a one to two pages, single-spaced, typed summary and critique of the article.

V+: truly evaluate the author's thesis, check out the author's references, and "take a stand" (especially if the article is a survey or controversial). You point of view is well-organized and using the material offered in class properly.

V: Explain your opinions based on what have taught in class. You digest everything important in the article you choose.

V--: Your personal opinions are generated without referring to the class material. You merely summarize the article without using any analysis or thought behind your summary. You choose an article from a trade journal or an article which is more than 5-years old.

Back to Top

Assignment #7: Usability

{ due 11/13/01 }

Part I. Requirements Verification. 

Determine if your system met (or will meet) the requirements you have developed for your project. Point out which method of requirements verification you would perform. Choose from IADT method (Inspection, Analysis, Demonstration, Test). You have to list all your system requirements for this assignment, developed as part of an earlier homework. You should present this verification in tabular form. Develop a table of your project requirements, which IADT method(s) apply, and add a column for comments and explanation.

v - means the project team did not write the requirements in the proper format, the student team did not complete the above table and the student team used poor explanations to describe how the project's requirements will be tested. Resubmit this homework.

v means that the project team completed the table satisfactorily. 

v + means that the student team completed the table and did an excellent job completing its contents, with a clear explanation of how the system will meet its requirements. The project team will be able to easily prove that the systems will meets its requirements.


Part II Usability Study

The project team needs to select method(s) to evaluate their user interface. The project team conducted a survey (like the QUIS) of Ben Shneiderman discussed in class or used a Usability Specification (like from the paper by Whiteside, Bennett and Holtzblatt), discussed in class, for their project.

v - means the project team only specified a sparse Usability Specification and did not take it further. Or, the project team copied a page or two from the QUIS survey right from the course notes/textbook and had 1-2 persons fill it out. Little or no analysis of the results was included. Resubmit this homework.

v means that the project team did develop Usability Specifications or copied relevant portions of the QUIS survey (one or the other). The project team had 2-3 persons complete the rudimentary survey. Cursory analysis of the results was included.

v+ means the students completed both a Usability Specification and customized survey of >3 persons, analyzed and summarized the results of the surveys and specification, and discussed the changes that will be made in the class project as the result of this analysis. The completed surveys are included in the homework submittal.

Back to Top

Assignment #9:Usability Study of an Existing System.

{ Due in Nov. 20 }

This is a team, not individual assignment. Please review an existing system (not your own project and not a web page) and perform the usability study. That system can be a device (e.g. VCR, cell phone, microwave, stereo, etc.) or a system (ATM machine, postage device, gas station pump) that has some user interface to analyze. Like in the previous week’s homework, develop and conduct a survey (like the QUIS) (developed by Ben Shneiderman) discussed in class or use a Usability Specification (like from the paper by Whiteside, Bennett and Holtzblatt), discussed in class, for the system. Since you were not the developer of this system, estimate the requirements of the system you pick. List all assumptions. 

v - means your team reviewed a web page and not a device or system as described above. You only specified a sparse Usability Specification and did not analyze it further. 

v means your team did a Usability Specification or copied part of the QUIS survey (1 or the other), with only cursory analysis of the results. 

v+ means your team completed both a Usability Specification and customized survey of more than 1 person, analyzed the results, and discussed the usability of that device or system in a clear, coherent fashion. This team made observations from this second usability homework and compared it briefly with what was performed last week. Any changes to your project as a result of this week’s assignment of an existing system or device is noted. 

Back to Top

Final Project Grading Criteria

Each team is required to hand out one hard copy for your final project on December 4, 2001. The date is firm. No delayed submission will be accepted. Remote students must ship their project to arrive at USC DEN on December 4. The outline is listed at the end of the grading criteria.

The number of pages for each subject is not fixed. However, the approximate number of pages is listed in Final Project Outline. Use more pages to include all your hard work is welcome. Don’t skip any part. You are also required to submit the hard copy of your codes. Posting web-based solutions on our class web-page will be an extra credit. Please fasten your project submittal in a way the pages won’t get lost.

Cover Page [4 points]:

You have to put your Team number, Team member name, student ID numbers, and Team location (Campus or off-site location name) on the cover page. The outline for the final project should be followed. Use the subject to label all sections.

Topic Explanation [4 points]:

Your topic name is included with one paragraph to describe your system. State all assumptions you made if any. (Assignment 1)

System Requirements [8 points]:

When you generate your system requirements, use full sentences to describe them. Although you will use full sentences, keep them simple, readable, and grammatical. We only need the high level requirements. Make your requirements "testable". Try to avoid a broad description. The requirements have to be written in the specified format .You need to cover all possible requirements for your system in the final project. Include from the homework whether your system met (or will meet) these requirements. Choose from IADT method (Inspection, Analysis, Demonstration, Test). Develop a matrix of your project requirements, which IADT method(s) apply, and add a column for comments and explanation. Make a clear explanation of how the system will meet its requirements. (First part of Assignment 2 and first part of Assignment 8)

Navigation or Site Map [8 points]:

For the site/navigation map, use approximately one page to present your map. It should resemble a web site map, or a menu-tree-like hierarchical structure that shows how a user can move within the system you plan to design. Include a few words of explanation to describe the main features of the site/navigation map. (Second part of Assignment 2)

User Analysis Discussion [8 points]:

For the User Characterization, you have to figure out what the user psychology, knowledge and experience, and job and characteristics of your project are and explain why. For User Analysis Discussion, a Data Flow Diagram of tasks being performed by your system should be added.

All possible user characteristics (who is the user), user tasks (what tasks the user performs), user workload (for example life-critical systems vs. casual browsing and information gathering sessions), and user environmental considerations are required to be analyzed and listed. Example environmental considerations are: working environment for your project system, e.g in a car, handheld computer, standing at the ATM, sitting at home, working alone or with others, etc.). Include some written explanation to clarify the decisions made. (Assignment 3)

Tools/System [8 points]:

Discuss why and what tools are selected for your prototyping and implementation. Use the tradeoff matrix of tool alternatives. Use a table format to present the tool capabilities or cost versus tool(s) used or considered. Include explanations in the table. List the challenges for the final tool decision, if any. (Not a previous homework)

Evaluation [8 points]:

Discuss methods used to evaluate your system. Include a Usability Specification. Use the survey you’ve conducted. Analyze the results and discuss the changes that will be made in the final project as the result of this analysis. (Second part of Assignment 8)

Specification [8 points]:

Specify a small portion of your system using one of the techniques covered. For example, UAN, IRG, or state diagrams discussed in the lecture on 11/27/01. Clear explanations are required. (Not a previous homework)

Screens [24 points]:

Use about 10 pages for your screen prototypes. Of course, the number of pages will vary. Please write some explanation for the screen designs and navigation. Sequence your slides following sample path(s) through your system. Try to create an example of how the system would work when the user is interacting with the system. The discussion of your improvements made since the October Project Status Reviews expected in this section also. You may want to include those previous screens and how you improved them. Write brief comments for the improvements you’ve made in the final project. For each screen, specify what inputs caused you to move to the next screen, for example the menu item selected or button pushed. Mark that on the hardcopy.

Comments/ issues/ lessons learned [8 points]:

State and explain the changes you made. Include the log of design decisions that affected your system design. Comment on any challenges you’ve encountered. Discuss what you’ve learned and the possible improvement or new work could be done in the future. Discuss what you would have done if you had more time. Address how newer UI technologies could enhance your project. Review the V+ sections of the posted grading criteria for previous assignments (on thee class web site).

Code Hardcopy [12 points]:

The print outs or CDs for your codes to prove the project was implemented as stated are required. Also, compress your codes into *.zip format and mail it to our grader, Frank. Please use the filename:”team20.zip” for team 20. Replace “20” with your own team number.

Class Project

Final Project Outline

for December 4, 2001 (FIRM)

Approximate:

Pages               Subject

 1                      Cover Page

                                Team Number,

                                Team Member Names,

                                Team Location (Campus or off-site

                                                        location name)

 1                      Topic explanation

                                (state assumptions, if any)

 1                      System Requirements

Top-level system requirements. Also, include from the homework whether your system met (or will meet) these requirements. Discuss IADT test verification methods illustrating how you tested that your requirements were met.

 1                      Navigation or Site Map

                                (navigation hierarchy, e.g. menu tree)

 3-4                   User Analysis  discussion

                                User Characterization

                                Task analysis

                                       Note: do a data flow diagram of

                                                 tasks being performed by

                                                 your system

                                Workload analysis

                                Environmental considerations

                                Dialogue/interaction style(s) selected

                                I/O device(s) selected

 1                      Tools/system

Tools selected for prototyping and implementation (if any). Discuss or present tradeoff matrix of tool alternatives

1-2                   Evaluation

Discuss methods used to evaluate your system, e.g. usability matrix, survey, etc. Then, evaluate it.

1-2                   Specification

Specify a small portion of your system using one of the techniques covered, e.g. UAN, IRG, state diagrams, etc.

10                   Screens

Screens with explanation
Follow sample path(s) through your system, i.e. an example of how the system should work

1                      Comments/issues/lessons learned. 

Explain changes. Summarize major decisions that effected your design. State what new UI or Input/Output technology changes you would make for your system design. Comment on any challenges, e.g. use of a tool that proved insufficient for your design. What did you learn this time that you would do differently next time.
                               

Open                        Code hardcopy

Comment: I fully understand that you will be resubmitting completed homework as part of this. Check grading criteria web page on class web site.

                 Final Project Grading

          4  Cover page/title/followed directions

          4  Topic explanation

          8  System requirements

          8  Menu tree or equivalent

          8  User Analysis

          8  Tools

          8  Evaluation

          8  Specification

         24  Screen design

         12  Credit for implementation/code

8         Comments/issues/lessons learned

+ Credit for posting web-based solutions on class web page

         +-  Teaming considerations

        100  Total possible points

Projects will be returned. Campus student projects will be left in or near the Computer Science Department Office. Remote students will have their projects returned via the normal process. Anyone who is out of the area (or graduating) and would like the projects returned by mail, please provide a stamped, large envelope to Professor Jacobs and the project will be mailed back to you.

Back to Top