TIM260, Fall 2013, Section 01: Project

Grading criteria

1. Project proposal: 20%  (Submit to ecommons)

2. Final project report: 80% (submit here)

· General steps*

1. Pick a topic

2. Form a team

3. Read related work

4. Write a project proposal

5. Work on the project (follow your schedule)

6. Project progress report

7. Present your project (optional, invitation only)

8. Write a project report

· 1. Pick a topic

You can either pick from the list suggested on ecommons or choose your own topic.

In general, a topic would fall into one of the following three categories:

1. Evaluation: Empirically evaluate one or more existing IR algorithms in order to better understand these methods, and possibly further improve them. 2. Original study of basic IR techniques: Propose a new method or formula, and compare it with a baseline algorithm on a standard collection. 3. Application: Develop a new IR application using an existing method. 4. Literature review of a special topic (single person), very different evaluation methodology, how you organize the literature is your technical depth and contribution   Example: SIGIR/CIKM/WWW papers

 

When picking a topic, try to ask yourself the following questions:

1. Is the topic addressing an important problem (or an important need, in the case of building an application) ? Would any one care about it, if you solve the problem, or build the application?

2. Do you have any idea at all about how to solve the problem, or build the publication? If not, can you reformulate the problem to make it easier?

3. A good topic is one that you like to work on, that you have some idea about it, and that is small enough for you to finish by the end of the semester.

· 2. Form a team

You are strongly encouraged to work with at least another student as a team. This not only would give your some experience on working with others, but also would allow you to work on a larger (presumably more interesting) topic.

Of course, you can also work on your own (i.e., one-person team), if you want.

· 3. Read related work

Once you have chosen a topic, you will need to read some papers to know about what people have already done on the topic, so that you can be sure your idea extends , rather than duplicates, the existing work, if any.

· 4. Write a proposal

If your project involves testing a hypothesis or original research, your proposal should state explicitlyall the following:

1. What your research question is.

2. Why this is an interesting question to ask and why we would care about the answer to this question.

3. Whether any existing research work has tried to answer the same or a similar question, and if so, what is still unknown.

4. How you plan to work out the answer to the question.

5. How you would evaluate your solution. That is, how you plan to demonstrate that your solution/answer is good or is reasonable.

6. A rough timeline to show when you expect to finish what.

 

If your project is to build a software tool, then your proposal should state explicitly all the following:

1. What the function of your tool exactly is. That is, what it will do.

2. Why we would need such a tool and who you would expect to use it.

3. Whether people have already built such a tool or a similar tool, and if so, how yours is different from or better than other tools.

4. How you plan to build it.

5. How you will demonstrate the usefulness of your tool.

6. A rough timeline to show when you expect to finish what.

7. VERY IMPORTANT: Please specify clearly what you plan to actually finish by the end of this semester.

8. The web site of your project 

 Please submit your proposal on ecommons. 

· 5. Work on the project

You should reuse any existing tools as much as possible. For example, consider using the Lemur toolkit or any of your assignment work, if possible. There are also many tools available on the Internet. See the resources page for some useful pointers.

Discuss any problems or issues with your teammate if any. Discuss them with other students in the class. Discuss them with the instructor at office hour, before or after the class.  If you need special support (data etc.), please let the instructor know.

Consider documenting your work regularly. This way, you will already have a lot of things written down by the end of the semester.

Please create a web site for your project. 

· 6. Project progress report

Sometime in the middle of the project, you will be asked to present a project progress report to the class. You do NOT need to turn in any written progress report.

· 7. Present your project

At the end of the semester, invited teams will be expected to make a PowerPoint presentation to the class. Each project will have *+2 minutes. You should leave at least 2 minutes for questions. Please give your presentation in PowerPoint. For a team with more than one person, it is up to the team to decide how you would like to run the presentation. You may decide to have one person to present the whole project, or have each person to present one part of the project. Either way is fine. In general, it is the project, not the person, that will be graded, so normally, those students in the same team would get an identical grade for the project work, unless one has not done sufficient project work to justify the project grade. For this purpose, you must say very briefly during the presentation how you collaborated and who did what.

In general, you will need to define and motivate the problem, describe your method(s), discuss the results, make conclusions, and discuss how the work can be further improved (i.e., future directions). Think very clearly about the key message you want to convey, and see how you can best use the given time slot to effectively convey the message to people. Since everyone has already listened to your progress presentation, you should try to avoid repeating too much what you have already presented, though you still need to say briefly about the motivation of your work. Grading of the presentation will be more based on the clarity of presentation than based on the quality of the project work being presented. The quality of the project work is the main criterion for grading the project report.

· 8. Write a project report

You should write your report as if you were writing a short conference paper. You can think of it as an expanded written description of your presentation; you may also think of your presentation as a summary of the report. Thus the same general guideline applies, i.e., you should (1) explain your problem clearly; (2) provide sufficient motivation for your work and explain how your work is connected with the existing/previous work; (3) explain your methods with sufficient details; (4) discuss the research results; (5) summarize your work, draw conclusions if possible, and discuss how you think the work can be further improved/extended; (6) what you have learned from this project. There is no strict length requirement. You may target at anywhere between 6 pages (font 10, single column) and 10 pages (font 11, single column) without counting any necessary appendices. Actually, given the same amount of essential information, the shorter the better; of course, you will have to judge what counts as "essential information". A good report is not just a straightforward description of what you did (Such a paper would probably never be accepted by a good conference); it should demonstrate your research contributions very clearly and convincingly. Thus it is important that you think very clearly about what are the major points you want to make and include arguments and empirical evidence that support your points. For example, you may want to summarize or plot your experiment results in certain particular way rather than some other way, because the "particular way" would support your point better. Always keep in mind what exactly you expect your readers to learn from your report, including both positive and negative findings.

 

(based on Zhai etc.)