[Please read carefully before attempting the task!]
I. General Instructions:
- Attempt HITs if you are a native speaker of English.
- We are happy to receive feedback and improve this job accordingly. Feel free to send your comments to: s.angelidis [at] ed [dot] ac [dot] uk or use figure-eight's messaging.
- Your responses are confidential. Any publications based on these will not include your specific responses, but rather aggregate information from many individuals. We will not ask any information that can be used to identify who you are.
- Important: This task is, to some extent, subjective, so we do not use test questions to evaluate the quality of answers. However, we monitor answers to identify workers who don't follow our basic instructions or give random answers. Such workers will be rejected and banned from this and future tasks.
II. The Goal: Summarizing Customer Opinions about Hotels
The goal of this job is to generate high-level summaries of the opinions expressed in a set of review sentences about a particular hotel. The review sentences presented to you have been previously identified as representative and useful.
At each page, you will see sentences about the hotel which have been grouped according to the aspect of the customers' experience they discuss, e.g., the quality of the rooms, or the cleanliness of the hotel, or the location of the hotel, etc.
You are asked to summarize the opinions at a high level, by producing 1 summary sentence for every aspect. The full summary must not exceed the upper limit of 100 words. We are not looking for summaries that cover all expressed opinions in detail. Instead, only the most common and useful opinions should be mentioned.
III. The Task
More specifically, your task is to:
- Read the review sentences that have been pre-selected and grouped into aspects for you.
- For each aspect, try to express the prevailing opinions of the reviewers in a single sentence. Add this sentence to the text box.
- There are one exception for the "1-sentence-per-aspect" limit:
- The "other" aspect, which you will encounter in some pages. This includes sentences that don't fit the rest of the aspects. Since these sentences might be diverse in content, you are allowed to write up to 3 sentences for it, if needed.
- Avoid including too many details in the summary. We enforce the limits of 1 sentence per aspect and 100 total words to make sure your summaries only cover the opinions at a high level.
- If contradicting opinions exist, use your judgment and summarize the prevailing ones.
- Your summaries should be simple, grammatical and coherent.
- Important: At times you will see irrelevant sentences which do not relate to the specified aspect. Please ignore these.
A few important notes regarding your summaries:
- Note #1: You should try to express opinions which relate to the specified aspect at a high level.
For example, the following sentences:The staff were only too happy to talk about the city of new orleans, with tips on where to go, best resturants etc.. We booked a swamp tour through the hotel which was great. Also all of the staff were effective and friendly and the rooms were very clean. Each time the front desk staff have been helpful and pleasant. The Good: The service really is as good as everyone says. To begin with, the staff was very friendly and helpful. The staff at the hotel are super nice and attentive. The staff was VERY accommodating and friendly. Front desk was friendly and helpful.
can be summarized as:
"The staff were friendly and helpful." - Note #2: If opinions in the pre-selected sentences directly contradict each other (e.g., "the rooms were large" and "the rooms were very comfortable" vs. "the rooms were smaller than I would like"), you should use your judgement and write a sentence that expresses the most common opinion. E.g., in this case: "The rooms were quite comfortable".
- Note #3: Your sentences should not be written in first person.
For example, you should write:
"The beds were really comfortable."
but you should not write:
"I found the beds really comfortable." - Note #3: Your summaries should not include references to the reviewers, like:
"Many reviewers found the beds comfortable" or "One person didn't like the bed"
III. The Interface
For this task, we have prepared an interface that, hopefully, helps you generate a summary as quickly as possible. We have attempted to make it easy for you to:
- Identify the main keywords discussed in the pre-selected sentences, and sort the sentences based on these.
- Discard irrelevant sentences.
- Keep track of the sentences that you have used so far to generate the summary.
Below is a sequence of screenshots highlighting the main interface features. An instruction video follows right after. Please spend a few minutes going over these to familiarize yourself with the task :)
1) Initially, the interface will look like below:
2) You can double-click on a keyword and all sentences that contain it will be grouped together at the top of the list:
3) The same can be achieved by double-clicking any word that gets highlighted when you hover the pointer over it:
4) Using these tools, you can start writing your summary. Remember: you must write 1 sentences per aspect! You can use the "Already included?" check-boxed to strike out sentences you've already considered (or decided not to use). Also, the "Is irrelevant?" check-boxes can help discard sentences that don't discuss the specified aspect. You DON'T have to use these check-boxes, they are only provided for your convenience.
5) After writing up the sentence of an aspect, move on to the next one! How much detail you provide in your sentences depends on your judgment (some details may seem more important than others), and on how much space you have (i.e., words left).
6) At the end, you have produced a high-level summary of opinions :) There are many ways to write such a summary, so don't stress about "getting it right". Just make sure that the important opinions are included, without getting into too much details.
IV. Video Demonstration