Comments on: Diversity in recommender systems: sketch of a bibliography

By: Andre Vellino

Andre Vellino — Wed, 26 Nov 2008 20:32:12 +0000

I would add this to your list of papers Daniel:

Recommender systems and their impact on sales diversity

http://portal.acm.org/citation.cfm?id=1250910.1250939
http://doi.acm.org/10.1145/1250910.1250939

By: Daniel Haran

Daniel Haran — Wed, 26 Nov 2008 16:27:18 +0000

Wow, Beyond Algorithms is great!

Fig 7 is particularly interesting in the debate about whether CF will narrow our interests.

It also shows the importance of transparency (Table 1 + Fig 10).

By: Paul

Paul — Mon, 24 Nov 2008 19:32:47 +0000

Some other references for diversity in recommendation:

Measuring playlist diversity for recommendation systems by Malcolm Slaney and William White
Oscar’s (http://mtg.upf.edu/~ocelma) thesis (when he finally publishes it
Beyond Algorithms: An HCI Perspective on Recommender Systems
Kirsten Swearingen & Rashmi Sinha
Evaluating Collaborative Filtering
Recommender Systems – Herlocker, Konstan, Terveen, Riedl

By: Daniel Lemire

Daniel Lemire — Mon, 24 Nov 2008 18:38:16 +0000

Oscar: Good point. And this stresses the fact that “probabilistically less accurate recommendations” may prove far more valuable.

By: Oscar

Oscar — Mon, 24 Nov 2008 17:44:24 +0000

Very interesting topic Daniel!

I see two approaches regarding ‘diversity’ in RS:

1) Try to avoid too similar items in the top-N recommended items.

E.g Avoid this: given a user that loves Woody Allen, then recommend Woody Allen films that are not in her profile.

Also know as “The White Album effect”:
http://www.amazon.com/Beatles-White-Album/dp/B000002UAX?tag=daniellemires-20

2) Try to present a mix of familiar and unknown items to the user.

E.g. Given a user that really likes The Rolling Stones, recommend to her:
The Beatles
The Who
Mick Jagger
Led Zeppelin
etc.

Most of these recommendations are obvious, so why do I need a recommender, then?

However, it would be great to get some more “interesting” recommendations, such as:
Robert Johnson
Muddy Waters
Chuck Berry
The Faces
The Georgia Satellites
Sol Lagarto
Diamond Dogs
The Dogs d’Amour
…

I’m particularly interested in the second approach, as it really helps users discover unknown items that otherwise would have been very difficult to get into.

Cheers, Oscar

By: Kevembuangga

Kevembuangga — Mon, 24 Nov 2008 16:08:01 +0000

Isn’t lack of diversity some sort of overfitting?
If not how you would you define the difference?

By: Thomas Deselaers

Thomas Deselaers — Mon, 24 Nov 2008 15:58:10 +0000

Here are some papers which I also consider relevant:

* Zhai, Cohen, Lafferty: “Beyond Independent Relevance: Methods and Evaluation Metrics for Subtopic Retrieval” SIGIR 03

* Song, Tian, Huang: “Improving the Image Retrieval Result Via Topic Coverage Grap” PCM2006

* Chen, Karger: “Less is More: Probabilistic Models for Retrieving Fewer Relevant Documents” SIGIR06

* Tang, Arni, Sanderson, Clough: “Building a Diversity Featured Search System by Fusing Existing Tools”

* Arni, Clough, Sanderson, Grubinger: “Overview of the ImageCLEF 2008 Photographic Retrieval Task”

* Clarke, Kolla, Cormack, Vechtomova, Ashkan, Büttcher, MacKinnon: “Novelty and Diversity in Information Retrieval Evaluation” SIGIR08

* Xu, Yin: “Novelty and Topicality in Interactive Information Retrieval”

You might also be interested in the photo retrieval task of ImageCLEF 2008, where diversity was taken into account.