Recommender Systems, User-based Perspectives, Recommending Research Papers

Building Document Recommender Systems to Meet User Information Seeking Needs
by Sean McNee, Computing Research Scientist at Attenex Corp.

Issue:
Information Overload – there is a massive amount of information today.

Solution:
Recommender Systems

Additional Issues:

Issue: There is an overload of Research Papers

Issue: Recommendation Systems need to be considered from a User-Centric Perspective

See below for summary. (Note the findings about the suitability of a particular algorithm and about user perspectives on lists of results)

Introduction

A Recommender System is a system that
- helps a user navigate a complex information space
- points out salient bits of information
- based on the current user, other users, and items in the domain
- usually within a process and context

Human-Recommender Interaction theory
- link recommenders to user information seeking needs

Kinds of Recommenders include filtering interfaces, classification interfaces, recommendation interfaces, and prediction interfaces.

Recommender Systems are the combination of Machine Learning and HCI. There are different kinds of algorithms for associating items: Collaborative filtering, Bayesian, Probablistic Latent Semantic Analysis, and TF/IDF Content-based filtering. Add these machine-learning algorithms to a human-centric interface, and you get a “Recommender System.”

Overview of Recommender Systems
1. The Ratings Matrix: Users vs Items. The recommender system decides how to completely populate a sparsely filled matrix. (Netflix example: Bill likes 6 movies, Tim likes 6 movies, Jesse like 8. How should Bill rate all the movies? How would everyone rate a particular movie?). One strategy is to identify similar users who are “neighbors” to a particular user.

2. Domains have different contexts and processes
- Taste-centric (Netflix, Pandora, Amazon, MovieLens, etc…)
- Research papers
etc…

3. Human-Recommender Interaction
A theory to bridge machine-learning and different-domains/user-needs.

Recommending Research Papers
Information Overload is an issue with research papers. There is a lot of effort to study this: bibliometrics, paper authorship networks, and recommendations from social networks (Referral Web, Relescope). There is a lot of related work: Theories on Human-Information Behavior, Information Retrieval, Sense-Making, etc…

Considering recommendation systems as a solution in this problem space, start with the recommendation matrix (items-to-users). Sean starts by reconceptualizing this matrix as a paper-to-paper matrix: Papers cite papers and Papers can be seen as potential-references to other papers.

Sean compared a set of recommender algorithms against co-citation matching, localized graph search, and google search. As part of the evaluation, Sean interviewed participants about relevance, and novelty. Notably, highly relevant documents were not the most novel.

Drilling into this finding, Sean found different algorithms to be better suited for “novelty” vs. “authoritative” vs. “introductory” vs. “survey/overview”.

Other findings: Finding 1 doc. in a list of 5 is considered a success to the users.

Rethinking Recommenders
Individually good recommendations do not equal a good recommendation list
Other factors are important
- Diversity
- Appropriateness

User Satisfaction is more complicated than just the individual items. The makeup of the list is important to users.

A User-Centric Perspective
Why do users come to recommenders?
- They want something
Users cannot express needs to a computer
- Need a common language
Recommenders as Social Actors
- “My TiVo thinks I’m gay”
- The Media Equation

Human-Recommender Interaction (HRI)
- User percieve recommendation quality in conext; users evaluate *lists*, one at a time.
- Users develop opinions of *recommenders* based on interactions over time
- Users have an information need and come to a recommender as a part of their information behavior

3 HRI Pillars and Aspects:
Recommendation Dialogue: Correctness, Spread, Saliency, Usability, etc…
Recommender Personality: Personalization, Adaptability, Freshness, etc…
End User’s Information Seeking Task

Sean also developed a process for developing recommender systems. This process accounts for Users/Tasks, HRI Principles, Metrics, and Benchmarks. Some measures of a recommender system include boldness and adaptability. Further findings show that different algorithm tunings are better and worse for a particular measure (e.g. for adaptability, different tunings of a recommendation system has a consistent effect on the adaptability of a system).

In relation to Digital Libraries
- Recommendation Systems need to adapt to the user the same way a librarian does.
- Recommendation Systems can link across an enormous number of specialized fields, finding latent connections.
Questions and Comments:

What are the success metrics for a recommender system? There are narrow and broad views of “better”.

How important is the rating system? 2 star ratings vs. 5 star ratings vs. other variables (e.g. I bought this book)

How important is length of the list of results?
It seems that the way that the list displayed is critical as well.

HRI seems to map a qualitative experience to a quantitative algorithm.

Suppose recommendation systems win and become ubiquitous. Would we be degrading people’s ability to judge good and bad stuff? (Suppose 6th graders are using this all the time for school). One approach: Consider LibraryThing’s “unsuggester”… Other approach: Randomly toss bad apples into the list as a learning tool? Design a system that doesn’t give you the answer right away?

The user-interface of recommendation systems appears to be important. Consider how the user-interface can help a user tailor the behavior of the algorithm to suit his preference for “boldness”, etc…