The advent of crowdsourcing is revolutionizing data annotation, evaluation, and other traditionally manual-labor intensive processes by dramatically reducing the time, cost, and effort involved. This in turn is driving a disruptive shift in search and data mining methodology in areas such as evaluation, supervised learing, and applications.
Evaluation: the Cranfield paradigm for evaluating search engines requires manually assessing document relevance to search queries. Recent work on stochastic evaluation has reduced but not removed this dependence on manual assessment.
Supervised Learning: while traditional costs associated with data annotation have driven recent machine learning work (e.g. Learning to Rank) toward greater use of unsupervised and semi-supervised methods, the emergence of crowdsourcing has made labeled data far easier to acquire, thereby driving a potential resurgence in fully-supervised methods.
Applications: Crowdsourcing has introduced exciting new opportunities to integrate human labor into automated systems: handling difficult cases where automation fails, exploiting the breadth of backgrounds, geographic dispersion, and real-time crowd response, etc.
Users have taken a more and more central role in the Web. Their role is both explicit, as they become more savvy, they have more expectations, and new interactive features keep appearing, and implicit, as their actions are monitored at various levels of granularity for various needs from live traffic evaluation for usage data mining to improve ranking, spelling etc.
In a few years, most Web applications will have the ability to successfully adapt to both the explicit and implicit needs and tastes of their users. Such adaptation requires the ability to model the user’s personal goals, interests, preferences and knowledge, and to apply this model while users interact with various applications. While adaptive applications that are based on user modeling have attracted the attention of multiple communities, from AI to UI, there is no forum that specifically focuses on user modeling and adaptive applications in the Web domain.
This workshop will focus on user modeling and the usage of these models in Web applications. The emphasis of the workshop will be on modeling techniques that scale for the Web. User modeling might be based on explicit and implicit user feedback gathered from variety of sources such as sign-on information, clickthrough data, user previous queries, social network, purchases, and real-world activity. Adaptive Web based applications include search personalization, advertisement targeting, recommendation systems, social networks, on-line shopping, etc.