Case for Support: foundations for the context-aware retrieval engine

Part 1: Previous research track record

We, Dr. Jones (GJFJ) and Professor Brown (PJB), have been working together on Context-Aware Retrieval since late 1999. The collaboration was strengthened when Professor Brown moved from Kent to Exeter in February 2000. Our early work involved bringing together the ideas of two previously disparate communities: information retrieval and mobile applications. We analysed how delivering information to mobile users, both by proactive triggering and interactive user queries, related to traditional Information Retrieval (IR) and Information Filtering (IF). Our conclusions [Bro01] were that Context-Aware Retrieval (CAR) represented a rather challenging hybrid between IR and IF. The challenge results from the dynamic nature of all the data involved, and from the need to provide near-continuous retrieval to the user. We further concluded that many of the huge advances in IF and IR over the past thirty years, many of them recently driven by web searching, could be exploited in CAR. In particular we concluded that the widely adopted retrieval strategy of "best-match" offered a much better foundation that the Boolean retrieval strategies adopted by most current CAR systems. We published some ideas on the scoring algorithms that could be used in CAR [Jon00]. These ideas have been further refined [Bro01a]. The essence of our new ideas is that, since CAR is so hard relative to conventional IR and IF, we must exploit those advantages that it does have, in particular that the user's context (which translates into a retrieval query) normally changes gradually and semi-predictably. In detail these ideas relate to scoring algorithms (e.g. is it better to score a distance field using an N^2 algorithm?), dynamic weighting algorithms (e.g. rapidly or suddenly changing fields being given higher weight), to the use of context caching, and to the use of a `Context Diary' that straddles the past and future. We give more details below

To support this work we have implemented a prototype CAR system called the Context Matcher; this is designed to act as a flexible testbed in which we can plug algorithms to test and to evaluate our ideas. This work was partly supported by an Emeritus Fellowship from the Leverhulme Foundation, which was awarded to PJB in December 1999 for two years. The implementation work has been carried out, in Java, by Dr. Lindsey Ford (LF).

As a result of this work we have attracted two potential industrial beneficiaries, Trilogy and Xerox, who are both collaborating with this bid.

In terms of combined expertise in retrieval and in mobile applications, we believe that Exeter University is now as well placed as any other research organisation in the world, with the exception of MIT, to contribute real advances in the burgeoning field of CAR. Todo: cite other relevant work at Exeter, especially Antony (temporal aspects) and Zoltan (distributed systems aspects). Possibility of Hungarian collaboration with Zoltan for discovery of relevant document collections; uses JINI.

The relevant track records of Dr. Jones, Professor Brown and Dr. Ford are as follows.

Dr. Jones Todo: existing grant, past awards, knowledge, etc.

Professor Brown has a track record of carrying research ideas into successful products. The most successful of these has been the Guide system, which embodied new research ideas in hypertext (this is becoming increasingly relevant to CAR, given Starner's view that location-aware applications are `physically-based hypertext'). Guide was turned into a product by OWL, a small company based in Edinburgh, and sold widely both to personal and corporate customers. In particular it was used by General Motors for their vehicle-bay information systems. Guide was a winner of the BCS Award. A promising candidate for future exploitation is the CAR system recently developed by Jason Pascoe, a research student working under Professor Brown. This system is used in fieldwork data-capture applications, and embodies new ideas in (a) `stick-e notes', (b) in using context and (c) in user interfaces. The system has already been adopted by Earthwatch, an ecology charity. Professor Brown has recently been a partner in a joint enterprise, involving authors from Xerox, Motorola, DARPA, MIT and the University of Oslo, to categorise context-aware applications [Bro01a].

Dr. Ford Dr. Ford is best known for his R and D work as a Principal Consultant for Logica Cambridge, and then for his HCI research when a Senior Lecturer at Exeter University. Since taking early retirement he has worked on various R&D projects that require technical programming expertise in Java at the interface. He now continues to work on an engine and associated software to efficiently deliver context-aware retrieval data to PDAs.

Part 2: Description of proposed research

Background

The idea of context-aware applications was first proposed by researchers from Xerox PARC [Sch94] in 1994. These applications relate to mobile users carrying a PDA that has attached sensors to detect location, nearby equipment, orientation, temperature, etc. The outputs from the sensors, directly or indirectly, represent a current context for each user of an application. An application can use the contextual information to tailor its actions to each specific user. Thus if a sensor detects the user as being in the library, then an application can provide provide information about the library's procedures, and if the user is detected as being near a certain printer, the Print button on their PDA can relate to that printer.

Over the past seven years, a wide range of context-aware applications has been developed. These include guides for tourists and visitors, such as the University of Lancaster's Guide system [Che00], the Sentient Computing work at AT&T Laboratories in Cambridge[Cur99], and the location-based products now being introduced by cellphone service providers (nearest restaurant, local weather). The AT&T work particularly addresses issues of modelling the contextual environment and HCI. This, together with the Georgia Tech work on representing context [Sal99], is useful when designing the fundamentals for our project.

The above projects are not specifically addressed to context-aware retrieval. Probably the largest amount of CAR work performed recently is that of Rhodes and others at MIT [Rho00]. They have built three different CAR systems, and have performed a number of user trials. A key finding is that precision is even more important in CAR than it is in traditional IR. For retrieval they use a hybrid database/IR approach called `fuzzy matching'.

Three developments in hardware are multiplying the future opportunities for context-aware application: (a) the convergence of PDAs and mobile phones; (b) the US Wireless E911 rules [FCC96] requiring that mobile phones be able to report their location, and (c) the increased cheapness and availability of sensors.

In terms of information retrieval, existing CAR applications either have a limited amount of data, such as tourist guides to a single city or attraction, or have a large body of data with identical structure (such as a list of restaurants). Such applications can either use database techniques, or simple Boolean query retrieval techniques. Users, however, could benefit from much larger and diverse sets of data; in addition, in order to tailor information to each user, applications will need to use a much richer context than just location. Overall, therefore, we believe that if the opportunities for context-aware applications are to be realised, much better retrieval methods will be needed. These retrieval methods need to cover both the possible approaches to generating retrieval requests: (a) proactive, where the application triggers information that the user may need (e.g. a document about a church says "trigger me when the user's location is near this church"), and (b) interactive, where requests are initiated by the user. Furthermore the really precise forms of retrieval are likely to be a combination of proactive and interactive (such as a body of triggered information that is selectively retrieved by the user); thus the retrieval methods must allow both these approaches to work in tandem.

Todo: discuss IR and IF research; progress made; relevance to CAR. In CAR the query is derived from the context. Discuss other relevant retrieval systems. Discuss special needs of CAR such as continuous high performance and dynamic data. Why a big challenge.

Programme and methodology

Two potential reactions by users can kill a CAR system: (1) `the application is too slow: by the time the information was delivered the need for it had passed'; (2) `I waste so much time looking at the irrelevant material delivered by this application, that all potential gains are lost'. Thus our first two aims are (1) to give good performance, remembering that information will usually be delivered on a resource-starved PDA; (2) to obtain good precision.

We have specific ideas for a retrieval system to meet the above aims, but this system would, of course, be only part of the overall application that met the user's needs. Therefore, as a third aim, we also wish to pursue more speculative research, suitable for a Ph.D. student, that examines wider issues. These include (a) the interface to the end-user (which can (i) affect the damage that delivery of an irrelevant document does (e.g. Rhodes' ramping interfaces), and (ii) help to guide the sorts of query the user issues); (b) user modelling; (c) trying to exploit future results of the semantic web community, and in particular trying to capture an associated context from an existing textual document; (d) trying to minimise the amount of re-calculation needed when one retrieval uses only a slightly changed context from its predecessor; (e) temporal aspects (time being a key field of almost any context).

Our specific objectives in realising these three aims are, respectively:

  1. to deliver information on a PDA at a speed which a majority of sample users will rate as acceptable, where the document collection consists of at least 10,000 documents, and the user's current context has at least eight separate fields.
  2. while still achieving the first objective, to deliver 30% better precision than a baseline system. Here a baseline system is defined as one that performs straight retrieval, treating each user context separately and thus taking no account of the way the context is changing. The baseline system uses no special algorithms for weighting, extrapolating, etc., from contextual fields. The precision will be judged by sample users.
  3. to publish ideas on research issues on how our retrieval system fits into an overall CAR solution. This ideas will largely lie within aspects (a) to (e) above. (Clearly this objective is more vague, given the more speculative nature of the underlying research that the PhD student will perform.)

To make our research cost-effective we will exploit existing infrastructure where we can. In particular we will ride on top of the World Wide Web -- we envisage that properly web-enabled cellphones/PDAs will be widely available during the lifetime of our project. We plan that the retrieval engine and document collection will reside on the server side, and the user's current context will be generated on the client side. We think this will be an adequate base for the user testing we wish to do. In the longer term, if our work was commercially exploited, rather more function might be built into the client side. The notations we plan to use for representing contexts, etc., can be readily mapped into XML, thus allowing us to exploit the increasing body of tools for XML, XSL, etc. Our implementation will be based throughout on Java, to give good portability and relatively easy interfacing to the web.

In addition we plan to exploit the Context Matcher we have already built. This has been specially designed to allow plug-ins whereby researchers can plug in new algorithms for scoring, weighting, pre-processing, etc., in order to see if performance can be improved. The Context Matcher is not a finished product, and will need further work as our ideas involve, but it is a good initial platform. The Context Matcher assumes that contexts are represented by a set of fields, each of which is a name/value pair. Values are tuples, and can be of several alternative datatypes, e.g. text, number, location.

Our work is aimed to cover applications where the data is not uniformly structured, and indeed may be largely unstructured. Thus our basic technologies are taken from the words of IR and IF rather than from databases.

The methodology for our research programme will be:

  1. in traditional retrieval considerable efforts go towards building relevance sets [[todo right term?]], which give the ideal response in terms of documents delivered to meet a given query. These relevance sets allow builders of retrieval systems to measure how well their systems perform. In CAR we are interested not in a single stand-alone query, but in a sequence of gradually changing queries representing the contexts of a user as they move around. Unfortunately no relevance sets exists for such query sequences. Thus we need to build them -- or an equivalent. To do this we need to study user needs.
  2. we will study user need by talking to experts, such as tour guides and our collaborators (Trilogy has special expertise on mobile workers in the water industry). We will also talk to and `shadow' individual users. In addition we will, where we can, user cheaper and more static methods, where a group of people in a room evaluates what documents are relevant to a certain sequence of contexts. The outcome of this will be some relevance sets, plus some more general information about user needs.
  3. in parallel with the above we will look at algorithms for adjusting the retrieval process in the light of the way the user's context is changing. To provide a solid foundation this work will also look at the underlying data structures for representing objects such as our Context Diary; the Georgia Tech. projects are relevant here. When the above work on user needs is complete we will refine our algorithms and implement them as plug-ins to the Context Matcher. We will also build a test harness round the Context Matcher: this will facilitate supplying sequences of inputs, perhaps using different plug-ins, and comparing the results (e.g. to see how the results are affected by adjusting the relative weight of each context field according to how fast it is changing).
  4. we will test the retrieved system using real data sets, and evaluate the results. There will doubtless be several iterations of refinement -- or even redesign -- of the algorithms, and re-testing.
  5. finally, towards the end of the programme, we plan to test our system first with simulated and then with real user trials. Ideally the real user trials will use web-enabled cellphones/PDAs; however a fall-back position, if technology is not available, if to use stand-alone PDAs.
  6. the research student will work in parallel to the RAs, with a more long-term and wide-ranging brief. We intend, however, that the research student plays a significant part in the study of user needs, and that he/she also provide some plug-ins for the Context Matcher in order to get practical results. The research student will, of course, take part in the University's training programme for research methods.

Our novel ideas that we wish to explore are currently:

  1. creating and using a Context Dairy. This will cover a history of the past values of each contextual field, together with forecast future values. The future values can be derived from a weather forecast, for a temperature field, or a personal diary entry (e.g. appointment at Town Hall at 4pm), for location. The Context Diary can be used for extrapolation (e.g. covering values of sensors that are not working) and forecasting.
  2. anticipating future contexts of the user. This is desirable for two reasons: (a) the user is normally interested in a context "ahead" of their current one; (b) if the retrieval can work in advance of the need, speed will appear to the user to be better. The Context Diary is the basis for anticipation. In case (a), when we formulate a query from the user's current context, we may pretend that the values of contextual fields are slightly ahead of their current values. Companies such as Vodaphone use algorithms for anticipating their users' locations in order to facilitate moving between cells; this work may be relevant to us.
  3. creating a context-aware cache. This is useful if a small change in the user's context will cause a correspondingly small change in the documents retrieved.. If we retrieve documents for a "super-context" covering all the user's possible contexts over the next 10 (say) minutes, and put these documents into a cache, we can then use the cache, rather than the original document collection, as a basis for retrieval. Obviously such caches are especially useful when the user is only periodically connected to the document collection.
  4. weighting contextual fields according to the way they are changing. Our conjecture is that a field that has suddenly changed (e.g. the temperature -- or heartrate -- suddenly rises) is likely to be more important than a static field.
  5. exploring algorithms for giving a score to a match between two fields: thus if a document is associated with a location L1, and the user location is currently L2, should the score be derived from the inverse of the square of the distance between L1 and L2?

The programme will be managed by GJFJ. The three participants already have a track record of working together, and have evolved a series of interfaces whereby their contributions can be joined. This collaborative infrastructure proved itself especially during an unusual 3-month period when LF was in Australia, 12,000 miles from the other two. Generally LF will concentrate on the detailed Java implementation work, PJB will focus on the user's context and its pattern of change, and GJFJ will concentrate on retrieval methods. The research student will, we plan, have GJFJ as his/her main supervisor, with support from PJB and LF. If the project starts early in 2002, as we hope, it is quite likely that the research student will not start until October. Overall we feel that this time offset will bring more advantages than disadvantages, particularly as the research student will benefit from a more refined infrastructure. However, given that research programmes rarely start as soon as the researchers hope, we assume below for simplicity that the research student does start at the same time as the overall programme.

The detailed workplan and milestones are as follows:

Year One
Study user needs, and produce some relevance sets (GJFJ and PJB); design and implement test harness and default plug-in algorithms (LF). Milestone: first tests are run using relevance sets, with results of 10% better precision than a baseline system; these results will be with small and easily manageable data sets, such as those supplied by South West Tourism (10,000-15,000 data items).
Year Two
Concentrated work on algorithms, exploiting what we can from existing IR/IF (all participants), and pioneering innovations based on the needs of CAR. Gather larger data sets, ideally using existing public web materials, and larger relevance sets (all participants). Extend test harness to be ready for our final user testing(LF). Milestone: firm ideas and implementations of algorithms, supported by limited user testing.
Year Three
The firm ideas from Year Two can now be used as the basis for more extensive (and hence expensive in our time) user testing. This in turn will lead to changes (small, we hope) in the algorithms and re-testing. The research student will have ideas to test, perhaps using our overall testing infrastructure, and perhaps, particularly for more specialised ideas, in a more limited, ad hoc, structure. Achievement of overall project goals.

Research issues NOT to be tackled

To avoid the project becoming too diverse there are several research areas that we will not focus on. These include: security and privacy, building distributed systems, agent technology (e.g. for context discovery), structured contexts and levels of abstraction, synthesising high-level context (e.g. that the user is busy) from low-level sensor values, collaborative applications with context-sharing. However, towards the end of the project -- particularly if exploitation is likely -- we may need to look more closely at these areas.

Relevance to beneficiaries

Feed-back from discussions with several parties (Trilogy, Hewlett-Packard, etc.) suggests that 2-5 years from now, suppliers of mobile services will be looking to move on from their current generation of systems to provide high-performance, tailored, high-precision systems, which retrieve from diverse data sources. If the project succeeds we hope that these suppliers will be beating a way to our door.

Dissemination and exploitation

Dissemination will be by the conventional means of papers in journals/conferences and conferences: to reach a wider audience, and to draw communities together, we plan to publish both in retrieval journals and in mobile computing journals/conferences. As regards exploitation, we do not see patents as the best way of protecting our IPR (though this is a changing world, and we may revise our ideas). Experience also shows that existing research-based implementations, such as our Context Matcher and its plug-ins, are not what exploiters want. Instead it is ideas, experience and expertise. Assuming our work is really successful, we would like to see a spin-off company set up; the University of Exeter now provides extensive support for doing this, and for formalising relationships with industrial partners.

Justification of resources

Our largest resource is people, and we plan to use existing named people (Prof. Brown and Dr. Ford). These people are rather more expensive than an average RA -- but not excessively so -- but bring considerable knowledge and expertise, and will not need any time to get up to speed. Todo: justify other resources; conference visits.

References

[Bro01]
Brown, P.J. and Jones, G.J.F. `Context-aware retrieval: exploring a new environment for information retrieval and information filtering', to be published in Personal & Ubiquitous Computing, 2001.
[Bro01a]
Brown, P.J, Burleson, W., Lamming, M., Rahlff, O-W., Romano, G., Scholz, J. and Snowdon, D. Context-awareness: some compelling applications, submitted for publication, http://www.dcs.ex.ac.uk/~pjbrown/papers/acm.html, 2001.
[Che00]
Cheverst, K. Davies, N., Mitchell, K., Friday, A. and Efstratiou, C. `Developing a Context-aware Electronic Tourist Guide: Some Issues and Experiences', Proc. of CHI'00, Netherlands. 17-24 March 2000.
[Cur99]
Curwen, R, Hopper, A., Steggles, P. and Ward, A. Sentient computing, Technical Report 1999.13, AT&T Laboratories, Cambridge, 1999.
[FCC96]
Federal Communications Commission. Revision of the Commission's Rules To Ensure Compatibility with Enhanced 911 Emergency Calling Systems (Wireless E911 Rules), OGC-96-34, August 19, 1996.
[Jon00]
Jones, G.J.F and Brown P.J. `Information access for context-aware applications', Proceedings of ACM SIGIR 2000, Athens, July 2000, pp. 382-4.
[Rho00]
Rhodes, B.J. and Maes, P. `Just-in-time information retrieval agents', IBM System Journal, 39, 4, pp. 685-704, 2000.
[Sal99]
Salber, D. Dey, A.K. and Abowd, G.D. `The context toolkit: aiding the development of context-enabled applications', Proc. Proceedings of CHI'99, Pittsburgh, PA, ACM Press, 1999.
[Sch94]
Schilit, W. N. Adams, N.I. and R. Want R. `Context-aware computing applications', Proceedings of the Workshop on Mobile Computing Systems and Applications, pp. 85-90, Santa Cruz, California, IEEE Computer Society Press, 1994.

Todo: ref to Antony and other Exeter work, ref to Gareth papers.

Part 3: Diagrammatic workplan