SUPERASSIST: PERSONAL ASSISTANTS FOR COOPERATIVE HEALTHCARE TREATMENT
Geert de Haan (1), Charles A.P.G. van der Mast (1), Olivier Blanson Henkemans (1),(2) and Mark A. Neerincx (1),(2)
(1) Unit of Man-Machine Interaction
Department of Mediamatics, Delft University of Technology, Mekelweg 4, 2628 CD,
Delft, the Netherlands
(2) TNO Human Factors Research Institute
P.O. Box 23, 3769 ZG,
Soesterberg, the Netherlands
KEYWORDS
Human-computer interaction, tele-medicine, supervision, personalization, ambient intelligence, health care, personal assistants, cooperative problem-solving.
ABSTRACT
This paper describes the research plans in the SuperAssist project, introducing personal assistants in the care of diabetes patients, assisting the patients themselves, the medical specialists looking after the patients' healthcare, and the technical specialists responsible for maintaining the health of the devices involved. The paper discusses the issues of trust and cooperation and the proposed scenario-based approach to design a future of agent-based healthcare to increase the self-help abilities of individual patients.
INTRODUCTION
TNO Human Factors (TNO), Delft University of Technology - Man-Machine Interaction group (TUDelft) and Leiden University Medical Centre - Medical Informatics (LUMC) are developing guidelines, models and methods for joint user-"electronic assistant" supervision of critical equipment and information.
The aim is to establish effective and efficient distributed supervision of networked information compilations and technical equipment, which is trustworthy for the user and takes place in a socially approved manner. Specific innovative project results are: communication and interaction model for these assistants; methods for joint human-computer supervision; improved test methods, tools and criteria for systematic assessment of user experience; "best practice" implementation-method and guidelines; and a "proof of concept" in the transmural health care domain (e.g. diabetes).
For the medical application domain, the SuperAssist framework aims to reduce the costs by improving the local, self-care capacity of people by efficient employment of remote, distributed expertise. Figure 1 provides a possible setting for the SuperAssist framework in which both, a patient, the medical specialists who treat the patient, and a technical specialist who looks after the proper functioning of the medical equipment involved are all provided with personal assistants to facilitate their own tasks as well as to support cooperative tasks which might occur, for instance, with the introduction of new drugs and therapies or with the inevitable malfunctioning or erroneous use of the medical equipment.
Fig. 1. Example situation in which all parties involved (the medical specialists, the medical dossier, the patient, and the technical specialists) are supported by personal agents
The project's business partners - Science & Technology, Philips Research, Pemstar and Sigmax PDA Solutions - bring in their technology and contribute to the development and validation of SuperAssist elements. The joint activities are included in cognitive engineering cycles, in which the foundation, specification and demos of the SuperAssist concept are being refined and extended. The SuperAssist research takes place in the medical domain, but it aims at a generic solution for the distributed supervision of complex environments.
A USER SCENARIO
Ms. Brown is a vital 72 year old who, after loosing her husband several years ago, refound her balance and takes a keen interest in bicycling and reading. Two years ago she was diagnosed with type II diabetes for which a diet and medication were prescribed. Although she is sincere in following up the medical directions, sometimes her memory fails her and she simply forgets things like when she has last taken her pills or that she should only buy low glucose products.
Since ms. Brown is well overweight, she has been asked to participate in a therapeutic program whose purpose is to attempt to let participants lose some of that weight in a responsible yet playful manner. As a result, every day before breakfast and before bedtime Ms. Brown consults her personal diabetes assistant, provided to her as part of the program, to check her blood glucose levels and to help her remind whether she has taken her pills or not. To address her deteriorating eyesight, the assistant is equipped with an extra-large extra-bright screen, a few large and clearly labelled buttons. In addition, the form and content of the dialogue have been adapted for ms. Brown according to her preferences and to the way she uses the diabetes assistant.
On three consecutive days, ms. Brown's blood glucose level has been slightly higher than normal and today it is rather high. An abnormal reading is not necessarily a cause for alarm. One possibility is that there is nothing wrong with ms. Brown but rather that the technique let her down. The reading of the blood glucose level is derived from the light passing through the finger that ms. Brown offered for testing, which is much more comfortable but also less reliable then blood sampling. Ms. Brown may call the district-nurse to do blood tests but she prefers not to, because he is busy enough with the "really sick people". Finally, the measurement might be genuine. It may be that ms. Brown has forgotten to take her night time pills. However, the display of the assistant clearly shows that she has confirmed taking her pills until this morning and besides, the compartments of her pill-box are empty. It may be -again- that she has accidentally taken a regular instead of a low sugar desert.
A little alarmed, ms. Brown at first doesn't know what to do but then she sees the help-button on the diabetes assistant and remembers from the instruction session that "if you would like to ask a question, first try the help-button". This she does and a friendly voice assures her that there is only a little worry but not an acute problem. Furthermore, the assistant suggests her to redo the measurement, this time using the little finger of her other hand. Using her other little finger, ms. Brown now learns that her blood glucose level is only slightly higher than normal and her assistant asks her to take her pills, including an extra TZD "you know, the big blue one" just to be on the safe side.
Behind the Scenario
By this time, also behind the scenes some activity has taken place. Because of the measurement error, a maintenance module inside the assistant increased the problem count of the measurement module, reset it and performed a self evaluation test in order to exclude a range of technical problems. Also a temporary wireless network is set-up for a number of routine and problem reports. When the assistants' problem count exceeds a certain level of serious measurement or transmission problems, a technical expert is informed for further maintenance or for a replacement of the device.
In addition, when serious problems occur, the district nurse will be informed of the existence of a possibly faulty device, so that he can check out to take over or provide a replacement diabetes assistant. In this case, the measurements remained within the safety boundaries, there was no need to alarm the specialist diabetes team, although, as a matter of routine, both blood glucose readings and the measurement problem are fed into ms. Browns medical dossier.
Finally, because there has been a change in ms. Brown's stable blood glucose level, a reminder is send to the district nurse. The personal patient-visiting assistant agent will remind the nurse the next time he goes on a periodical visit to ms. Brown. Even though the diabetes assistant is "allowed" within certain boundaries to change a patient's medication, only medical specialists (or in their place the district nurses) are allowed to make any enduring or substantial changes.
Diabetes Type II
Diabetes type II or "old-age diabetes", occurs when the body doesn't use the available insulin effectively. Because of the low insulin level, the blood glucose level may build up, causing a sick feeling. Type II diabetes is generally caused by a life-long combination of a bad diet, lack of exercise and overweight. The treatment of diabetes II is focused on keeping a healthy blood-glucose level which should neither be too high not too low. If the glucose level drops too low, there is the possibility of passing out, coma and eventually death. A minority of diabetes II patients has to inject themselves with insulin (about 15%) but for most patients the treatment consists of a diet, in combination with medicines to boost the production and the effectiveness of insulin.
Diabetes II is not deadly by itself but its symptoms, in the form of a deterioration of small blood vessels may lead to blindness, numbness, inflammations, etc. which may eventually cause death, although, in principle, diabetes II is not difficult to treat.
CRITICAL ISSUES IN COOPERATIVE HEALTHCARE
In the SuperAssist project we intent to follow a scenario-based approach (Carroll 1995) in that the project consists of a number of phases, each of which is approached iteratively, and guided by hypothetical and real scenarios of would-be situations (Rizzo and Bacigalupo 2004). In dealing with the novel situation of having multiple types of users communicate and cooperate with multiple types of agents, the domain is relatively new and, according to Erickson (1995), a scenario-based approach is the preferred method. In addition to such standard HCI practices, it is a "sine qua non" that users should participate in the development of a personal service or consumer product that SuperAssist aims at (Kemp and van Gelderen 1996).
Even though the project targets at a novel type of application in a new area of multi-user multi-agent (MUMA) interaction, it does not start from scratch. Apart from being concerned with software agents which assist human beings in the area of ubiquitous computing (Maes 1998; Weiser 1991), SuperAssist attempts to build further upon the PALS project, a Dutch project investigating the requirements to meet changing user needs and usage contexts with respect to web-services (see: PALS; Herder 2003; Neerincx and Streefkerk 2003).
From this and other projects, two factors turned out to be of critical importance for the success of human-agent interaction: trust and cooperation. Trust in the sense that people need a basic level of trust in a system or personal assistant to "do business" with them, and cooperation in the sense that, like in human-human cooperation, participants rely on images, models or ideas about the communication and cooperation abilities of their discussion partners in order to set appropriate expectations.
Trust
The trust of a patient, a medical specialist or a nurse in their personal assistant is a prerequisite to delegate part of their responsibilities to the electronic device. To enable trust and delegation between people and electronic assistants, it is a first necessity that the form and content of the interaction dialogue are acceptable to the human user, including a possibly frightened patient or an excessively busy medical worker.
Here, it may be possible to adapt the dialogue to the current context of use including aspects like fear and stress (Neerincx and Streefkerk 2003).
Apart from a humanly usable dialogue for solving problems, it is necessary that there is a shared view on the problem domain. This may be achieved by creating a representation of the problem space in which rules exist or may be derived to guide the behaviour of the assistants. Note that, even in a well-delimited problem domain as diabetes, this is a major undertaking due to influences from outside the system, such as the nurses' other responsibilities or the complex (side) effects between things like medication, stress and diet.
Cooperation
Cooperation for problem solving between people and their assistants and the cooperation between assistants among themselves requires some kind of model which at least describes 'what to expect from whom' in terms of questions, actions, etc. In (software) multi-agent architectures like e.g. FIPA (see: FIPA) the question 'what to expect from whom' is specified in agent-communication models, either by way of static models, specified beforehand, or in on-request models, used in dynamically changing environments.
Also at a much higher level, the concept of cooperation is important. For example, in the discussions about the introduction of an electronic medical passport and in several projects in which the cooperation between medical staff and patients goes beyond exchanging complaints, pills and advise. In the Telemedicine-Diabetes project at the University of Oxford (Tarassenko et al. 2004) for example, diabetes patients do not merely monitor their blood glucose level to treat themselves, but by additionally phoning-in the measurement results to a diabetes treatment centre, these can be stored in a database and used for personal advice.
From these and related investigations, a general "lesson learned" is that patients are generally able and willing to take participate in such mutual medical specialist-patient cooperation project, but only if the personal benefits to the patient are clear, and when advice and treatment are personalised to the patients context and circumstances. As a result, a main research idea in the SuperAssist project is to use a PDA-like device to present diabetes patients with an existing (work)book "the diabetes manual" (Stewart 2004), to be translated in Dutch) in a personalised, adaptable and context-sensitive way to help patients cope with their disease and the often associated overweight.
SCENARIOS, PROTOTYPES AND DESIGN
Within SuperAssist, scenario-based design (Carroll 1995) will be the main technique to develop research ideas in the project. Scenario-based design will be used in combination with prototyping as the projects main technique to test the applicability and the usefulness of the ideas in more-or-less tangible form. In combination, scenario's and prototypes and mock-ups form cognitive engineering cycles in which they are connected by different levels of specification. Since the project is in an early stage, the main focus is currently on the use of scenarios for two different purposes.
First, scenarios are used to collect 'wild ideas' to envision the outcomes of the SuperAssist project in the form of integrated real-world situations. Scenario-envisioning is thus used to provide some -however idiosyncratic and erroneous- idea of what the project is going to deliver eventually.
In addition, by doing so, the scenarios also provide a means to create a common as well as shared understanding among the project members. The first scenario that was discussed in this paper and figure 1 present the use of scenario's for envisioning purposes.
Cognitive Engineering and Scenario-based Design
Research and design in the SuperAssist project will make use of cognitive engineering cycles, an engineering approach to HCI design with a focus on the human abilities and limitations as the most critical of all design aspects. In relation to other HCI design approaches this means that design is user-centred, involves user participation and takes the need for design iterations for granted. This is not all too different from the common HCI approach to design, except perhaps that an engineering stance is taken in that both, a particular problem has to be solved and that the solution should be systematically applicable to a whole class of similar problems.
The cognitive engineering framework in one in which practical theories, communication and interaction models, and prototypes are being developed in a structured iterative process (Lindenberg, Nagata and Neerincx 2003). A practical theory includes accepted features of cognition and affection, distinguishes different task contexts within a specific application domain, provides descriptions of computer behavior, and predicts the interaction effects between these features, contexts and computer behavior. An interaction model formalizes and quantifies these interaction effects and is used to design a prototype of an "intelligent" user interface, which adapts its behavior to the present goals state, and to the cognitive and affective processes of the user. In addition to the individual user-assistant cooperation model, proposed in Lindenberg et al., the SuperAssist project will develop a co-operation model for the overall, multi-actor supervision, which formalizes the information exchange between all of the actors involved, both agents and men.
The research in SuperAssist is further organized around three main experiments or rather three series of experiments to investigate the main research questions in the project: task allocation among agents and men, the organization of joint anomaly management or problem solving, and finally, the "look & feel" of the assistants, respectively. In addition, a number of flanking experiments and scenario exercises is planned, intended to delimit the design space for the main experiments by excluding in an early stage the less- or irrelevant factors and to find solutions for design details for which a full-size experimental setup is overkill.
Apart from empirical validation of models and theories, the three experiments also serve to validate and improve the design of the prototypes which are designed to run these experiments. The prototypes are used as vehicles to develop both the required knowledge about human-agent cooperation as well as drive the actual design of the specific solution to the problem at hand. As such, in the course of the project, the design of the prototypes being tested changes and becomes more refined and detailed. Whereas the initial experiments will take place using mock-ups, at the end of the project the aim is to present and test a demonstrator in the form of a prototype. It should be noted, however, that the primary aim of SuperAssist is to acquire and validate knowledge to design MUMA applications. As such, on the one hand, the creation of mock-ups, demonstrators and prototypes is a vehicle to facilitate research; not a 'deliverable' as in many EC research projects. On the other hand, the prototypes play a dual role, to validate the underlying theoretical knowledge as well as that they themselves are tested as solutions to the specific design problem.
Scenarios in SuperAssist
How scenario's are used in each stage of design and investigating is exemplified by Rizzo and Bacigalupo (2004) who describe a full range of using scenario's from early design stages, like familiarization and discovery of the main issues to later stages involving validating of ideas and design proposals with stakeholders, creating specifications for design and implementation, and validation of design solutions. According to Rizzo, particularly in new design contexts, it is important not to get lost in the false security of formal specifications. Instead they propose to use scenarios throughout design for envisioning, specification and validation.
In the early stages, the design process starts with observing the relevant aspects of the 'old' design situation, laid down in a number of scenarios or storyboards. Such descriptive scenario's are subsequently used to discuss and develop a common view with stakeholders what the essential problems and principled solutions may be. Note that this stage, like any of the following may require a number of iterations to arrive at a suitable set of working scenarios. In the project, interviews and observation sessions are currently held with both support engineers, patients and medical personal.
Next, the chosen scenario's are further analyzed to derive a number of so-called micro-scenarios in which for each part of a scenario, a number of essential elements is identified. This step is most similar to classical task-analysis but instead of aiming at fully describing the stakeholders' tasks, the aim is to arrive at a level of description which allows one to develop and feed-back or validate partial design solutions. The analysis step mostly involves desk work, but to evaluate the feasibility of each of the partial design proposals required cooperation with the stakeholders who actively have to decide which solutions will in practice make sense and which do not. It is at this step that the experiment about task-allocation will be planned and executed.
After analysis follows a synthetic step in which a number of design solutions is taken together as proposals to solve the underlying design problem. The results of this step are called macro-scenarios which describe a number newly proposed task-scenarios in combination together with the supporting technology. Describing design proposals in scenario form rather then in some formal way, makes it easier for stakeholders to be involved. Also here, stakeholders involvement is required, not as individual stakeholders but rather as a community of stakeholders. In such planned feedback sessions, including the planned experiment on joint human-agent problem solving, the role of stakeholders is both to evaluate and select (some) overall design solutions as well as to propose improvements to the selected proposals.
Finally, after the subsequent creation and implementation of one or more of the proposed design solutions, one or more concluding evaluation sessions are held, guided high-level scenario's from the early stages, to validate the overall design solutions and, where necessary, to improve them at a more detailed level. This step results in some form of a final design solution which can be evaluated in the last experiment, concerning the "look & feel" of the design.
It may be clear from the preceding description that scenario's seem particularly useful vehicles to enable user- or stakeholder participation during all the design steps. In addition, scenario-based design approaches also seem particularly useful in situations in which design does not merely involve the creation of a computer application but rather the design of solutions which involve both men and machines -or rather- men, agents and machines with each specific roles and responsibilities to fulfil. What is not yet clear at this stage of the project is how exactly to fit in the planned experiments into a scenario-based approach and to establish a 'best practice' for experiment-backed scenario-based design.
RESULTS AND OUTLOOK
The SuperAssist project attempts to set up an integrated healthcare service in the area of diabetes treatment, assisted by electronic devices and software agents. From this research we expect to be able to derive models, architectures, guidelines and general design knowledge to introduce human-agent systems that enable people to help themselves to a greater extend than they are able to do, now. As such, the SuperAssist case is intended as a test case for a range or healthcare and other services and not just as yet another smart solution for a particular problem.
Only by introducing, testing and validating general tools, we will be able to help keep future healthcare costs within the boundaries of affordability and manageability - at the very least for individual patients like ms. Brown and ms. Blair.
The SuperAssist project also employs diabetes as a vehicle to learn general lessons about how to introduce trustworthy and cooperative electronic or software assistants into the everyday life of real people. Until recently, human-agent interaction has been studied in settings where a single agent assisted a single user, as in supporting searching, information filtering, and critiquing systems. In the SuperAssist project, research and development are extended to MUMA or multi-user multi-agent communication and cooperative problem solving. This novel research area does not only introduce new types of questions and new types of problems but it is also a starting point for the integration of separate intelligent applications into intelligent overall services.
REFERENCES
Carroll, J.M. 1995. Scenario-based Design: Envisioning Work and Technology in System Development. John Wiley, New York.
Erickson, T. 1995. "Notes on Design Practice: Stories and Prototypes as Catalysts for Communication". In Scenario-Based Design: Envisioning Work and Technology in System development, 1995, J.M. Carroll (Ed.). John Wiley, New York, 37-58.
FIPA. The Foundation for Intelligent Physical Agents. See: www.fipa.org
Herder, E. 2003. "Utility-Based Evaluation of Adaptive Systems". In Proceedings of the 2nd Workshop on Empirical Evaluation of Adaptive Systems (Johnstown, PA), 25-30.
Kemp, J.A.M. and van Gelderen, T. 1996. "Co-discovery Exploration: an informal method for the iterative design of consumer products". In Usability Evaluation in Industry, 1996, P.W. Jordan, B. Thomas, B.A. Weerdmeester and I.L. McClelland (Eds.). Taylor and Francis, 139-146.
Lindenberg, J., Nagata, S.F. and Neerincx, M.A. 2003. "Personal Assistant for onLine Services: Addressing human factors". In Human-Centred Computing: Cognitive, Social and Ergonomic Aspect, 2003, D. Harris, V. Duffy, M. Smith & C. Stephanides (Eds.). Erlbaum, London, UK, 497-501.
Maes, P. 1998. "Reflections on ... Agents that Reduce Work and Information Overload". In Readings in Intelligent User Interfaces, 1998, M.T. Maybury and W. Wahlster (Eds.). Morgan Kaufmann, 525-536.
Neerincx, M.A. and Streefkerk, J.W. 2003. "Interacting in Desktop and Mobile Context: Emotion, Trust and Task Performance". In Proceedings of the first European Symposium on Ambient Intelligence - EUSAI (Eindhoven, The Netherlands). Springer-Verlag.
PALS. Personal Assistant for onLine Services. IOP/MMI project website; see http://www.tm.tno.nl/pals
Rizzo, A and Bacigalupo, M. 2004. Report on testing of full scale scenarios. Deliverable 3.2 of CARE/CREA Project. Available from: www.dblue.it/pdf/crea/CARE as CREA_UNISI_WP3_D3.2_v1.1.pdf
Stewart, R. 2004 (Ed.). The Diabetes Manual. Warwick Diabetes Care, University of Warwick, UK.
Tarassenko, L., Gibson, O. J., Hayton, P. M., Cobern, W. R., Farmer, A. J., Hannaby, K., Dudley, C. and Neil, A. 2004. "Mobile Phone Technology to Support the Self-management of Diabetes". In Proceedings of the Diabetes UK Annual Professional Conference (Birmingham, UK, March 2004), 17-19.
Weiser, M. 1991. "The Computer for the 21st Century". Scientific American 265, No.3, 94-104.
BIBLIOGRAPHY
GEERT DE HAAN studied Cognitive Psychology at the University of Leiden and has a PHD in Mathematics and Computer Science from the Vrije Universiteit Amsterdam. After a number of projects in which he worked on formal design models, usability evaluation, scenario-based design in Digital Libraries and Ubiquitous Computing, he is currently employed at the man-Machine Interaction Group at the Department of Electronic Engineering, Mathematics and Computer Science at Delft University of Technology, as a Postdoc on the SuperAssist project. His main interest is in design methods for, among others, personalization and ubiquitous computing.
CHARLES VAN DER MAST has a PHD from Mathematics and Computer Science from Delft University of Technology where he is employed at the Man-Machine Interaction group at the Department of Electronic Engineering, Mathematics and Computer Science as a Associate Professor. He teaches Multimodal Interfaces and Virtual Reality, Developing Highly Interactive Systems, Multimedia, and Usability Engineering, and developed the Bachelor/Master curriculum in Media & Knowledge Engineering. His interests include using various media to improve teaching, VR therapy for phobia treatment.
OLIVIER BLANSON HENKEMANS studied Business Communication and Media at Tilburg University and worked on website development and Augmented Multiparty Interaction. He is currently pursuing a PHD at both TNO Human Factors and the Man-Machine Interaction group at the Department of Electronic Engineering, Mathematics and Computer Science at Delft University of Technology.
MARK NEERINCX studied Cognitive Psychology at the University of Leiden and has a PHD in Cognitive Psychology from the University of Groningen. He worked at the TNO Institute of Preventive Health Care, the University of Leiden and the University of Amsterdam, after which he became research scientist at TNO Human Factors and heads the Usability Engineering group. He is also parttime professor in the Man-Machine Interaction group at the Department of Electronic Engineering, Mathematics and Computer Science at Delft University of Technology. His research interests include usability engineering, human-computer interaction, cognitive task analysis, mental load, ambient intelligence, personalization, electronic partners, adaptive interfaces and multi-modality.
This is a draft version of the paper: de Haan, G., van der Mast, C.A.P.G., Blanson Henkemans. O.A. and Neerincx, M.A. (2005). SUPERASSIST: Personal Assistants for Cooperative Healthcare Treatment. In: Al-Akaidi, M. and Rothkrantz, L. (eds.), proceedings Euromedia 2005, EUROSIS publication, pp. 124-128. 11-13 april 2005, Toulouse, France. Presented by Charles van der Mast.