Deliverables of the third year:




DR 3.3: Second set of prototypes of user model and action selection algorithms; fi rst evaluation.

This document describes the second set of prototypes of the PAL user modeling algorithms and action selection that have been developed during the third year of the project in PAL’s work package 3 (WP3). The overall objective of WP3 is to adapt the behavior of the PAL system to each of its users.


DR 5.3: Coherent User Experience Recordings

In this document we report on the work in work package 5 that relates to the recording, structuring and use of user experiences with the PAL system. This deliverable focusses on those aspects of the PAL system that relate to the development of what can be called an artificial episodic memory for the PAL agent, in order to personalize and adapt its functioning to the user, as well as provide a coherent service over time and location (hospital and at home). Parts of the work summarized in this deliverable have already been reported upon in other deliverables (most notable the deliverables from work package 4 and work package 2) as these work packages focus on the strategic use of the data (WP 2) and the data storage format and formalism (WP 4). As such this deliverable is kept succinct on purpose as to not repeat information that has been given already. We focus here on novel developments, and on a coherent view of the activities that have been undertaken in the last period to leverage the data that now can be stored and structured.


DRs 6.3, 6.7, 6.11: Y3 Dissemination ReportDR 1.3:

The present report describes the work carried out during the third project year regarding PAL’s Dissemination activities. It is the summary of three different WP6 Deliverables: Deliverable 6.3 “Website y3”, Deliverable 6.7 “Publications and proceedings report y3” and Deliverable 6.11 “Dissemination events promoted y3”.


Deliverables of the second year:





DR 1.3: Evaluation of PAL prototype supporting knowledge and awareness:

This report presents the work WP1 with a focus on T1.4: The results from the first design and evaluation cycle. This cycle contained a number of related experiments (i.e., formative evaluations of PAL prototypes). Each experiment had specific research questions concerning PAL functionality and expected outcomes (claims) concerning (1) child’s knowledge, awareness, attitude (towards PAL and T1DM), self-efficacy and skills (usage and adherence), or (2), HCP’s trust and acceptance, or (3) parent’s attitude and knowledge.


Mind map of diabetes type1 in children

DR 2.2: Context and engagement-based strategy selection:

In this document we report on the work in work package 2 that relates to context, engagement, and explanation of robot actions. Context and engagement directly relate to this deliverable, explanation of robot action relates to work needed for task 2.1 to be able to implement multi-user explanation of the actions proposed by the system. We have focused on several fundamental issues in this year. First, can children perceive differences in robot educational style? This is important for engagement because human teachers use different styles and the ability to adapt educational style to an individual increases educational efficiency and effectiveness. Second, we have evaluated (with children and healthcare professionals) the PAL Control tool developed last year. PAL Control is our health care professional authoring tool. Again this is important for engagement and also for contextualized goal setting. Being able to set learning goals appropriately for children contributes to focusing on an achievable set of goals and contributes to structuring the learning process. Goal setting has been integrated with the quiz module of the PAL system. This enables the system to propose educational quizzes dependent on the context as defined
by the learning goals. Third, we have shown in an experiment at a diabetes autumn camp that children and adults prefer robot action explanations differently. This is important fundamental knowledge enabling us (in the following years) to define how the agent should explain its strategy to the children, parents, and healthcare professionals.

DR 3.2: First prototype of user model construction algorithms featuring input from sentiment and interaction mining:

This document describes the first prototype of the PAL user modeling algorithm based on sentiment and interaction developed during the second year of the project in work package 3. The overall objective of workpackage 3 is to adapt the behavior of the PAL system to each of its users. This adaptation is necessary to ensure 1) engagement of the user and 2) increased effectiveness in personal goal achievement. People tend to adapt their interaction style in a conversation based on observations such as the perceived sentiment of the conversation partner. For this purpose it is worthwhile to include sentiment in the interaction process, such that the robot or its avatar can adapt its behavior accordingly. Sentiment extracted from the child’s diaries can assist in estimating the current well-being and emotional state of the child. Moreover, sentiment can be used as feedback mechanism to tune the user model.

Success rate predictions based on raw interaction information

DR 4.2: Multimodal Interaction with Basic Adaptivity and Personalization:

The present report describes the work carried out in the second project year regarding Natural Multimodal Interaction. It summarises the Deliverable D4.2: \Natural Multimodal Interaction Basic Adaptivity and Personalisation”. The formalism to implement dialogue strategies that was defined in the first year has been implemented to a great extend, and the functionality contained in the year 1 prototype has been implemented in the new formalism to prove the feasibility of the approach. On the natural language generation side, we implemented basic adaptivity and personalisation, using the data contained in the underlying knowledge source to adapt the conversational style, as well as the content, to the current user. Two studies concerning personalised interaction were conducted. The first concerns the relation of the child’s disclosure depending on self-disclosure
of the robotic agent, while the seconds develops strategies how to react to the child’s behaviour patterns in using the timeline.

DR 5.2: Robot and avatar behavior implementation:

In this document we report on the work done in WP5 as part of deliverable D5.2, in particular task 5.2. Task 5.2 focuses on actual implementation of behaviors (e.g., the actual specification of the gestures) on the physical and virtual NAO.

The relations between de various components of the PAL system

DRs 6.2, 6.6, 6.10: Y2 Dissemination Report:

The present report describes the work carried out in the second project year regarding PAL’s Dissemination activities. It is the summary of three different WP6 Deliverables: Deliverable 6.2 “Website y2”, Deliverable 6.6 “Publications and proceedings report y2” and Deliverable 6.10 “Dissemination events promoted y2”.



Deliverables of the first year:




DR1.2 – Design and ontology of PAL prototype supporting knowledge and awareness:

This report presents the first user needs analyses, designs and evaluations of workpackage 1 (WP1) of the PAL project.


DR2.1 – PAL Control: a prototype authoring tool interface for entering coaching protocols:

DR2.1 focuses on the development of PAL Control, an authoring tool for care professionals. This tool enables health care professionals (HCP) to set learning goals for children during meetings.


Model for management of pedagogical interaction style depicting factors (user, goal, location, users and activity) in uencing selection of role and communication style, that together determine which pedagogical interaction style is chosen, which is then expressed in robot behaviour using, for example, prosody, speech and posture. (con- textual factors and behaviours are included for illustration purposes only.)


DR3.1 – Personalized adaptive action selection Y1:

This document reports the work achieved during the first year of the PAL project on Personalized adaptive action selection work package (WP3), and in particular on the fi rst prototype of the action selection computational algorithms.

Overview of the HAMMER architecture


DR4.1 – Natural Multimodal Interaction Baseline:

We defined a new formalism for the speci cation of dialogue policies that combines dialogue rules, knowledge representation and dialogue history in a unique way. We developed the first version of an ontology which specifies the data structures to be used by the dialogue specifications, dialogue history, and information state, and adapted our reasoning components, so that this knowledge source can be used efficiently once the formalism specification is fully implemented. We implemented a prototype for the experiments in year 1, which provide us with interaction data that can be used later to develop and evaluate the components in this work package.

Example: Punctuation marks give the TTS system important guidelines for the utterance prosody and intonation.

DR5.1 – PAL Technical Architecture and Software Architecture:

This document represents the first deliverable of WP5 and describes the PAL architecture of the IT infrastructure and of its main software modules.

PAL’s IT Architecture

DR6.1 – Y1 Dissemination Report:

The present report describes the work carried out in the first project year regarding PAL’s dissemination activities; website, publications, dissemination.

The PAL project Dissemination time line.