Building a Benchmark for Task Progress in Digital Assistants

Publication Year: 2019 Publication Type : JournalArticle


Understanding how tasks progress over time enables digital assistants to help with current activities and support future activities. Imbuing assistants with the ability to track task progress requires machine-learned models. In this paper, we describe an ongoing effort to collect signals from Cyber, Physical, and Social (CPS) activities, together with human assessments of task progression, to serve as a benchmark for training and testing such models. Collecting this data over time is inherently challenging in the daily sensing scenario. Consequently, lessons learned from our ongoing data collection are discussed to provide more insights for future research innovations in task intelligence.


@article{liono2019building, title={Building a benchmark for task progress in digital assistants},
    author={Liono, Jonathan and Trippas, Johanne R and Spina, Damiano and Rahaman, Mohammad S and Ren, Yongli and Salim, Flora D and Sanderson, Mark and Scholer, Falk and White, Ryen W},
    journal={TI@ WSDM19},


Related Publications

RUP: Large Room Utilisation Prediction with carbon dioxide sensor
Type : JournalArticle
Show More
A Scalable Room Occupancy Prediction with Transferable Time Series Decomposition of CO 2 Sensor Data
Type : JournalArticle
Show More
Topical Event Detection on Twitter
Type : ConferenceProceeding
Show More

© 2021 Flora Salim - CRUISE Research Group.