
第一巻『人とAIのインタラクション・ガイド』1.3関連研究(Related Work)
2 RELATED WORK
For over 20 years, the academic community has proposed nu-
merous guidelines and recommendations for how to design
for efective human interaction with AI-infused systems. For
example, Norman [33] and Höök [16] both recommended
building in safeguards like verifcation steps or controlling
levels of autonomy to help prevent unwanted adaptations
or actions from intelligent systems. Others recommended
managing expectations so as not to mislead or frustrate
users during interaction with unpredictable adaptive agents
[16, 20, 33]. Horvitz’s formative paper on mixed-initiative
systems [17] proposed principles for balancing autonomous
actions with direct manipulation constructs, such as support-
ing user-driven invocation of intelligent services, scoping
actions based on inferred goals and confdences, and infer-
ring ideal action in light of costs, benefts, and uncertainties.
The latter guideline was operationalized via the introduc-
tion of a decision-theoretic methodology to guide decisions
about acting on AI inferences versus waiting for user in-
put, based on consideration of expected costs and benefts
of performing AI automation under uncertainty.
In some cases, specifc AI design recommendations have
received considerable attention within the academic commu-
nity. For example, a large body of work exists and continues
to grow around how to increase transparency or explain
the behaviors of AI systems (e.g., [14, 21, 23, 36, 38, 44], to
name a few). Similarly, when and how to automatically adapt
or personalize interfaces has been studied extensively in a
variety of scenarios (e.g., [9, 11–13]).
Others in the community have studied how to design for
specifc human-AI interaction scenarios. For example, re-
searchers have been studying how to efectively interact
with intelligent agents for many years (e.g., [18, 33]). This
scenario has also had a recent resurgence of interest given ad-
vances in natural language processing and embedded devices
driving the proliferation of conversational agents [26, 29, 35].
Similarly, researchers have for decades studied human in-
teraction with intelligent context-aware computing systems
including how to design for understandability and control
of the underlying sensing systems [3, 23] and how to sup-
port ambiguity resolution [10]. Recent advances in sensing
technologies and the widespread availability of commercial
ftness and activity trackers have continued to drive interac-
tion research in these domains [37, 45].
Despite all of this work, the ongoing stream of articles and
editorials in the public domain about how to design in the
face of AI (e.g., [2, 24, 25, 39, 42]) suggests designers need
more guidance. This may be partly due to design suggestions
being scattered throughout diferent academic circles and
venues, making them difcult to fnd (e.g., there is relevant
work in a wide variety of venues including AAAI, UbiComp,
RecSys, SIGIR, HRI, KDD). Moreover, potential design sug-
gestions for AI are often not presented explicitly as such.
In many cases, researchers identify usability issues with AI
systems and suggest possible solutions in the discussion or
future work sections of their academic papers. For example,
Lugar and Sellen [26] identify variability in user expecta-
tions of conversational agents as causing usability issues and
propose setting realistic expectations as a possible solution
in their discussion. Similarly, Lee et al [22] studied automatic
changes to search result lists during user interaction and
suggested caution in updating those lists to balance stability
with presenting new content to users. While these proposed
solutions could be generalized into principles for designers,
not presenting them as such makes them difcult to discover.
It can also be difcult to understand if and how design guid-
ance stemming from one community or interaction scenario
extends to others. For example, Bunt et al. [4] showed that,
while explanations of AI behaviors have shown promise in
complex and high-risk scenarios such as sensor-based ubiq-
uitous computing systems or decision-support systems for
medical or fnancial domains, they may be less important
for relatively low-cost scenarios such as search and music or
movie recommenders.
In this work we 1) synthesize a unifed set of design guide-
lines from a variety of communities and sources and 2)
systematically examine those guidelines in a variety of AI-
infused systems to validate their applicability and relevance.
The closest to our work is Horvitz’s set of principles for
mixed-initiative systems [17], noting that 8 of our 18 guide-
lines map to principles outlined in that work. We celebrate
its 20-year anniversary by refecting on learnings from the
community since its publication. Moreover, recent work has
warned that the lack of rigorous validations of proposed
design heuristics in specifc domains makes it difcult to
gauge the utility of those heuristics [15]. We developed the
guidelines shown in Table 1 using a four-phase process. In
Phase 1 we consolidated more than 150 design recommen-
dations from multiple sources into a set of 20 guidelines. In
Phase 2 we conducted an internal modifed heuristic evalu-
ation of the guidelines, revising the set down to 18. Phase
3 consisted of a user study in which 49 participants used
heuristic evaluation to assess the guidelines’ relevance and
clarity. Based on their feedback, we rephrased some of the
guidelines to improve clarity and, in Phase 4, conducted an
expert evaluation of the revisions to validate the fnal set.
読書メモ
この部分を読んで、私が最も大きな学びを得たのは、その背後にある研究方法でした。
過去20年以上にわたり、ヒューマン・コンピュータ・インタラクション(HCI)、人工知能、推薦システム、インテリジェントエージェントなど、さまざまな分野でAIシステム設計に関する知見や提案が蓄積されてきました。しかし、それらの知識は異なる学術コミュニティや学会に分散して存在し、個別事例の研究や課題の議論、将来への提案という形で示されることが多く、体系的に発見し統合することは容易ではありません。
この論文は、新しい理論を急いで提案するのではなく、まず多くの先行研究から設計上の知見を抽出し、整理し、統合するという基礎的で重要な作業を行っています。そして、それらを複数回の評価を通じて検証・修正しながら、より汎用的な設計ガイドラインへとまとめています。
このことから、学術的な革新は必ずしも新しい発見だけから生まれるものではないことが分かります。ときには、散在している知識を整理し、構造化された枠組みとして示すこと自体が重要な貢献になります。
新しい主張を行う前に、まず既存研究が何を明らかにし、何を解決し、そして何を見落としているのかを理解する必要があります。既存の知識体系を十分に理解してこそ、本当に探究する価値のある空白領域をより正確に見つけることができるのだと思います。
私にとって、この論文が提供しているのはAI設計に関する知識だけではありません。そこには、一つの学ぶべき研究方法があります。
まず整理すること。
次に統合すること。
そして理解した上で、新しい問いを立てること。
その順序の大切さを改めて学びました。