メインコンテンツへスキップ
見出し画像

第一巻『人とAIのインタラクション・ガイド』1.3関連研究(Related Work)

    2 RELATED WORK

    For over 20 years, the academic community has proposed nu-

    merous guidelines and recommendations for how to design

    for efective human interaction with AI-infused systems. For

    example, Norman [33] and Höök [16] both recommended

    building in safeguards like verifcation steps or controlling

    levels of autonomy to help prevent unwanted adaptations

    or actions from intelligent systems. Others recommended

    managing expectations so as not to mislead or frustrate

    users during interaction with unpredictable adaptive agents

    [16, 20, 33]. Horvitz’s formative paper on mixed-initiative

    systems [17] proposed principles for balancing autonomous

    actions with direct manipulation constructs, such as support-

    ing user-driven invocation of intelligent services, scoping

    actions based on inferred goals and confdences, and infer-

    ring ideal action in light of costs, benefts, and uncertainties.

    The latter guideline was operationalized via the introduc-

    tion of a decision-theoretic methodology to guide decisions

    about acting on AI inferences versus waiting for user in-

    put, based on consideration of expected costs and benefts

    of performing AI automation under uncertainty.

    In some cases, specifc AI design recommendations have

    received considerable attention within the academic commu-

    nity. For example, a large body of work exists and continues

    to grow around how to increase transparency or explain

    the behaviors of AI systems (e.g., [14, 21, 23, 36, 38, 44], to

    name a few). Similarly, when and how to automatically adapt

    or personalize interfaces has been studied extensively in a

    variety of scenarios (e.g., [9, 11–13]).

    Others in the community have studied how to design for

    specifc human-AI interaction scenarios. For example, re-

    searchers have been studying how to efectively interact

    with intelligent agents for many years (e.g., [18, 33]). This

    scenario has also had a recent resurgence of interest given ad-

    vances in natural language processing and embedded devices

    driving the proliferation of conversational agents [26, 29, 35].

    Similarly, researchers have for decades studied human in-

    teraction with intelligent context-aware computing systems

    including how to design for understandability and control

    of the underlying sensing systems [3, 23] and how to sup-

    port ambiguity resolution [10]. Recent advances in sensing

    technologies and the widespread availability of commercial

    ftness and activity trackers have continued to drive interac-

    tion research in these domains [37, 45].

    Despite all of this work, the ongoing stream of articles and

    editorials in the public domain about how to design in the

    face of AI (e.g., [2, 24, 25, 39, 42]) suggests designers need

    more guidance. This may be partly due to design suggestions
    being scattered throughout diferent academic circles and

    venues, making them difcult to fnd (e.g., there is relevant

    work in a wide variety of venues including AAAI, UbiComp,

    RecSys, SIGIR, HRI, KDD). Moreover, potential design sug-

    gestions for AI are often not presented explicitly as such.

    In many cases, researchers identify usability issues with AI

    systems and suggest possible solutions in the discussion or

    future work sections of their academic papers. For example,

    Lugar and Sellen [26] identify variability in user expecta-

    tions of conversational agents as causing usability issues and

    propose setting realistic expectations as a possible solution

    in their discussion. Similarly, Lee et al [22] studied automatic

    changes to search result lists during user interaction and

    suggested caution in updating those lists to balance stability

    with presenting new content to users. While these proposed

    solutions could be generalized into principles for designers,

    not presenting them as such makes them difcult to discover.

    It can also be difcult to understand if and how design guid-

    ance stemming from one community or interaction scenario

    extends to others. For example, Bunt et al. [4] showed that,

    while explanations of AI behaviors have shown promise in

    complex and high-risk scenarios such as sensor-based ubiq-

    uitous computing systems or decision-support systems for

    medical or fnancial domains, they may be less important

    for relatively low-cost scenarios such as search and music or

    movie recommenders.

    In this work we 1) synthesize a unifed set of design guide-

    lines from a variety of communities and sources and 2)

    systematically examine those guidelines in a variety of AI-

    infused systems to validate their applicability and relevance.

    The closest to our work is Horvitz’s set of principles for

    mixed-initiative systems [17], noting that 8 of our 18 guide-

    lines map to principles outlined in that work. We celebrate

    its 20-year anniversary by refecting on learnings from the

    community since its publication. Moreover, recent work has

    warned that the lack of rigorous validations of proposed

    design heuristics in specifc domains makes it difcult to

    gauge the utility of those heuristics [15]. We developed the

    guidelines shown in Table 1 using a four-phase process. In

    Phase 1 we consolidated more than 150 design recommen-

    dations from multiple sources into a set of 20 guidelines. In

    Phase 2 we conducted an internal modifed heuristic evalu-

    ation of the guidelines, revising the set down to 18. Phase

    3 consisted of a user study in which 49 participants used

    heuristic evaluation to assess the guidelines’ relevance and

    clarity. Based on their feedback, we rephrased some of the

    guidelines to improve clarity and, in Phase 4, conducted an

    expert evaluation of the revisions to validate the fnal set.

    読書メモ

    この部分を読んで、私が最も大きな学びを得たのは、その背後にある研究方法でした。

    過去20年以上にわたり、ヒューマン・コンピュータ・インタラクション(HCI)、人工知能、推薦システム、インテリジェントエージェントなど、さまざまな分野でAIシステム設計に関する知見や提案が蓄積されてきました。しかし、それらの知識は異なる学術コミュニティや学会に分散して存在し、個別事例の研究や課題の議論、将来への提案という形で示されることが多く、体系的に発見し統合することは容易ではありません。

    この論文は、新しい理論を急いで提案するのではなく、まず多くの先行研究から設計上の知見を抽出し、整理し、統合するという基礎的で重要な作業を行っています。そして、それらを複数回の評価を通じて検証・修正しながら、より汎用的な設計ガイドラインへとまとめています。

    このことから、学術的な革新は必ずしも新しい発見だけから生まれるものではないことが分かります。ときには、散在している知識を整理し、構造化された枠組みとして示すこと自体が重要な貢献になります。

    新しい主張を行う前に、まず既存研究が何を明らかにし、何を解決し、そして何を見落としているのかを理解する必要があります。既存の知識体系を十分に理解してこそ、本当に探究する価値のある空白領域をより正確に見つけることができるのだと思います。

    私にとって、この論文が提供しているのはAI設計に関する知識だけではありません。そこには、一つの学ぶべき研究方法があります。

    まず整理すること。
    次に統合すること。
    そして理解した上で、新しい問いを立てること。

    その順序の大切さを改めて学びました。

     
     
    私はこの世界に来た。 生き、経験してきた。

    あなたへのおすすめ