Redefining Affordance via Computational Rationality Conference: 30th International Conference on Intelligent User Interfaces; March 24–27, 2025; Cagliari, Italy 30th International Conference on Intelligent User Interfaces (IUI ’25), March 24–27, 2025, Cagliari, Italy DOI: 10.1145/3708359.3712114 ISB
Yi-Chi Liao, Christian Holz · 2025
Read Redefining Affordance via Computational Rationality Conference: 30th International Conference on Intelligent User Interfaces; March 24–27, 2025; Cagliari, Italy 30th International Conference on Intelligent User Interfaces (IUI ’25), March 24–27, 2025, Cagliari, Italy DOI: 10.1145/3708359.3712114 ISB by Yi-Chi Liao, Christian Holz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-09233.txt.
Introduction
Yi-Chi Liao and Christian Holz wrote this research paper, published in 2025. It redefines affordance, the classic design idea of what an object invites you to do with it, through the lens of how the mind makes bounded, sensible decisions.
Older theories say people perceive action chances directly from the world. The authors argue this cannot explain how such perceptions are learned, corrected, or mistaken, or how people pick among several chances. Their theory says the brain builds an inner model from limited input. Inside that model, affordances are guessed from two mechanisms: spotting features and imagining motion paths. Each candidate action is scored by confidence it will work and by the value of its outcome. Perception then becomes a decision that keeps updating through feedback. The authors test the idea with thought experiments across physical, digital, and social cases.
Catalog record
| CREATOR | Yi-Chi Liao, Christian Holz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-09233
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Redefining Affordance via Computational Rationality Conference: 30th International Conference on Intelligent User Interfaces; March 24–27, 2025; Cagliari, Italy 30th International Conference on Intelligent User Interfaces (IUI ’25), March 24–27, 2025, Cagliari, Italy DOI: 10.1145/3708359.3712114 ISBN: 979-8-4007-1306-4/25/03 CCS: Human-centered computing HCI theory, concepts and models CCS: Human-centered computing Human computer interaction (HCI) Yi-Chi Liao, Christian Holz Abstract . Affordances, a foundational concept in human-computer interaction and design, have traditionally been explained by direct-perception theories, which assume that individuals perceive action possibilities directly from the environment. However, these theories fall short of explaining how affordances are perceived, learned, refined, or misperceived, and how users choose between multiple affordances in dynamic contexts. This paper introduces a novel affordance theory grounded in Computational Rationality , positing that humans construct internal representations of the world based on bounded sensory inputs. Within these internal models, affordances are inferred through two core mechanisms: feature recognition and hypothetical motion trajectories. Our theory redefines affordance perception as a decision-making process, driven by two components: confidence (the perceived likelihood of successfully executing an action) and predicted utility (the expected value of the outcome). By balancing these factors, individuals make informed decisions about which actions to take. Our theory frames affordances perception as dynamic, continuously learned, and refined through reinforcement and feedback. We validate the theory via thought experiments and demonstrate its applicability across diverse types of affordances (e.g., physical, digital, social). Beyond clarifying and generalizing the understanding of affordances across contexts, our theory serves as a foundation for improving design communication and guiding the development of more adaptive and intuitive systems that evolve with user capabilities. 1. Introduction Human beings have a remarkable ability to intuitively identify appropriate actions to take within their environments. For example, when seeing a mug, we instantly perceive that the mug affords us to grasp it, hold it, etc. Coined by Gibson ( Gibson and Carmichael 1966 ) , the term affordance refers to “the action possibilities that the environment presents.” Rooted in his direct-perception view, Gibson believes humans can directly “pick up” affordance information from the external world. Consequently, affordances and their corresponding perception are aligned. Within the framework, the definition and the perception mechanisms of affordance are unclear, or even missing, from the first place. To provide a concrete definition of affordance, ecological psychologists have developed divergent viewpoints. Property-based affordance characterizes affordance as an inherent property of the environment ( Turvey 1992 ) , while relation-based affordance posits that affordance emerges from the relation between an actor and their environment ( Chemero 2003 ) . These views are conflicting ( Chemero 2003 ) and either provide a clear mechanism for perceiving affordances. Later on, the concept of affordance was introduced to the fields of design and human-computer interaction (HCI) ( Norman 1999 ; Gaver 1991a ) . Due to the lack of concrete definition and perceiving mechanisms, researchers soon notice several significant challenges that the original affordance theory can not explain. First, there are occasions when the perceived affordance deviates from the actual action possibilities, e.g., "false affordance" and "hidden affordance" ( Gaver 1991a ) . The direct-perception theory can not explain this discrepancy between the physical world and perception. Norman attempted to answer this discrepancy by separating affordance and perceived affordance ( Norman 1988 ) , but did not answer how we perceive affordance and why there are discrepancies. Second, there exist complicated actions that can not be easily explained by human’s direct perception; for example, the “save icon” on an interface allows “saving files” ( McGrenere and Ho 2000 ; Norman 1999 ; Gaver 1991a ) . To address this issue, prior HCI researchers coined different terms, such as Technology or Digital Affordance (action possibilities within the digital world) ( McGrenere and Ho 2000 ) , Sequential Affordance (a series of meaningful actions) ( Gaver 1991a ) , Complex Affordance (requiring previous experience or practice) ( McGrenere and Ho 2000 ; Albrechtsen et al. 2001 ) , Social Affordance (appropriate actions in certain social dynamics) ( Vyas et al. 2006 ) , etc. While these terms expand the meaning of affordance, these deviations imply that there are multiple affordance definitions and perceiving channels, which are still unexplored or undefined ( McGrenere and Ho 2000 ; Gaver 1996 ) . Further, it shows that humans learn and adapt to new affordances, but the learning mechanism is unclear ( Franchak et al. 2010 ) . Lastly, traditional affordance theories tend to detach affordances from the actor’s internal goals and the decision-making processes that guide action selection, such as evaluating utility and confidence. Thus, they do not explicitly explain how humans make choices when faced with multiple affordances, especially in complex or ambiguous environments where goals and expected outcomes play a crucial role ( Oliver 2005 ; Bach et al. 2014 ) . In a nutshell, the affordance theories extended from Gibson’s view can not provide concrete definitions of affordance or the perceiving mechanisms, leading to more ununified terms and confusion. Further, these confusions hinder its actual application in design practice. Figure 1. (a) Traditional affordance theory (Gibson’s view): The organism directly perceives affordances as action possibilities presented by the environment, without the need for internal processing or cognitive mediation. These directly perceived affordances guide subsequent actions. (b) Computational Rationality-based affordance: Our theory proposes that while “external affordances” exist in the external environment, organisms can not directly perceive it. Instead, an internal representation (the “internal environment”) is constructed from constrained sensory inputs. Within this internal environment, “internal affordances” are inferred, which contain two components: confidence in executing the action and predicted utility (expected outcome of taking an action). The organism refines these internal affordances through reinforcement learning, using feedback from previous actions (e.g., rewards) to continuously update and improve affordance inferences. We aim to fill in the gap. Instead of inheriting Gibson’s view in which affordance can be directly perceived as “information pickup”, we argue that affordance perception is constructed, and can be computationally modeled. We present a novel affordance theory based on Computational Rationality (CR) ( Gershman et al. 2015 ; Oulasvirta et al. 2022 ) — a framework that models human decision-making as an optimal process of reasoning and problem-solving under constraints in an internally constructed environment. In CR, humans make decisions by evaluating actions based on their predicted outcomes, while accounting for the limited information and resources available. This framework emphasizes that decisions are not made based on direct access to external reality but through internal representations shaped by experience and sensory data. In our CR-based affordance (illustrated in Figure 1 ), we argue that humans (i.e., organisms) can not directly perceive information from the external world. Instead, we construct an internal representation of the world based on constrained sensory inputs. This mental construct, denoted as the "internal environment," is what humans have access to. Because of the separation of external and internal environment, our theory deliberately segments between external affordance and internal affordance . External affordances refer to actions physically feasible for a given actor within an environment, which humans can not directly observe. Internal affordances are humans’ inferences within the internal environment. For each action, the internal affordance contains two pieces of information: the inferred confidence of executing it