Preemptive Action

Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation Without Compromising Agency

Shunichi Kasahara, Jun Nishida, and Pedro Lopes
ACM CHI 2019

This is a collaborative research project with Jun Nishida and Pedro Lopes (The University of Chicago).

本プロジェクトは、シカゴ大学の西田惇氏、Pedro Lopes氏との共同研究です。

Abstract

We enable preemptive force-feedback systems to speed up human reaction time without fully compromising the user’s sense of agency. Typically these interfaces actuate by means of electrical muscle stimulation (EMS) or mechanical actuators; they preemptively move the user to perform a task, such as to improve movement performance (e.g., EMS-assisted drumming). Unfortunately, when using preemptive force-feedback users do not feel in control and loose their sense of agency. We address this by actuating the user’s body, using EMS, within a particular time window (160 ms after visual stimulus), which we found to speed up reaction time by 80 ms in our first study. With this preemptive timing, when the user and system move congruently, the user feels that they initiated the motion, yet their reaction time is faster than usual. As our second study demonstrated, this particular timing significantly increased agency when compared to the current practice in EMS-based devices. We conclude by illustrating, using examples from the HCI literature, how to leverage our findings to provide more agency to automated haptic interfaces.

Moving the body faster than one’s own will

As computers get better at predicting our actions and technology learns to drive the body directly, it becomes possible to move our body faster and more accurately than our own will. Force-feedback systems already do this with electrical muscle stimulation (EMS) or mechanical actuators: they move the user preemptively, for example to keep a drumming rhythm or to steer a pedestrian into safety. What such a system has to preserve is the sense of agency, the subjective awareness of initiating, executing and controlling one’s own voluntary actions. A system that acts automatically and preemptively tends to take it away, and the user loses the sense of “I did this”.

At first glance it seems paradoxical to ask for both: if a system moves the user faster than they normally move, how could the user still feel that they started the movement? Our approach builds on the finding that conscious awareness of an intention takes a moment, around 200 ms, to arise. This time window may leave room to attribute a movement to oneself even when a device generated it, as long as the movement matches what the user intended. The question is exactly when the device should act.

A hand with EMS electrodes on the forearm, its finger moved by the stimulation, next to a sketch of a curve of sense of agency against preemptive gain — EMS moves the user’s finger ahead of their own reaction (preemptive action). The sketch on the right shows the relationship the project set out to measure: the sense of agency against the preemptive gain, the time the system saves.

Tapping a target while the EMS moves the finger

In the first study, 12 participants tapped a target on a touchscreen as fast as possible. EMS on the forearm flexed the ring finger at a random time between 200 ms before and 400 ms after the target appeared, and vibration motors on the arm masked the tingling of the stimulation. After every tap, participants rated their sense of agency from 1 (“I did not do it”) to 7 (“I did it”).

The earlier the EMS acted, the faster the tap and the lower the agency. When the EMS fired as the target appeared, the tap came 47.5 ms after the target on average, far faster than a human reaction, and participants barely felt that they had tapped (agency 1.25). When it fired about 300 ms after the target, close to their own reaction time, agency approached the maximum (6.81), but the tap was no faster than usual.

Feeling “I did it” ahead of one’s own reaction

Participants felt that they had made the movement themselves even when the EMS had moved their finger, as long as it moved them no more than about 80 ms ahead of their own reaction time: within this window agency stayed high, and beyond it agency dropped sharply. Measured against each participant’s usual reaction time, agency averaged 5.13 on the 7-point scale when the EMS sped them up by up to 80 ms; at a speed-up of 160 ms it fell to 1.89. A model fitted to all participants’ ratings puts the balance between agency and speed at a preemptive gain of 80 ms, where normalized agency is 0.5. Assuming a reaction time of 280 ms and 40 ms from triggering the EMS to the touch, this means triggering the EMS 160 ms after the target appears.

A sigmoid curve of normalized perceived agency against preemptive gain from 400 to minus 300 ms, with agency 0.5 at a gain of about 80 ms marked as the optimal preemptive gain — The model fitted to all ratings from the first study: normalized agency against preemptive gain, the time by which the EMS speeds up the user’s reaction. At a gain of 80 ms (P_g*), agency and speed are balanced.

Many participants described stimulated taps as their own: “I rated 6 or 5 when I felt some EMS sensation but still I believe I control my finger.” With the EMS 200 ms before the target, one participant noticed that “it’s too fast, it moves before I do”. With the EMS around 160 ms after the target, the same participant said: “I know I felt some tingling but I thought it was some weak feedback, but, now [in hindsight] I actually believe I moved my finger too.”

Faster than one’s own reaction, with more agency than current practice

The second study, with 12 new participants, tested the 160 ms timing (Preemptive Action) against three other conditions in the same tapping task: EMS at the moment the target appears, which is how EMS-based systems usually act (Baseline); EMS at 160 ms while the participant relaxed (EMS Only); and the participant tapping alone while a sham stimulation produced only the tingling (Sham). Participants were not told about the conditions.

Preemptive Action was significantly faster than participants’ own taps in the Sham condition, and it gave significantly more agency than Baseline, whose agency was no higher than when participants relaxed and let the EMS move them. Agency also required the user’s own intention: in EMS Only, the EMS moved the finger at the same moment, with no difference in reaction time, yet participants felt barely any agency (median 1.0). Preemptive Action still gave less agency than tapping alone.

Two box plots for Preemptive Action, EMS Only, Sham and Baseline: reaction time on the left, perceived agency score on the right, with significance brackets — Second study: reaction time (left) and perceived agency (right) in the four conditions. Preemptive Action was faster than tapping alone (Sham) and gave more agency than EMS at the moment the target appeared (Baseline).

Why the stimulation can feel like one’s own movement

The paper explains the result with Libet’s finding that the desire to act can reach consciousness after the motor commands have been issued. Triggering the EMS about 80 ms before the participants’ average reaction time places the involuntary movement in that range, around 200–300 ms before the conscious act, so the origin of the movement becomes unclear. The intention still comes first, and the outcome on the screen matches it, which preserves two of Wegner’s principles of agency, priority and consistency. One participant put it this way: “I know this [the EMS] moved my finger, at some point, but I feel this [EMS] and my movement to get integrated, rather than just moved by it.”

Retuning existing systems

The model lets designers choose how much agency to trade for speed. In Wired Muscle, an earlier system of ours, a person catches a falling pen with help from EMS that is triggered by the muscle activity of the person releasing it. The grasp was complete 60 ms after the release, 190 ms faster than a usual reaction of 250 ms, a point where the model predicts a normalized agency of only 0.18. Catching the pen needs a reaction within 200 ms, so a gain of 50 ms is enough: triggering the EMS 150 ms after the release instead of 10 ms still catches the pen and raises the predicted agency to 0.65. The paper suggests the same retuning for other EMS-based systems and for mechanically actuated haptics such as exoskeletons.

Timelines for a human baseline grab at 250 ms, a naive EMS preemption grabbing at 60 ms and an optimal EMS preemption grabbing at 200 ms, above the agency curve showing expected agency 0.18 and 0.65 — Retuning Wired Muscle with the model. The original timing (Naive EMS Preemption) gains 190 ms with an expected agency of 0.18; stimulating later (Optimal EMS Preemption) still grabs the pen within 200 ms and raises the expected agency to 0.65.

Jun Nishida, Shunichi Kasahara, and Pedro Lopes. 2019. Demonstrating preemptive reaction: accelerating human reaction using electrical muscle stimulation without compromising agency. In ACM SIGGRAPH 2019 Emerging Technologies (SIGGRAPH ’19). ACM, New York, NY, USA, Article 10, 2 pages. DOI:https://doi.org/10.1145/3305367.3327997 (Laval Virtual Grand Prize)
Jun Nishida, Shunichi Kasahara, and Kenji Suzuki. 2017. Wired muscle: generating faster kinesthetic reaction by inter-personally connecting muscles. In ACM SIGGRAPH 2017 Emerging Technologies (SIGGRAPH ’17). ACM, New York, NY, USA, Article 26, 2 pages. DOI: https://doi.org/10.1145/3084822.3084844 (project page)

日本語

Preemptive Action : 意志よりも早く動かされる身体”自分がやった”知覚の境界を探る

自分の意志によって身体が動く前に、コンピュータによって“動かされた”としても、その行為を”自らが行った” (Sense of Agency)と知覚することがある。本研究は、EMS（筋電気刺激) を用いて、その時間的領域と成立条件を探る。

行為主体感を生み出す時間境界

コンピュータによる予測性能の向上と、身体そのものを直接制御するテクノロジの進化によって、自分の意志よりも早く・正確に自身の身体を駆動させることが可能になる。その時保障するべきは、行為の主体は自分であるという感覚(Sense of Agency)であり、これを生み出すための制御基盤が今後重要になる。

本研究では、 EMS（筋電気刺激) を用いて、人工的に自分の意志と同じであるが、自分の能力よりも早く身体（指）を駆動させ、その行為に対する行為主体感計測した。結果、通常の視覚反応よりも約80 ms 時間先行させても行為主体感を保持する時間境界があることが示唆された。

Reference

Shunichi Kasahara, Jun Nishida, and Pedro Lopes. 2019. Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation Without Compromising Agency. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA, Paper 643, 1–15. https://doi.org/10.1145/3290605.3300873

BibTeX

@inproceedings{10.1145/3290605.3300873,
  author    = {Kasahara, Shunichi and Nishida, Jun and Lopes, Pedro},
  title     = {Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation Without Compromising Agency},
  booktitle = {Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems},
  series    = {CHI '19},
  year      = {2019},
  articleno = {643},
  pages     = {1--15},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  doi       = {10.1145/3290605.3300873}
}

News: Project featured in the IEEE Spectrum web article, Received Laval Virtual Award 2020
Related projects: Wired Muscle, EMS Speeds Up, Whose touch is this?, Faster Than Muscle