Newsletter about recent updates

This is an article contributed to a newsletter published by Sony CSL.

【Exploring the “New Humanity” Emerging from the Integration of Humans and Computers】

New trends begin at the frontier.
That’s why I’m in the southern island of Japan.

Cybernetic Humanity

The Cybernetic Humanity research project launched in its original form in April 2021. “Cybernetic Humanity” is a research topic that focuses on the “new humanity” emerged from the integration of humans and computers, and which approaches its investigations through both computer science and human science. We are conducting research on how the dynamics of human nature will be interwoven through the integration of human beings and computers: augmented perception and cognitions through technology , the potential new types of “self” forged when we merge with computers, and interpersonal interactions mediated by AI.

At the time I write this, our project has a total of 12 members, including the project leader (Shunichi Kasahara), one full-time project researcher, two affiliated researchers, three visiting researchers, and five research assistants. We are working on projects that combine our members’ diverse fields of expertise, which include engineering, cognition & perception, wearables, experimental psychology, community design, and interactive systems.

In addition, we are proactively collaborating with cognitive neuroscience, media design, systems neuroscience, and behavioral economics researchers, as well as with media artists. Notably, as part of joint research between Sony Computer Science Laboratories and the Okinawa Institute of Science and Technology Graduate University (OIST), the Cybernetic Humanity Studio has been started on the OIST campus, and our two teams in Tokyo and Okinawa are also sharing research remotely.

In this article, I will discuss the background of our research concept, why we initiated work in Okinawa, and the directions we hope to take our research.

The Augmented Self and the Invariant Us

“How do we define the self?” This was once a purely philosophical question, but today it is a vital issue in the research in human-computer interaction.

When humans integrate with computers, to what extent are we still ourselves? Computers are no longer just tools for humans, but are deeply intervening in our bodies and behavior. As computing technology evolves, it is becoming possible to assist, reinforce, and augment human action, decision-making, and more. When humans integrate with computers and acquire abilities and different bodies beyond what we currently possess, how do we define our own humanity?

However, if we are to create a world where humans and technology are truly integrated, I believe we cannot avoid considering the self, the human side of things, and ask the question, “To what extent is this really me?” Even if we are able to give ourselves abilities far beyond those of ordinary human beings, or put ourselves in entirely new bodies, if we do not feel both cognitively and perceptually that this technology is part to us, then we cannot say that humans and computers have reached a state of mutual adaptation. This is why I believe it is so important to delineate and design the integration of humans and computers with the “self” in mind.

In addition to assistive technologies including augmenting human ability, advances in large language models and generative AI are now making it possible to generate copies of ourselves—our images, sounds, thoughts and actions.

You can release a generative network that speaks different languages fluently over footage of your face, allowing anyone to create another version of you, even a voice generator that creates songs using your own voice. This is just one example of we can now “resynthesize” ourselves using computers.

Apple has announced an iOS accessibility feature for people at risk of losing the ability to speak; it can learn your own voice and read text aloud in it [1]. Technologies like these offer so many possibilities for augmenting the expressive power of the self. Augmented selves will become our new user interfaces.

However, as these technologies evolve and democratize so rapidly, many issues remain. The impacts of this self-augmentation on social norms and on how we perceive ourselves are not yet fully understood, and many potential issue are unveiled.

Actually, many concerns are being raised. The use of synthetic versions of other people’s voices or appearance raises ethical concerns regarding infringement of privacy and identity, and creates disintegrated digital identities that may result in psychological issues; assistive systems and other automatically controlled technologies may rob us of our sense of agency and embodiment, in turn making it unclear where to assign responsibility, corroding moral standards as we lose independence.

Recently, SAG-AFTRA (a US-based actors’ union) went on strike. One of their serious concerns is producers using 3D scans of actors in ways that the actors did not consent to.

We believe that dealing with social issues raised by progress in AI and other technologies should not only involve restrains and restrictions. It is important to understand the changes that technological progress will cause to our sense of self, independence, and identity, and then augment ourselves accordingly and appropriately. Concerns about the evolution of AI could lead to blanket regulations and restrictions on its use that will ultimately hinder technological progress.

When we think about how the development of science and technology can be reflected in society in desirable ways, it is important that, as humans and computers integrate more and more, we do not think of AI in terms of something we “use” or “are used by”—rather, we must understand how to incorporate users’ identities and augment their subjective sense of self. We believe this will make it possible to create designs that address social issues.

Based on those backgrounds, we believe it is vital to understand how the “new humanity” created by the integration of humans and computers will affect awareness of self, communication, and social relationships, in terms of our sensory experience and perception as well as in terms of the social and human sciences.  

Cybernetic Humanity

If we augment the self, what can be the self? The augmentation brought about by the evolution of technology and the subjectivity with which we define ourselves. How do we reconcile these two seemingly contradictory elements? To express this question, we have named the concept “Cybernetic Humanity”, focusing on the “new humanity” created by the integration of people and computers. We see this as a new field of research to be explored through both computer science and human science.

The exploration of Cybernetic Humanity requires deep thinking that seeks to connect seemingly contradictory and complex characteristics of embodiment expanded by technology and the human nature inherent in ourselves. To grapple with this complex set of problems, we have moved beyond a reductive, factorizing approach and adopted a flexible, we are adopting research cycle that treats human bodies and humanity itself as Open Systems through generating and analyzing new experiences.

So far, our team has conducted research on the fusion of computers and humans[7], the sense of agency in human augmentation (the feeling that “I am performing this action”)[6,8], the relationship between generative AI and humans perception and cognition[5], and studies related to facial identity[3]. We are also undertaking research regarding self-understanding in integrated human-computer frameworks, both on the nature of subjectivity [6] and on transformations in human social behavior in systems in which humans and AI are working together [4].

We are conducting a circular research approach that involves implementing those studies as interactive systems, carrying out empirical experiments through social implementation, exhibiting experiential works, and citizen participation workshops, among other fieldwork. We engage with society by analyzing both qualitative and quantitative data, which helps us further develop and advance our research.

Starting from the Frontier

Where should we start our investigations into Cybernetic Humanity? As for when I started thinking about this concept, I was rethinking my own research frameworks, and, because of the COVID-19 pandemic, rethinking where exactly I could do my research. I homed in on this “new humanity” created by the integration of humans and computers, and on the points at which computer science and the natural sciences meet—what kind of research framework would I need to create a new scientific trend? 

My attention then turned towards the Okinawa Institute of Science and Technology Graduate University (OIST), which has an extremely strong research portfolio in the theoretical sciences and the natural sciences. Dr. Svante Pääbo, 2022 winner of the Nobel Prize in Physiology or Medicine, is an adjunct professor at OIST.

Located in the village of Onna in the central part of Okinawa, OIST is a very precious place, surround by the ocean and the Yanbaru forests. (And it’s about 90 minutes from Naha Airport by express bus and shuttle bus!) With the unique natural environment, research on plant and marine biology is extremely active. As of 2023, roughly 80% of the students are international, and they come from many different countries, making OIST a truly exceptional graduate school.

What makes me so extremely interested in OIST is that it is a frontier. As I just said, the natural sciences are flourishing at OIST, but there are not many computer scientists here, and actually no researchers in my specialty of human-computer interaction. I thus thought this would be the perfect place to bring Cybernetic Humanity in order to catalyze it.

In my own career, I have tried to move between different domains while sublimating what they have in common, undertaking new research and uncovering issues that need to be worked out. The research projects I want to do, the characteristics of OIST, and my work at CSL all fit together, and I have been able to initiate research at the Cybernetic Humanity Studio, a collaboration between CSL and OIST.

In April 2023, I moved to Okinawa and started full-fledged work here. Immediately after, on April 21, together with Dr. Anatole Lécuyer of INRIA (the French National Institute for Research in Digital Science and Technology), who had a visiting position at OIST, we held a seminar and public lecture on embodiment, virtual reality and human augmentation [9], and also held an open house for the Cybernetic Humanity Studio [10]. In the event, we welcomed more than 100 attendees, some from OIST and some from outside. I am extremely pleased that our activities are off to such a successful start.

OIST is also very interested in this new trends, and we have already initiated several joint research projects. Can we reveal unique aspects of humanity using the perspectives of other species? What are the differences between the physical phenomena of interaction and communication? How we define the self in embodied cognitive science? These questions are just some of the joint research that are already underway. My vision is to establish a framework for scientific research of “new humanity” that connects computer science and natural science and integrates humans and computers. This is what I want to explore as we forge bonds between Sony CSL and OIST.

Coincidentally, Sony Computer Science Laboratories has changed an organizational structure, and I am now part of Transboundary Research, meaning I’m not based in Tokyo or Kyoto. A nomad researcher at the southern edge of Japan, I will work here at the frontier to create new trends, crossing the borders between industry and academia; science, engineering, and design; and computer science and natural science.

[1] Apple, Personal Voice, https://www.apple.com/newsroom/2023/05/apple-previews-live-speech-personal-voice-and-more-new-accessibility-features/
[2] 2023 SAG-AFTRA strike, https://www.sagaftrastrike.org/
[3] Morphing Identity, doi:10.1145/3544548.3580853
[4] Extinction of reciprocity in a semi-automated driving coordination experiment, IC2S2
[5] Human Latent Metrics, doi:10.1145/3548814.3551460
[6] Whose Touch Is This?, doi: 10.1145/3489608
[7] Transparency in Human-Machine Mutual Action, doi:10.20965/jrm.2021.p0987
[8] Preemptive Action, doi:10.1145/3290605.3300873
[9] French-Japanese seminar, https://www.oist.jp/news-center/news/2023/5/18/oist-hosts-french-japanese-seminar-metaverse-and-virtual-reality
[10] SonyCSL web, https://www.sonycsl.co.jp/event/15158/

Report by Sony CSL

(Shunichi Kasahara, Sony Computer Science Laboratories)

Scroll to top