Art and Design, Vol. 3, Issue 1, Mar  2020, Pages 69-77; DOI:

Human-Computer Interaction in the History of New Media Art

, Vol. 3, Issue 1, Mar  2020, Pages 69-77.


Ke Du 1*

1 School of Architecture and Urban Planning, Huazhong University of Science and Technology, Wuhan, China

Received: 5 December 2019; Accepted: 31 December 2019; Published: 6 February 2020


With the rapid development of digital electronic technology, new media technology is also showing the rapid development of export, thus providing more possibilities for the creation of new media art. New media artists began to try different media, combined with theatre, dance, film, video and other different artistic forms to create, not only led to the development of visual art, science and technology art and audio-visual technology, but also produced cross-cutting creation and other multi-element development. Under the influence of the development of science and technology, interactive art and emerging media, the development of art gradually moves from cross-media to cross-border cooperation of art, so that artists can express their creative ideas in a diversified attitude. Thus communication and cooperation have also become the focus of artists attention in the new media art creation, and the structure of interactive situation has become the first factor to be considered in artistic creation. The interactivity and integration of new media art make art more affinity, reduce the sense of distance of elite art, and receive wide attention and love from the public.


New Media Art, Human-Computer Interaction, Digital Image, Science and Technology, Interactive Art

1. Introduction

1.1. Concept of Human-Computer Interaction

Interaction means mutual communication, human-computer interaction literally is the interaction between people and computers. Strictly speaking, human-computer interaction refers to the use of a dialogue language between people and computers, in a certain way of interaction, to complete the task of the human and computer information exchange process [1].

Human-computer interaction is done through the user and the computer two main bodies: first, the user' s various operational instructions are first received by the computer input device, and then the device will convert these operational instructions into their own recognizable special language, and then input into the computer, The computer then translates the information it is passing into a form that the user can understand through the output of the device, which is accepted by the user. In this process, the state of the computer system, the implementation and operation of the behavior process will be constantly displayed to the user by the machine interface [2]. It can be seen that human-computer interaction realizes the transformation between the internal information mode of the computer and the information form that human beings can perceive.

1.2. Meaning of Human-Computer Interaction

As an independent and important research field, human-computer interface has attracted the attention of computer manufacturers all over the world, and became the 90' s computer industry another competitive field. From the development process of computer system, human-computer interface technology also guides the development of related hardware and software technology, which is the guarantee of the success of a new generation of computer system. In the 1980s, computer software and hardware technology has made great progress, at the same time, computer users have been rapidly expanded from computer experts to the vast number of untrained ordinary users, which greatly enhanced the user interface in system design and software development, strongly stimulate the progress of human-computer interface. The interaction of human-computer is achieved through the user interface [3].

Compared with similar research in international research, there is still a big gap between the designs of human-machine interface in China. The current research focuses only on the software and hardware that supports the interface, and the interface itself lacks in-depth research, and the design of the user interface has not yet become an independent part of the software design. There is also a lack of new human-machine interface designtechnology. On the other hand, the development of computer science and the popularization of computer application have put forward higher and higher requirements for the study of human-machine interface.

2. The Development of Human-Computer Interaction

Computer hardware technology has developed at a rapid pace in the past 20 years, and its history is not only the history of rapid increase in processor speed and memory capacity, but also the history of continuous improvement of human-computer interaction technology. Human-computer interaction technology, such as mouse, window system, hypertext, browser, etc., has had a great impact on the development of computers, and will continue to affect the lives of all mankind. Multi-channel user interface, computer-supported collaborative work, 3D human-computer interaction, etc. are the key to achieve efficient and natural human-computer interaction. [4] The development of human-computer interaction technology has gone through the following stages.

2.1. Non-Interactive Batch Ingress in the Early Days of Computer Development

In the early days of computer birth and development, the interaction between people and computers is very laborious, using perforated cards or paper tape input devices to enter programs and data, the indicators on the control panel show register content, and can only be programmed in binary machine language, debugging is also quite cumbersome.

2.2. Assembly Language from the mid-1950s to the 1960s

People can use assembly language, advanced programming language to program, and with the operation control language, and then appeared interactive terminal, keyboard, monochrome display, stylus and so on [5]. Command language and function keys are the main input methods, their expression function is strong, is still a professional workers cannot give up the interaction, but the memory load of heavy, and key knocking inevitably errors, ordinary people feel inconvenienced.

2.3. " Direct Manipulation Interface" Since the Late 1970s

The direct manipulation interface is generally welcomed by the user, the object to be controlled has a representation on the screen (simulation representation), the person with the mouse or keyboard to operate on it, that is, with the action instead of the complex syntax, and can immediately see the effect (feedback information), and thus also known as direct manipulation interface. Direct manipulation of the interface is a major advance in human-computer interaction technology, making it easier for laymen to have a conversation with a computer [6].

2.4. How Contemporary Multimedia Interacts with Hypermedia

Multimedia technology is considered to be a transition technology before the breakthrough of intelligent user interface and natural interaction technology. Prior to the emergence of the multimedia user interface, the user interface went through a transition from text to graphics, especially the introduction of audio media, greatly enriched the form of computer performance information, improve the efficiency of user acceptance of information. Hypermedia technology, that is, the output form of the computer is no longer limited to a single text, but text, graphics, images, sound, animation or film and other media integration. Super-media breaks the traditional single media boundaries and traditional thinking, the flat media, radio media, network media integration into a region and even the whole society can be transformed into a super power.

3. Interaction of New Media Art

3.1. Interaction of New Media Art

New media art actually embodies the pursuit of humanization on the basis of technology. The humanization of technology is the common pursuit of many scientists and artists. In line with this, ideological and artistic should be an important standard to measure the works of new media art. In the artistic practice of all high-tech media, the most vital works are often those that combine the power of technology with the wisdom of artists. In the new media art, the concept of humanity and technology is a common function of the new media art, in which the concept of humanity is the premise. Liu Yang pointed out: The real works of art first need to be based on the artist' s concept, thinking as the premise of existence, that is, we mentioned earlier human concepts. Its existence allows the artist to establish his own style, to make the work of art have its own soul, and then by the technology to propose the most ingenious and clever solutions to complete it, which is called the technical concept.

As an art form that combines many art, new media art has many characteristics, but the most distinct from other art forms is its innate interaction, which is based on the integration of information technology and art. This trait makes him more realistic than the art on the shelf, more expressive than simple installation art, more affinity than music, this personal experience of presence for the new media art added more fun, but also caters to the post-modern society' s entertainment, fun psychology, it also makes the new media art in the experience economy era more than other art cannot match the vitality [7].

In the digital media and globalization today, artists work time, space, media is also changing, traditional artists began to shift positions, they more through virtual space, network transmission and other channels to communicate and collaborate with the audience. New interactive technologies are widely used, intelligent buildings, smart homes, intelligent robots, molecular processes, cloning technology, genetic engineering and so on are affecting people' s lives and ways of thinking. In the art, the individual expression creativity has been extended from the artist to the audience, the demand of the artist is not just a personal creation, but requires the artist to create the environment, space, to provide the audience with the opportunity to participate in the creation. What an artist is doing now is no longer sampling in the real world to reflect his personal views, but building a framework in which the audience creates his own world.” The freedom to empower individuals to construct their own reality, reinvent themselves, and make the most of the circuit space is the importance of interaction and the true meaning of the digital revolution."

3.2. Interactive Representations

The interactivity of new media art makes new media artists pay more attention to the dialogue feeling and on-the-go experience. Each era has its own art form, and the new media art is undoubtedly one of the most important art forms in the era of experience economy. The development and progress of art is based on the development of economy, the economic foundation is the foundation of all art, and art as a form of ideology is based on economy, that is, material foundation. First of all, the continuous improvement of the tools needed for artistic creation has an important role in advancing the progress of art [8]. Secondly, the emergence of the artist' s thought is based on a certain level of social development, and social progress cannot be separated from economic progress. Finally, the content of art should also be based on the material production of society.

The concept of experience comes from psychology, but the meaning of experience goes far beyond psychology. Zhuangzi Tiandi has a story, Zigong through the south bank of Hanshui, saw an old man is holding the water to water the ground, asked him why he did not use orange cotton, less force and more effect. And the old man replied: with machinery and things like things will certainly appear clever and so on, with clever things such as machine changes and so on. This shows the Chinese ancients simple sense of experience, accustomed to their own understanding, work is not only a means to achieve the goal, but also experience the existence of the " theatre."

In the new media art, the subject' s experience level to the object is as follows: first, in the stage of experience recognition and design coding, on the one hand, it is the artist' s subject' s experience recognition of the visitor' s object. It is necessary to fully and accurately identify the characteristics and needs of the object. On the other hand, the artist' s experience recognition of the properties of the object is the basis of the function and form of the positioning work. Secondly, in the process of art decoding and experience marketization, the consumer subject scoffs and experiences the object of the work.

3.3. The Principle of Interaction in the Art of New Media

3.3.1. Principles of Richness

In the interactive process of the new media art set, the system interacts with people, which leads to the transformation of the work and the human consciousness, and through the engineer' s effective design, the external performance is controllable, dedicated, easy to change, and also can meet the aesthetic concept of the work. Individual expressions and individual creativity have dominated or even dominated the results of the work, and the ultimate artistic effect is not certain from person to person. People' s requirements for artists are no longer the creation of moving content, but the environment, space scene design, or effective communication with intelligent machines, so we emphasize that interactive content itself is a design project, it needs to integrate the value of people, experience design will not be excluded by technology, but will only be excluded by people. Therefore, it is necessary to create a rich interactive experience to help people communicate and understand, artists can also through a variety of rich interaction experience to convey and express their ideas.

3.3.2. Principles of Events

Interactive design is the design of things, on the one hand, is a way of speaking (understanding, action, and state) design. It encourages the use of artificial language models to replace mechanical models [9]. Today' s new technologies allow us to create new products for language models and use them to communicate, creating a things society. The interactive process allows users to exchange their knowledge, exchange their different perspectives and understandings of the world, and by sharing, open up a vast creative space, thus expanding our cognition and understanding. These smart products help people communicate, understand and influence each other, and create them with others.

On the other hand, the process of interaction is an event in motion, which is not a series of individual sensory data, but is based on a certain sense of co-ordination. This event refers not only to the mechanical motion characteristics of the design elements, but also to the design object' s own growth, variability, and order concern. Interactive design is the description and navigation of relevant experiences, and the process of interaction with the user is the process by which the user participates in the story, while the user is creating his own story and thus more participatory and dynamic. The interactive process of new media art is not to treat the user' s behavior as an isolated fact, but to combine the actions, the way of behavior and their connection with time.

3.3.3. Principles of Ease of Use

Obviously, the best user interface is the absence of a user interface while creating meaningful experiences, which is what interactive design is all about. The difficulty is how to use the interface to engage the audience and what ways does it drive the audience to activate and complete the work? The only answer is simplification, simplification, and simplification. New media artists provide a concept, and the ease of interaction is left to the technical staff to solve, the final form may change the way the work participates, but in any case in the face of interactive events, simple convenience is the principle, the audience may only need to intervene or simple action can trigger, the image will be strange changes [10].

3.4. Related Interaction Techniques

In the interactive part of new media art, in addition to the need for the intervention of the audience, the construction of interactive environment system also needs the support of interactive technology.

The interactive technology mentioned here mainly refers to the human-computer interaction technology, that is, the communication technology between computers and people. Human-Computer Interaction Technology (Human-Computer Interaction Techniques) refers to the technology that enables human-computer conversations in an effective way through computer input and output devices [11]. It includes the machine through the output or display equipment to provide a large number of relevant information and tips, etc. , people through the input equipment to the machine input relevant information and tips, etc. , people through the input equipment to the machine input relevant information, answer questions and so on. It involves many popular computer technologies, such as artificial intelligence research, computer graphics, software technology research, fractal geometry research, network technology, multimedia technology, digital video technology, natural language processing, etc., but also absorbs linguistics, ergonomics and sociology research results, is a cross-cutting, marginal, Comprehensive discipline [12].

3.5. Interactive Device Art Mode

3.5.1. The Art of Interface Interactive Devices

Interface interactive new media works are the most common art forms in domestic and foreign exhibitions. Its human-computer interaction mode is: the audience can use language, touch, gesture or control lever, remote control and other body movements to interact with the work, and through the audience in front of the screen or interactive wall to produce feedback results [13]. This interactive work is usually placed in a more open space, and these interactive works are generally clear-cut human-machine interfaces that emphasize the result of interactions by the audience through direct contact. The pattern is primarily a visual, direct, or immediate experience. Examples include Teamlab' s exhibition Plants on 2019 World Travels” in Japan, which features digital works of art. The Ocean Road in the Botanical Garden, The Color of Life, the thousands of light-driven fish drawn by computers in the exhibition hall roam freely in space. Some people pass by, the fish will automatically disperse, their escape route will be integrated into a " light road" , once people touch with their hands, the fish will change the color, nearby fish will gradually dye color. (Figure 1)

Figure 1. Teamlab' s exhibition " Plants on 2019 World Travels" .

There' s also the “Snowfall” of the American fuse studio, which uses multiple cameras to capture images of people in real time, blocking the fall of snowflakes projected onto the wall with outlined outlines. Every move of a person, all affect the screen of the amount of snow and direction. And projection device to create a dreamlike white world, but also make people feel like an immersive, feel the charm of the world of ice and snow. (Figure2)

Figure 2. “Snowfall”.

3.5.2. Immersive Interactive Device Art

Unlike the above-mentioned interface human-computer interaction works, immersive interactive works are mainly displayed in art venues, museums, public spaces and in the form of human-computer interaction mode of environmental experience or multi-channel interaction. This work of art captures a variety of people' s feelings (e.g. speech, eye, facial expression, lip movement, touch, smell or taste) and movements (e.g. speech, handwriting, posture, line of sight, expression, etc.) through a computer, and often has indirect, circumstantial, lagging, delayed features, while the audience is often immersed interact with the work through interactive detection devices such as fiber optics, touch, and projection sensors. Works respond to the audience with the help of light, sound, vibration, or animation [14].

Immersive interactive works often require a relatively closed interactive work space or a dark immersion space similar to a cinema. This not only makes it easier for the viewer to concentrate on the work of art, but also helps to create a space that combines electronic screens, acoustic photoreceptors and simulated natural environments, such as sound effects such as water flow, collisions, or waterfalls, so that the audience can experience a richer sensory experience in the dark [15].

4. Conclusions

Because of its device hardware as its bearing base, interactive art combines its unique interactivity, which makes it widely used in many other fields. Due to the difference in the speed of the development of science and technology in different fields, and the strong economic impetus in some areas, some interactive products are relatively mature and have independent research fields.

With the rapid development of science and technology, the change of technical means and mode will certainly have a great impact on the development of interactive art. At the same time, its development should also be affected by the new media art. We have entered an era of rapid development of information technology. Society is constantly improving and art is developing. The history of human art development is in fact a history of the evolution of media and science and technology, and every scientific and technological progress will bring about great changes in art. Interactive art is the combination of art and technology. Not only as an experimental art of existence, but also gradually integrated into people' s daily life, in business activities, living environment and industrial production are playing a unique role.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this article.


This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.


© 2017 by the authors. Licensee International Technology and Science Press Limited. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


[1]Milgram, P.; Kishino, F. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, 1994, 77(12), 1321-1329.

[2]Azuma, R.; Baillot, Y.; Behringer, R.; et al. Recent advances in augmented reality. IEEE Computer Graphics and Applications, 2001, 21(6), 34-47.

[3]Azuma, R.T. A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 1997, 6(4), 355-385.

[4]Zhou, F.; Duh, H.B.L.; Billinghurst, M. Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. Los Alamitos: IEEE Computer Society Press, 2008, pp. 193-202.

[5]Lee, J.; Olwal, A.; Ishii, H.; et al. SpaceTop: integrating 2D and spatial 3D interactions in a see-through desktop environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM Press, 2013, pp. 189-192.

[6]Wang, X.C.; Tian, F.; Dai, G.Z. Table-form making based on pen interaction. Journal of Computer-Aided Design & Computer Graphics, 2007, 19(5), 558-562.

[7]Zhou, Z.; Zhou, Y.; Xiao, J.J. Survey on augmented virtual environment and augmented reality. Science China: Information Sciences, 2015, 45(2), 157-180.

[8]Azuma, R.; Baillot, Y.; Behringer, R.; et al. Recent advances in augmented reality. IEEE Computer Graphics and Applications, 2001, 21(6), 34-47.

[9]Rusu, R.B.; Marton, Z.C.; Blodow, N.; Dolha M.; Beetz, M. Towards 3D Point cloud based object maps for household environments. Robotics and Autonomous Systems. 2008, 56(11), 927-941.

[10]Poo, M.M.; Du, J.L.; Ip, N.Y.; Xiong, Z.Q.; Xu, B.; Tan, T. China Brain Project: Basic Neuroscience, Brain Diseases, and Brain-Inspired Computing. Neuron. 2016, 92(3), 591-596.

[11]Stephen, H. Fairclough. Fundamentals of physiological computing. Interacting with Computers. 2009, 21(1-2), 133-145.

[12]Curran, E.A.; Stokes, M.J. Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems. Brain and Cognition. 2003, 3, 326-36.

[13]Luis Fernando Nicolas-Alonso, Jaime Gomez-Gil. Brain Computer Interfaces, a Review. Sensors (Basel). 2012, 12(2), 1211-1279.

[14]Curran, E.A.; Stokes, M.J. Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems. Brain and Cognition. 2003, 51(3), 326-36.

[15]Fan, J.; McCandliss, B.D.; Sommer, T.; Raz, A.; Posner, M.I. Testing the Efficiency and Independence of Attentional Networks. J Cogn Neurosci. 2002, 14(3), 340-7.