MULTI-OBJECTIVE OPTIMIZATION RESEARCH ON VR TASK SCENARIO DESIGN BASED ON COGNITIVE LOAD

Qian-Wen Fu, Qing-Hua Liu, Tao Hu

DOI Number
https://doi.org/10.22190/FUME240122029F
First page
293
Last page
313

Abstract


In order to improve the efficiency of information acquisition and task selection in Virtual Reality (VR) systems, enhance the interactive experience, and reduce cognitive load for users, it is crucial to effectively organize and leverage user cognitive psychology and design elements during the VR scene design phase. This paper focuses on analyzing the low cognitive load requirements of users and the need for a satisfactory user perceptual experience based on the cognitive resource theory. We propose a method for optimizing the design of VR system scenario tasks under low cognitive load requirements. By utilizing human-computer hybrid intelligent assistance for predicting user cognitive load and incorporating intelligent optimization genetic algorithms into the optimization of VR system design elements, we aim to minimize cognitive load as the objective function based on the principle of low cognitive load. Important knowledge granularity nodes are used as fitness functions in the optimization process of VR system design resource elements. An application study is conducted, combining the multi-channel cognition in a smart city VR system task information interface, to optimize the system resource features. The study validates and compares the solutions obtained through traditional design processes and the solutions optimized by the method proposed in this paper, using virtual reality eye-tracking experiments for the same design task requirements in VR systems. The results demonstrate that users experience lower cognitive load and better task operation experience when interacting with the optimized solutions proposed in this paper. Therefore, the optimization method studied in this paper can serve as a reference for the construction of virtual reality systems.

Keywords

Human-computer hybrid intelligence, Cognitive load, Virtual reality, Multi-objective optimization, Task scenarios

Full Text:

PDF

References


Yongjae, L., Byounghyun, Y., 2021, XR collaboration beyond virtual reality: work in the real world, Journal of Computational Design and Engineering, 8(2), pp. 756-772.

Shi, M.D., Hu, T., Yu, J.W., 2022, Pointing cursor interaction in virtual reality from the perspective of distance perception, Traitement du Signal, 39(2), pp. 475-483.

Marinkovic D., Zehn M., 2019, Survey of finite element method-based real-time simulations, Applied Sciences, 9(14), 2775.

Zdravković M., Korunović N., 2023, Novel methodology for real-time structural analysis assistance in custom product design, Facta Universitatis-Series Mechanical Engineering, 21(2), pp. 239-305.

Liu, Z., Xu, Y., Hu, T., 2023, On a moving target selection model in virtual reality based on decision trees, Traitement du Signal, 40(1), pp. 367-373.

Keil, J., Korte, A., Ratmer, A., Edler, D., Dickmann, F., 2020, Augmented Reality (AR) and spatial cognition: Effects of holographic grids on distance estimation and location memory in a 3D indoor scenario, PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science, 88(2), pp. 165-172.

Muguro, J.K., Laksono, P.W., Sasatake, Y., Matsushita, K., Sasaki, M., 2021, User Monitoring in autonomous driving system using gamified task: A case for VR/AR in-car gaming, Multimodal Technologies and Interaction, 5(8), 40.

Goncalves, R.C., Louw, T.L., Madigan, R., Quaresma, M., Romano, R., Merat, 2022, The effect of information from dash-based human-machine interfaces on drivers' gaze patterns and lane-change manoeuvres after conditionally automated driving, Accident Analysis and Prevention, 2022(174), 106726.

Weng, C., Yuan, Q., Suarez-Ruiz, F., Chen, I.M., 2020, A telemanipulation-based human-robot collaboration method to teach aerospace masking skills, IEEE Transactions on Industrial Informatics, 16(5), pp. 3076-3084.

Chakraborti, T., Sreedharan, S., Kulkarni, A., Kambhampati, S., 2018, Projection-aware task planning and execution for human-in-the-loop operation of robots in a mixed-reality workspace, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4476-4482.

Hernandez, J.D., Sobti, S., Sciola, A., Moll, M., Kavraki, L.E., 2020, Increasing Robot Autonomy via Motion Planning and an Augmented Reality Interface, IEEE Robotics and Automation Letters, 5(2), pp. 1017-1023.

Su, Y., Chen, X., Zhou, T., Pretty, C., Chase, G., 2022, Mixed reality-integrated 3D/2D vision mapping for intuitive teleoperation of mobile manipulator, Robotics and Computer-Integrated Manufacturing, 77, 102332.

Wang, P., Bai, X., Billinghurst, M., Zhang, S.S., Han, D.C., Lv, H., He, W.P., Yan, Y.X., Zhang, X.Y., Min, H.T., 2019, An MR remote collaborative platform based on 3D CAD models for training in industry, IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 91-92.

Paul, S., Reardon, C., Williams, T., Zhang, H., 2020, Designing augmented reality visualizations for synchronized and time-dominant human-robot teaming, In Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-Domain Operations, 11426.

Luong, T., Lecuyer, A., Martin, N., Argelaguet, F., 2022, A survey on affective and cognitive VR, IEEE Transactions on Visualization and Computer Graphics, 28(12), pp. 5154-5171.

Ariali, S., Zinn, B., 2021, Adaptive training of the mental rotation ability in an immersive virtual environment, International Journal of Emerging Technologies in Learning, 16(9), pp. 20-39.

Lerner, D., Mohr, S., Schild, J., Goring, M., Luiz, T., 2020, An immersive multi-user virtual reality for emergency simulation training: Usability study, JMIR Serious Games, 8(3), 18822.

Villar, B.F., Vinas, P.F., Turiel, J.P., Marinero, J.C.F., Gordaliza, A., 2020, Influence on the user's emotional state of the graphic complexity level in virtual therapies based on a robot-assisted neuro-rehabilitation platform, Computer Methods and Programs in Biomedicine, 190, 105359.

Lamsa, J., Mannonen, J., Tuhkala, A., Heilala, V., Helovuo, A., Tynkkynen, I., Lampi, E., Sipilainen, K., Karkkainen, T., Hamalainen, R., 2023, Capturing cognitive load management during authentic virtual reality flight training with behavioural and physiological indicators, Journal of Computer Assisted Learning, 5(39), pp. 1553-1563.

Wang, Y., Weng, T., Tsai, I., Kao, J.Y., Chang, Y.S., 2023, Effects of virtual reality on creativity performance and perceived immersion: A study of brain waves, British Journal of Educational Technology, 54(2), pp. 581-602.

Xu, X., Wang, F., 2022, Engineering Lab in Immersive VR-An Embodied Approach to Training Wafer Preparation, Journal of Educational Computing Research, 60(2), pp. 455-480.

Pletz, C., Zinn, B., 2020, Evaluation of an immersive virtual learning environment for operator training in mechanical and plant engineering using video analysis, British Journal of Educational Technology, 51(6), pp. 2159-2179.

Baceviciute, S., Lucas, G., Terkildsen, T., Makransky, G., 2022, Investigating the redundancy principle in immersive virtual reality environments: An eye-tracking and EEG study, Journal of Computer Assisted Learning, 38(61), pp. 120-136.

Sipatchin, A., Wahl, S., Rifai, K., 2021, Eye-tracking for clinical ophthalmology with Virtual Reality (VR): A case study of the HTC vive pro eye's usability, Healthcare, 9(2), 180.

Armougum, A., Orriols, E., Gaston-Bellegarde, A., Joie-La Marle, C., Piolino, P., 2019, Virtual reality: A new method to investigate cognitive load during navigation, Journal of Environmental Psychology, 65, 101338.

Clark, L.D., Bhagat, A.B., Riggs, S.L., 2020, Extending Fitts' law in three-dimensional virtual environments with current low-cost virtual reality technology, International Journal of Human-Computer Studies, 139, 102413.

Yan, S., Tran, C.C., Chen, Y., Tan, K., Habiyaremye, J.L., 2017, Effect of user interface layout on the operators' mental workload in emergency operating procedures in nuclear power plants, Nuclear Engineering and Design, 322, pp. 266-276.

Akyeampong, J., Udoka, S., Caruso, G., Bordegoni, M., 2014, Evaluation of hydraulic excavator Human-Machine Interface concepts using NASA TLX, International Journal of Industrial Ergonomics, 44(3), pp. 374-382.

Emami, Z., Chau, T., 2020, The effects of visual distractors on cognitive load in a motor imagery brain-computer interface, Behavioural Brain Research, 378, 112240.

Callahan-Flintoft, C., Barentine, C., Touryan, J., Ries, A.J., 2021, A Case for studying naturalistic eye and head movements in virtual environments, Frontiers in Psychology, 12, 650693.

Joo, H., Jeong, H., 2020, A study on eye-tracking-based Interface for VR/AR education platform, Multimedia Tools and Applications, 79(23-24), pp. 16719-16730

Lee, H., Taek, C.M., Hwan, M.S., 2020, A study on tracking sensor technology interface for vr/ar visual programming platform, The Journal of Korean Institute of Next Generation Computing, 16(4), pp. 52-57.

Kapp, S., Barz, M., Mukhametov, S., Sonntag, D., Kuhn, J., 2021, ARETT: Augmented reality eye tracking toolkit for head mounted displays, Sensors, 21(6), 2234.

Pastel, S., Chen, C., Martin, L., Naujoks, M., Petri, K., Witte, K., 2022, Comparison of gaze accuracy and precision in real-world and virtual reality, Virtual Reality, 25(1), pp. 175-189.

Wang, Y., Shi, Y., Du, J., Lin, Y.Z., Wang, Q., 2020, A CNN-based personalized system for attention detection in wayfinding tasks, Advanced Engineering Informatics, 46, 101180.

Li, H., Fan, L., 2020, A flexible technique to select objects via convolutional neural network in VR space, Science China-Information Sciences, 63, 112101.

Vaughan, N., Gabrys, B., 2020, Scoring and assessment in medical VR training simulators with dynamic time series classification, Engineering Applications of Artificial Intelligence, 94, 103760.

Bao, J., Liu, X., Xiang, Z., Wei, G., 2020, Multi-objective optimization algorithm and preference multi-objective decision-making based on artificial intelligence biological immune system, IEEE Access, 8, pp. 160221-160230.

Moszkowicz, J , 2011, Gestalt and Graphic Design: An Exploration of the Humanistic and Therapeutic Effects of Visual Organization, Design Issues, 27(4), pp. 56-67.




DOI: https://doi.org/10.22190/FUME240122029F

Refbacks

  • There are currently no refbacks.


ISSN: 0354-2025 (Print)

ISSN: 2335-0164 (Online)

COBISS.SR-ID 98732551

ZDB-ID: 2766459-4