Fig.1

Scene graph of a certain modern flight cockpit"

Fig.2

Simulation scene graph of the experiment"

Fig.3

Schematic size and positions of the three types of window"

Fig.4

Main menu interface and the three types of window of situation function"

Fig.5

Evaluation system of collaborative operation interface"

Fig.6

Manned/unmanned aerial vehicle collaborative task process"

Table 1

Keystroke interaction and multimodal interaction for experimental task"

窗口操 作行为	任务场景	任务详情	按键交互实验	多模态交互实验
				触控手势	语音	触控手势+语音
切换 窗口	态势	1.查看无人机是否到达任务区域1	单指点击	单指点击	—	—
	自卫	2.找到雷达告警系统	单指点击	—	“打开自卫”+“切换 最大化/标准化”	—
	飞控	3.打开飞控查看无人机起飞， 并切换为最大化	单指点击	—	—	“打开飞控”
移动 窗口	火控	4.查看武器信息，并移到区域4	单指点击		—	—
	机电	5.移到区域3	单指点击	—	“打开机电”+“ 移到区域3”	—
	垂直	6.设置空速，并将其移到区域2	单指点击	—	—	“打开垂直”
关闭 窗口	维护	7.打开维护全屏，再返回主菜单	单指点击		—	—
	协同	8.查看无人机2的载荷后关闭	单指点击	—	“无人机2信息”+ “关闭”	—
	协同	9.查看无人机3和无人机4的 编队信息后关闭	单指点击	—	—	“回到主菜单”

Fig.7

Schematic diagram of the experimental flow"

Fig.8

Boxplot of total task completion time for keystroke interaction and multimodal interaction"

Fig.9

Boxplot of intermodal task completion time"

Fig.10

Heat map of task completion time for each subject under different tasks"

Table 2

Wilcoxon signed-rank test of first fixation duration for keystroke interaction and multimodal interaction"

Fig.11

Boxplot of eye movement metrics data for keystroke interaction and multimodal interaction"

Table 3

Wilcoxon signed-rank test of first fixation duration for voice interaction and keystroke interaction"

Table 4

Wilcoxon signed-rank test of first fixation duration for touch gesture-voice combined interaction and keystroke interaction"

Fig.12

Histogram of intermodal total fixation duration averages"

Fig.13

Histogram of intermodal fixation counts averages"

Fig.14

Histogram of intermodal average pupil diameter averages"

Fig.15

Heat map of fixation counts for each subject under different tasks"

Table 5

Samples of SUS questions"

Fig.16

Boxplot of SUS scores for keystroke interaction and multimodal interaction"

Table 6

Weighting of the six dimensions of the scale"

Table 7

Subjective cognitive load data of keystroke interaction and multimodal interaction"

Fig.17

Histogram of total subjective cognitive load"