While spatial orientation abilities vary between individuals, a common strategy for gaining understanding of a terrain is the Landmark-Route-Survey method: become familiar with landmarks in the world, join sequences of familiar objects together to learn routes within the world, and finally gain an overall understanding of the entire region (refs). The goal of this experiment was to test wayfinding abilities in a computer-rendered 3D virtual world, to:
In the "Learning" phase, participants were placed above a flat terrain, which contained two teapots sitting on the virtual ground. The purpose of this phase was to give them an opportunity to become familiar with the experiment system, and to gain some practice in driving using the 6DOF mouse interface. To encourage this process, subjects were asked to identify the flavour of tea in each teapot. This task required that they descend to ground level, approach the teapots, and circle them until they could read the labels on one side of each teapot.
Figure 1: Screen shot of "Learning" phase (28K)
Upon completion of the "Learning" phase, subjects proceeded to the "Testing" phase. The purpose of this second phase was to ensure a minimum level of comfort and competency with the interface before subjects were allowed to advance to the main body of the experiment. This phase, therefore, served as a pre-test Driver's Exam; the participation of those subjects who were not able to complete it, within the time limit, was terminated at this point, on the assumption that an extrapolation of their time would exceed the maximum time limit set for the complete study. The "Testing" phase task was to traverse twelve targets, scattered through another flat terrain. Two visual cues were provided, to ease this task. First, the targets were placed progressively father along a blue trail, visible at all times, which ran along the ground. Second, the targets were coloured such that traversing a target with the wall of a certain colour (red) on one side (the right) indicated a proper direction along the path. Only one of the twelve targets was visible at a time, and constituted the "next target"; upon successful traversal, this target would disappear and the subsequent one, farther along the blue trail, would appear. The task tested the ability of subjects to change elevation in the terrain, by varying the height at which the target was raised from the ground by its base. The trail also required that subjects steer through both sharp and gradual turns to both the right and the left.
Figure 2: Screen shot of start of "Testing" phase. the first target is directly ahead. (28K)
The remainder of the experiment is conducted in a three-dimensional terrain, consisting of hills, plains, valleys, and mountain passes. The terrain is coloured according to height, to better distinguish landscape elements from one another. Ten scattered locations within the terrain are identified with uniquely coloured boxes, or stations. These stations are slightly larger than the targets of the Testing phase, and are set just above the ground. Half of the stations are exclusively Start stations, points of departure, from which trials are begun. The other half are End stations, destinations at which trials conclude. The task in each of the remaining phases is to drive from one Start station to one End station. Every trial required at least one and as many as three turns of at least 90 degrees. The ten stations are visible at all times, thereby becoming additional landmarks to the terrain, by which the subjects can orient themselves. The height above the terrain to which subjects can ascend is limited to approximately 75% of the height of the tallest mountain in the landscape.
The "Following" phase introduces the participants to this landscape and the stations. The task is to follow a pilot vehicle along a set of paths through the virtual world, travelling from a Start station to an End station in another part of the terrain. Upon completion, the next path is loaded, and the subject is instantly taken to the next Start station. Familiarity with the terrain is developed through repeated visits to the same plains, mountain passes, and End stations. Subjects are also asked to observe the specific path travelled by the pilot vehicle, since they will subsequently be asked to duplicate it without the benefit of a pilot vehicle. They are also asked to observe the terrain as a whole, to be able to find new paths through it on their own. For this phase, each subject is randomly assigned to one of two main terrain learning conditions, and to one of two sub-conditions within each main division. The main conditions are whether the subject is required to actively steer along the path of the pilot vehicle, or whether they are passively towed along at a fixed distance. Within the Passive condition, subjects may be given the steering interface as additional visual cues to the action, akin to watching the driver from the Front seat, or these cues may be absent, akin to sitting in the Back seat, watching the terrain go by. Within the Active main condition, subjects must steer behind the pilot vehicle, but are not given complete freedom in the terrain. Their location and position relative to the pilot vehicle is monitored, and they are prevented from wandering too far from the correct path, or from falling too far behind or passing the pilot vehicle. When subjects exceed these invisible boundaries, their heading is adjusted to match the correct path, and they begin to have their velocity reduced; persisting in leaving the correct path results in coming to a complete halt until corrective action is taken. The sub-conditions for Active steering are whether these constraints are Tight (maximum deviation of 10% of the terrain before intervention) or Loose (maximum deviation of 20% of the terrain before intervention).
Figure 4: Screen shot of part of the terrain. 3 stations are in view. (36K)
In the "Repeating" phase, subjects are asked to duplicate the paths traversed in the "Following" phase. The pilot vehicle and all constraints are removed, and all subjects must do the driving themselves. The computer records their times and scores each trial on the basis of how closely subjects followed the correct path, as well as how much of the total trial time was on the correct path.
Finally, in the "Wayfinding" phase, subjects are given a set of trials from known Start stations to known End stations, but for which they have not been shown an explicit path. They are then asked to demonstrate their knowledge of the entire terrain by creating their own paths between the points of departure and destination. By this point, subjects have seen each Start station and each End station at least four times, with the number of other incidental encounters dependent upon the frequency with which individual subjects looked around, and in which parts of the terrain.
To complete these phases, subjects were scheduled for a single session, two hours in duration. After a brief introduction to the experiment and the FledermausVR system, they advanced through the five phases described above. At the conclusion of the study, subjects completed a brief questionnaire, assessing their subjective reactions to the tasks from the condition in which they were placed. Subjects were also asked to locate the ten stations on a coloured map of the terrain, as viewed from above.
All 63 participants completed the driver's test. The participation of those subjects whose Driver's Exam completion time exceeded 600 seconds (10 minutes) was terminated at this point, as they were deemed to have not reached an acceptable comfort level with the mouse-based interface. This arbitrary cut-off was not strictly enforced; if the session administrators observed the necessary skill level in the subjects by the end of the test, then at their discretion they could request that the subjects retake the Driver's Exam. This happened on two occasions, and in both instances the subjects passed the retest by a wide margin.
This process of selection allowed fifty subjects to advance to the final three phases; the Passive condition was assigned 25 (12 in the Backseat, 13 in the Frontseat conditions), and 25 in the Active conditions (12 with Tight constraints, 13 with Loose constraints). Six more subjects were eliminated from the final analyses; one subject refused to complete the experiment, and five others, all of whom were over 570 seconds in the Driver's Exam, were consistently outliers in the data. This left 44 subjects, half in each of the main conditions, and 11 in each of the four sub-conditions. A single-factor analysis of variance (ANOVA) was performed on the Driver's Exam completion times of these remaining subjects, to determine if there was a significant difference between the conditions (see Table 1) and as desired, none was found (F(1,42) = 0.2503, p = 0.619). There was likewise no significant difference in the average distances travelled (F(1,42) = 1.91, p = 0.174). It is therefore assumed that a reasonable distribution of skill levels through all conditions was achieved.
Table 1 : Driver's Exam Times by Condition
| Condition | Time (secs) |
|---|---|
| Passive Backseat | 307.02 |
| Passive Frontseat | 378.98 |
| Active Tight | 314.34 |
| Active Loose | 337.36 |
| Passive total | 343.00 |
| Active total | 325.85 |
In the "Repeating" phase, subjects were asked to repeat the paths along which they had followed the pilot vehicle. The individual paths varied in length and complexity, but all were completed in the same sequence by the subjects. The average Trial Completion Times for all subjects in each experiment condition, after removal of outliers, are summarized in table 2. These times are influenced by several factors, some of which vary with each subject, such as the forward velocity at which an individual is most comfortable advancing, individual steering styles, familiarity with the terrain, and amount of time taken for reflection. Some of the other dependent variables are also related to the trial completion times, such as the distance travelled in the terrain during the trial. With these compounding factors, and the high variance and noise in the task, no significant difference was found, either between the active and passive main conditions (F(1,42) = 1.99, p = 0.166) or between all four sub-conditions (F(3,40) = 0.786, p = 0.509). Despite this lack of statistical significance, there is an observed tendency for the Passive conditions to have the faster times on more trials than the Active conditions. This is summarized in Table 3.
Table 2: "Repeating" phase average trial times, by condition
| Condition | Time (secs) |
|---|---|
| Passive Backseat | 106.09 |
| Passive Frontseat | 107.16 |
| Active Tight | 120.84 |
| Active Loose | 135.33 |
| Passive total | 106.62 |
| Active total | 128.09 |
Table 3: Count of fastest averages on "Repeating" phase trials
| Condition | Fastest | 2nd | 3rd | Slowest |
|---|---|---|---|---|
| Passive Backseat | 3 | 4 | 3 | 0 |
| Passive Frontseat | 2 | 5 | 2 | 1 |
| Active Tight | 5 | 0 | 2 | 3 |
| Active Loose | 0 | 1 | 3 | 6 |
| Passive Total | 5 | 9 | 5 | 1 |
| Active total | 5 | 1 | 5 | 9 |
Since the constraints on movement are removed for this phase, subjects are free to travel anywhere in the terrain, exploring the world and looking for the trial's specific End station. They are asked to repeat the paths as closely as they are able, but one wrong turn would lead them onto potentially lengthy detours. The actual paths of the pilot vehicle followed a reasonably direct route, but kept to low elevations, flying through passes rather than over mountains. As a result, the subject's distance travelled could potentially be less than that of the actual path. Since individual velocity and steering style could affect the Trial Completion Times, Trial Distance measures were taken as another means of identifying differences in terrain and path knowledge between conditions. The average distance traveled per trial in each of the conditions is summarized in Table 4. The units listed in the table are consistent with the global coordinate system of the terrain in the computer. The terrain spanned 100,000 units in both the X and Y directions, and 1,200 units of vertical difference. The distance differences are not of statistical significance ( F(3,40) = 0.910, p = 0.444 between all sub-conditions, and F(1,42) = 2.386, p = 0.130 between main conditions). Nevertheless, it is worth noting that the average trial of the Passive condition subjects was nearly 20% shorter than those in the Active condition.
Table 4: Average Trial Distances, "Repeating" phase
| Condition | Distance (units) |
|---|---|
| Passive Backseat | 199,552.46 |
| Passive Frontseat | 228,674.90 |
| Active Tight | 272,126.94 |
| Active Loose | 259,497.75 |
| Passive Total | 214,113.68 |
| Active total | 265,812.35 |
The Actual Path Percent was calculated by determining if the subject's path came within an arbitrary distance from the actual path of the pilot vehicle, for a given trial, and which the subjects were attempting to duplicate. For each point location in the actual path, the subject's path was examined to determine if it fell at any time within a 3,000 unit radius (encompassing a volume approximately equivalent to 1.5% of the total terrain volume). These percentages were then averaged across all trials and for all subjects within each subcondition. Table 5 summarizes these percentages. No statistical significance was found between main conditions (F(1,42) = 0.615, p = 0.437), or within the sub-conditions (F(3,40) = 1.044, p = 0.384). Differences in conditions could be accounted for solely through random variations. It is noteworthy, as summarized in Table 6, that the Active and Tight constraint sub-condition anecdotally appeared to be the best condition for subjects to reproduce exact paths; it was the best or second-best condition for all ten paths.
Table 5: Average Actual Path Percent, by condition
| Condition | % |
|---|---|
| Passive Backseat | 56.91 |
| Passive Frontseat | 57.57 |
| Active Tight | 62.09 |
| Active Loose | 56.42 |
| Passive Total | 57.24 |
| Active total | 59.25 |
Table 6: Rankings of Actual Path Percent for each trial
| Condition | Best | 2nd | 3rd | Worst |
|---|---|---|---|---|
| Passive Backseat | 2 | 0 | 4 | 4 |
| Passive Frontseat | 0 | 5 | 3 | 2 |
| Active Tight | 6 | 4 | 0 | 0 |
| Active Loose | 2 | 1 | 3 | 4 |
| Passive Total | 2 | 5 | 7 | 6 |
| Active total | 8 | 5 | 3 | 4 |
The Subject Path Percent was calculated by determining, for each point location in the trial paths, whether or not a portion of the actual path fell within 3,000 units of it. These percentages were then averaged across all trials and for all subjects within each subcondition. Table 7 summarizes these percentages. No statistical significance was found between main conditions (F(1,42) = 0.008, p = 0.926), or within the sub-conditions (F(3,40) = 0.985, p = 0.409). Differences in conditions could be accounted for solely through random variations. It is noteworthy, however, that the variance among Active condition values was half that of the Passive condition (Active variance = 61.63, Passive variance = 125.96) suggesting greater consistency in path repetition among those in the Active conditions.
Table 7: Average Subject Path Percent, by condition
| Condition | % |
|---|---|
| Passive Backseat | 47.08 |
| Passive Frontseat | 47.79 |
| Active Tight | 50.65 |
| Active Loose | 43.67 |
| Passive Total | 47.43 |
| Active total | 47.16 |
In the "Wayfinding" phase, the subjects were asked to travel from a known Start station to a known End station, but for which they had not previously encountered an explicit path. Since they did not follow a pilot vehicle for these paths, subjects were required to travel as efficiently as possible between the starting and ending points. Since this phase has no "Actual Path" to repeat, the Actual and Subject percentages seen above are meaningless. No statistical significance was found in the Trial Completion Times, either between Passive and Active main conditions (F(1,42) = 0.431, p = 0.515) or between all four conditions (F(3,40) = 1.294, p=0.290). The average trial times by condition are summarized in Table 8. Despite this lack of statistical significance, there is an observed tendency for the Passive Backseat condition to be consistently faster on more trials than any other conditions. This is summarized in Table 9.
Table 8: "Wayfinding" phase Trial Completion Times, by condition
| Condition | Time |
|---|---|
| Passive Backseat | 56.56 |
| Passive Frontseat | 80.47 |
| Active Tight | 72.63 |
| Active Loose | 76.76 |
| Passive Total | 68.51 |
| Active total | 74.69 |
Table 9: Count of fastest average times in "Wayfinding" phase trials
| Condition | Fastest | 2nd | 3rd | Slowest |
|---|---|---|---|---|
| Passive Backseat | 7 | 2 | 1 | 0 |
| Passive Frontseat | 0 | 1 | 2 | 7 |
| Active Tight | 0 | 3 | 6 | 1 |
| Active Loose | 3 | 4 | 1 | 2 |
| Passive Total | 7 | 3 | 3 | 7 |
| Active total | 3 | 7 | 7 | 3 |
Since one purpose of the "Wayfinding" phase was to test knowledge of the global relationships within the terrain as a whole, the Trial Distance measure is a less noisy and more helpful dependent variable for assessing this hypothesis. A shorter path (using the same units as in the "Repeating" phase) from Start station to End station suggests a better understanding of the layout of the entire terrain. While there was no statistical significance between main conditions for average path distance (F(1,42) = 0.772, p = 0.385), statistical significance was found between the individual sub-conditions (F(3,40) = 3.504, p = 0.024). The average trial distances are listed in Table 10, below. Those subjects in the Passive Backseat condition had the shortest average trial distance in this phase. The second shortest average, somewhat surprisingly, belonged to the Active condition with Loose constraints. These results are echoed in Table 11, which summarizes the number of trials for which given conditions had the shortest to the longest average distance.
Table 10: "Wayfinding" phase Trial Distances, by condition
| Condition | Distance |
|---|---|
| Passive Backseat | 133,392 |
| Passive Frontseat | 222,599 |
| Active Tight | 254,551 |
| Active Loose | 158,553 |
| Passive Total | 177,996 |
| Active total | 206,552 |
Table 11: Ranking distances in "Wayfinding" phase trials
| Condition | Shortest | 2nd | 3rd | Longest |
|---|---|---|---|---|
| Passive Backseat | 7 | 3 | 0 | 0 |
| Passive Frontseat | 0 | 2 | 4 | 4 |
| Active Tight | 0 | 1 | 4 | 5 |
| Active Loose | 3 | 4 | 2 | 1 |
| Passive Total | 7 | 5 | 4 | 4 |
| Active total | 3 | 5 | 6 | 6 |
The definition of "actual path" is meaningless in this phase, since at no time did the pilot vehicle show a path for any of these trials. Nevertheless, these paths were longer than those of the previous phase, by virtue of the greater distance between Start and End stations in each trial; the average distance was 91,302 units in this phase, compared to only 79,903 units in the previous phase. That some amount of terrain learning over the course of the study occurred is evident in the shorter average trial distances in each sub-condition for the "Wayfinding" phase over the "Repeating" phase (compare the distances in Table 10 against those in Table 4 above). The improvement of the average trial distance between the "Repeating" and "Wayfinding" phases was statistically significant (F(1,86) = 4.122, p = 0.045). Anecdotally, 29 of the 44 subjects had a shorter average trial distance in this phase than in the previous one.
The experiment was run in two distinct blocks of time. The first wave of subjects participated in the spring and early summer, and the second half volunteered during the autumn academic term. Subjects in the first wave were required to sketch freehand a map of the terrain and the location of the stations. These results were deemed to be unsatisfactory for providing insight into the mental model from which the subjects were working, so the task was changed for the second wave of subjects. Instead of sketching the terrain, subjects were given a map of the terrain, viewed from directly above the landscape, and coloured to match the world through which the participants had been traveling. The map was placed on a cork board, and subjects were asked to place 10 push-pins, one labeled for each of the stations in the terrain, where they believed the stations to be located. Since only the second wave of subjects completed this map, only 25 maps are available for analysis. The error measure was the number of millimeters by which the placement of a coloured push-pin deviated from its correct location. Table 12 summarizes the total error values by condition.
Table 12: Sums of Errors in Map-based Pin Placement task (in mm)
| Condition | Source Error | Destination Error | Total Error |
|---|---|---|---|
| Passive Backseat | 205.8 | 126.1 | 331.8 |
| Passive Frontseat | 178.5 | 131.0 | 309.5 |
| Active Tight | 375.7 | 306.4 | 682.1 |
| Active Loose | 200.2 | 165.3 | 365.5 |
| Passive Total | 192.1 | 128.5 | 320.7 |
| Active total | 281.2 | 230.4 | 511.6 |
There was significance in the differences in total map error scores between the main conditions (F(1,23) = 4.717, p = 0.040), indicating subjects in the Passive main condition fared better than those in the Active main condition. Statistical significance was also found between sub-conditions in the overall placement of the stations (F(3,21) = 4.935, p = 0.0095), with the Active condition with Tight constraints being the worst by far. It was intuitively anticipated that subjects would score a smaller error in placing the End stations, since these were destinations they would be seeking, whereas the Start stations are sources, which subjects need otherwise only encounter incidentally. This anticipation held true, as 19 of the 25 completed maps had less error in placing the destinations than the sources. Even with the greater difficulty in placing the Start stations, statistical significance was still found between sub-conditions (F(3,21) = 3.502, p = 0.033), but not between the Active/Passive main conditions (F(1,23) = 2.819, p = 0.107). The superiority of the Passive condition subjects at placing the End stations was also found (F(1,23) = 5.482, p = 0.028), and there was also a significant difference between sub-condition, again with the Active and Tight sub-condition by far the worst (F(3,21) = 4.362, p = 0.015).
Figure 5: Start and End Errors by Condition. (8K)
When asked to complete known paths in the "Repeating" phase, subjects in different conditions did not show a significant difference in Trial Completion Times. It was expected that, having had more experience with the interface, and having actively steered along the paths previously, the Active main condition would show better trial times in this phase. If anything, the results showed a tendency for the opposite to occur, with the Passive conditions having the faster average times (see Tables 2 and 3 above). It is possible that this is indicative of the attention of the Active subjects being wholly absorbed in the task of following the pilot vehicle in the "Following" phase, to the detriment of becoming familiar with the direction of the actual path. Passive subjects, on the other hand, were able to observe in greater detail the location of turns and the general direction of, and local landmarks along the paths. Several factors which impact the trial completion times include the subject's propensity to pause and think before acting, their velocity of choice, their familiarity with the terrain and their individual driving methods and styles. The complexity of the task combined with the complexity and size of the virtual environment also contributed to the noise of this measure.
The complexity of both the terrain and the tasks may have contributed to the lack of statistical significance found between main conditions in both the "Repeating" and "Wayfinding" phases. The terrain (see the map in Figure 6) contains two primary plains, located in the north and the south, and two secondary plains located on the eastern edge (all directions are with respect to the map as shown). One significant mountain range bisects the landscape by running west to east, and another by running north to south. Several less significant mountain ranges, such as the southern highlands and the western mountains, further divide the terrain, adding more complexity to the land and complication to the tasks.
Figure 6: Terrain Map ( 384 K)
The Trial Distance measure is a better indicator, than the trial time, of the knowledge of the terrain as a whole, and the relationships among the stations, based upon the assumption that shorter trial distances are indicative of fewer wrong turns or time spent aimlessly wandering. This distance was calculated as the sum of the distances in 3-space between individual point locations along the trial path. While no statistical significance was found with this measure in the "Repeating" phase, and the null hypothesis cannot be rejected, the results again were suggestive of greater terrain knowledge among those in the Passive conditions, as their average distance was almost 20% shorter than that of the Active conditions.
The best indicators for reflecting subject knowledge about the specific paths they were asked to repeat were the two percentages of path overlap. The Actual Path Percent reflects the amount of the total actual path (that of the pilot vehicle) that was covered at some point by the participant; if they followed it exactly, they would score 100%; they would also achieve 100% if they followed the first half correctly, deviated temporarily, and returned to cover the rest of the path correctly. This measure was quite close across all conditions, but there was evidence of a trend indicating greater knowledge of the specific details of the paths among subjects in the Active conditions; they scored the best average percentages on 8 of the 10 trials.
The Subject Trial Percent looked at the issue of path correctness from the other perspective; it represents the amount of the subject's trial path which coincided with the actual path of the pilot vehicle. If they followed the actual path exactly, they would score 100%. On the other hand, if they followed the first half correctly, deviated temporarily, and returned to properly cover the rest of the path, their score would be the amount of actual path covered, over the subject trial path. As with the Actual Path Percent, the numbers were too close for significance (see Table 7 above). There are indications of greater consistency in path knowledge in the Active conditions, however, since their variance was half that of the Passive conditions.
The "Wayfinding" phase, in which subjects were asked to find paths between Start and End stations based solely on their knowledge of the terrain, again failed to reject the null hypothesis for Trial Completion Times. One sub-condition was clearly faster than the rest, however; the Passive Backseat subjects were 15 seconds faster than the average completion times for all subjects, and had the fastest or second fastest times on 9 of the 10 paths. One apparent anomaly in the trial time data occurs within the Active sub-conditions. The average time within the Active Tight sub-condition was slightly faster than that of the subjects in the Active Loose sub-condition (see Table 8), yet the former one frequently ranked lower than the latter condition in the individual trials (see Table 9). This is the result, primarily, of a concentration of poor results in a small number of trials in the Loose condition, skewing the results somewhat. By omiting the worst trial time in this phase for every subject, the main change is in the count of fastest averages. Ignoring the worst trials would change Table 9 above to Table 13, which more closely mirrors the average times by condition.
Table 13: Revised Count of fastest average times on "Wayfinding" phase trials
| Condition | Fastest | 2nd | 3rd | Slowest |
|---|---|---|---|---|
| Passive Backseat | 7 | 2 | 1 | 0 |
| Passive Frontseat | 0 | 0 | 2 | 8 |
| Active Tight | 2 | 3 | 4 | 1 |
| Active Loose | 1 | 5 | 3 | 1 |
| Passive Total | 7 | 2 | 3 | 8 |
| Active total | 3 | 8 | 7 | 2 |
The Trial Distance measure revealed a significant difference in the "Wayfinding" phase, between the four sub-conditions. The subjects in the Passive Backseat sub-condition showed clearly superior knowledge about the relationships within the terrain, and were both faster and more efficient in their finding of new routes through the terrain than the other conditions. It was surprising to see the Active Loose condition improve so significantly in this phase; while their trial completion times were similar to those for the Active Tight condition, they were clearly more efficient in this final phase at the wayfinding tasks. Likewise, the Passive Frontseat condition was surprisingly poor in its performance during this phase.
As mentioned previously, the process for glimpsing the mental maps of the subjects was altered midway through the study. The two map-based activities were attempts to record the mental picture the subjects had of the terrain. They tested the level of their recall of the relationships between the stations and the landmarks, and forced the subjects to explicitly unify their associations, if they had not to that time. The initial method was to provide the subjects with a blank sheet of paper and require them to sketch freehand the general layout of the terrain, with particular emphasis on the location of the stations. This initial method had the advantages of allowing the participants the freedom to customize their maps, thereby making the results more truly their own, and likely better reflecting their mental conceptions. This method was discarded after the first half of the study, however, due to its inadequacies. The subject-drawn maps varied wildly in size (from thumbnail-sized to a full page), scale (variation between maps and often inconsistent within a given map), detail (some had no context information of mountains and plains), and completeness (some maps were complete, others lacked one or more station, a couple held no labels for the stations that were drawn). Since the usefulness and clarity of the data varied, it was not possible to extract useful and easily measurable data.
note to Kelly & Colin: I have several interesting examples that could be scanned in if desired
At the mid-point of the study, the map task was changed: at the conclusion of the "Wayfinding" phase, subjects were provided with a 2-dimensional map of the terrain, minus the stations. The map was coloured to match the colour variation of the terrain, to facilitate recognition. The map was presented to them on a cork board, and subjects were asked to place labeled push-pins in the appropriate locations, according to their understanding of the overall layout of the terrain. This created a consistency across all subjects, making the size, scale, shape, and orientation of the map homogeneous. Subjects were more likely to consider the locations and relationships for all 10 stations, and place them all, rather than omitting any they did not know. These factors created maps which were more quickly and easily assessed subjectively for rough accuracy. It also created a standard error measure: the deviation (in millimeters) of the subject's placement of a given station to its actual location.
The map task provides the clearest evidence of greater overall terrain knowledge in the Passive conditions than in the Active ones. Subjects in the Passive main condition were significantly better at placing the End stations, and at placing all the stations, than their counterparts in the Active main condition. This may be the result of being able to look around and tour the terrain behind the pilot vehicle, while the Active subjects were absorbed in steering, taking less time to relate objects in the terrain to one another. This is further supported by the significant effect between sub-conditions in the placement of the stations, with the extremely poor relative showing of the Active Tight sub-condition. These subjects were so absorbed within their tight constraints that they scored the poorest in the map task by a considerable margin, with their total standard error nearly double that of the next-worst (682.1 to 365.5).
Anecdotally, two interface factors caused difficulty for a majority of subjects. The first was the XY-Translation icon in the mouse-based interface itself. Subjects were encouraged to experiment with all aspects of the flying interface, which set their terrain motion vector by the addition of three components: Z-Translation (forward and backward along the current heading vector), XY-Rotation (rotating the current vector heading horizontally and vertically), and XY-Translation (shifting horizontally or vertically along the same heading). These icons acted upon the coordinate system of the viewpoint in the terrain, relative to the viewer. The most common method of flying through the terrain with this interface was to set some forward motion and then utilize the XY-Rotation icon as a rudder or steering wheel to navigate. This still allowed movement in 3 dimensions, but simplified the driving task. Any subjects who had difficulty relating to the transformations happening in their movements were recommended to adopt this strategy. Most subjects, even without prompting, quickly abandoned the use of the XY-Translation icon. It tended to introduce a motion which initially confused subjects, many of whom incorrectly expected to be able to scan the horizon with it. Instead they altered their position, and as a result would often miss their desired targets altogether.
The second system problem which contributed to difficulties for several subjects was the narrow field of view of the system. With its limited perspective of ?? degrees, the field of view resulted in subjects occasionally missing a target by failing to see it. If, for example, subjects made the correct turn but too slowly, the target might not appear in their view, and they would often conclude they were wrong, and continue to wander, rather than stop in the terrain and rotate in place to find the target.
The 10 targets were scattered throughout the terrain, with several placed in the primary plains, in view of one another, while a few were placed in more isolated corners of the terrain. This mix required the subjects to become reasonably familiar with the terrain to find some of them, with the expectation that destinations in the primary plain would be more easily found than isolated ones. This proved anecdotally true, as the most isolated station proved repeatedly to be the most difficult one to find. Paths ending at this destination in the "Repeating" phase accounted for the worst trial completion times for 21 of the 44 subjects. The ability to find this isolated station appeared to improve by the "Wayfinding" phase, as the two paths ending there in that phase accounted for the worst trial completion time for only 18 of the 44 subjects.
