Helmet-mounted displays

[spazsinbad]

IF the PDF is under 11Mb then you can attach it to a message on this forum.

Removal Notice

"The document Enhancing HMD-Based F-35 Training through Integration of Eye Tracking and Electroencephalography Technology has been automatically removed by BookID, Scribd’s copyright protection system.

Like all automated systems, BookID sometimes makes mistakes. If you believe this document is not infringing and was removed in error, please contact copyright@scribd.com.

Please see Scribd's Copyright Resource Center to learn more."

[bring_it_on]

Enhancing HMD-Based F-35 Training through Integration of Eye Tracking and Electroencephalography Technology

[spazsinbad]

Great Stuff! Thanks - I am going to read it Intently. Excerpt below from above PDF....

Enhancing HMD-Based F-35 Training through Integration of Eye Tracking and Electroencephalography Technology
05 Apr 2015 Meredith Carroll, Glenn Surpris, Shayna Strally, Matthew Archer, Frank Hannigan, Kelly Hale, and Wink Bennett | Design Interactive, Oviedo, Florida

"Abstract. The ever increasing complexity of knowledge, skills and abilities (KSAs) demanded of Department of Defense (DoD) personnel has created the need to develop tools to increase the efficiency and effectiveness of training. This is especially true for the F-35, the first 5th-generation aircraft to use an HMD as the primary instrument display. Additionally, the F-35 can perform operations previously performed by multiple operators, which potentially places incredible strain on the pilot’s cognitive resources by exposing him to large amounts of data from disparate sources. It is critical to ensure training results in pilots learning optimal strategies for operating in this information rich environment. This paper discusses current efforts to develop and evaluate a performance monitoring and assessment system which integrates eye tracking and Electroencephalography (EEG) technology into an HMD enabled F-35 training environment to extend traditional behavioral metrics and better understand how a pilot interacts with data presented in the HMD....

...The current training program for F-35 transition pilots is 8 weeks long. The transition pilots are comprised of legacy aircraft experts such as experienced F-16 or F-22 pilots. These pilots will become F-35 instructors upon the completion of the program. Training begins with a week of military lectures, followed by 3 weeks of lectures and academic courses specific to the F-35. A pilot training aid (PTA) laptop simulator is flown by transition pilots during these early phases of the course. The last phase of the training program is a mixture of 8-10 F-35 Full Mission Simulator (FMS) sessions and 4-5 actual flights in the F-35. The PTA and the FMS are the two main simulators used in the transition curriculum. The PTA has a large touchscreen monitor that displays both the out-the-window view of the aircraft as well as the touchscreen instrumentation (i.e. Main Forward Display). In addition to the touchscreen monitor, the PTA also has a full replication of the F-35 Hands-On Throttle and Stick (HOTAS). The PTA is mainly used during academic lectures to familiarize the pilot with the controls and procedures for the F-35. An HMD is not used in conjunction with the PTA.

The FMS is a high fidelity flight simulator which contains a full 1-to-1 replication of the F-35 cockpit surrounded by a dome with almost 360 degrees of visual coverage. The pilot trainee is outfitted with an HMD visor that reveals a HUD fixed on the center windscreen. Additionally, a de-cluttered, un-fixed version of the main HUD with a reduced selection of essential symbols (e.g., airspeed, altitude) appears on the HMD when the pilot turns his/her head off bore-sight (i.e., left, right, up, or down). The simulator sessions in the FMS are 1.5 hours in duration and are preceded by a 1 hour pre-brief and followed by a 1 hour debrief. Each trainee in the FMS has the individualized, one-on-one attention of an instructor. The instructor has an operator station where he can launch scenarios and insert abnormal aircraft conditions. During the training session, the instructor can also view the pilot’s performance unfolding from a series of view, including the field of view (FOV) in the cockpit due to a head-tracker associated with the HMD.

The debrief then provides the opportunity for the instructor to playback any flight segment during the simulator session and review notes, exceptional performance, and trainee performance errors. Control inputs, the pilot’s FOV, and other simulator information can be accessed by the instructor to facilitate this debrief. Instructors depend on overt behavioral actions and communications to identify performance errors. One limitation of this approach is the inability of the instructor to determine the specific instruments the pilot is monitoring, both within the HMD and on the MFD. Heads up/heads down status can typically be inferred based on the FOV presented by the HMD, however, the specific information that the pilot is visually integrating is not accessible. Given that a large portion of the task is monitoring information presented by a range of instruments; this limits the instructors understanding of how pilot performance is unfolding....

...4 HMD ASSESS Use Case (pp.28-29) HMD-ASSESS is designed to be utilized during the actual training session and debrief. A use-case was developed to demonstrate the HMD ASSESS concept of operations for F-35 FMS training sessions and is presented in summary in this section.

A typical training session in the FMS may include several abnormal malfunctions from which a pilot must attempt to recover. During this particular training session, the instructor has inserted an Integrated Power Package (IPP) failure into the scenario. As the pilot trainee attempts to recover from the IPP failure, he performs three key errors: 1) the pilot misses a critical checklist item (i.e., arming the backup oxygen system); 2) the pilot spends too much heads down time looking at his checklist and fails to scan his primary flight instruments (altitude, attitude, airspeed) at the necessary intervals; 3) the pilot develops tunnel vision on an area of the cockpit irrelevant to the appropriate task, e.g., determining the best place to land, resulting in a delay in conducting a critical checklist item (i.e., open RAM door).

After the training session in the simulator has ended, the instructor uses the HMD ASSESS after action review displays to facilitate his debrief to the pilot trainee as follows. The instructor is interested in assessing the students handling of the IPP failure, so the instructor clicks on this segment of the timeline and the timeline automatically zooms into the IPP failure event. The instructor points out overall timing summary for that segment to the pilot, including total time heads up vs. heads down and total time in high priority areas. The instructor can illustrate to the pilot trainee that he spent a large amount of time heads down while handling the IPP Failure.

The instructor then clicks on the first system identified error, which automatically zooms the timeline down to a system default of 30 seconds on either side of the error. The instructor plays back the error and points out that, based on the eye tracking data, the pilot was distracted from reading the checklist by focusing on blinking lights on the IPP Panel.

The instructor then moves on to the next error (i.e., breakdown in a periodic eye scan of flight instruments), by selecting the error from the error summary list. The instructor wants to show the pilot how he failed to scan his primary flight instruments frequently enough. By using the Overview mode containing a summary of all eye tracking data for 30 seconds on either side of the error, the instructor illustrates to the trainee that a scan of these three primary flight instruments did not occur during this time period. The instructor confirms this by pointing out the timing summary which shows that the pilot spent very few seconds looking at the altitude, attitude, and airspeed instruments for the specified window of time.

The instructor then points to the section of the timing summary that shows the total time spent on each MFD page for the segment of flight in focus. He uses this data to illustrate that the pilot spent only 30 seconds looking at the navigation page and flight instruments because he started to look for the nearest airport to land too early, instead of following the checklist steps. This caused the pilot to delay in opening the RAM (i.e., air intake) door, which resulted in systems overheating more quickly.

As illustrated in the se case, HMD ASSESS will allow an instructor to more accurately and efficiently diagnose a performance issue. Instructors will be better able to direct a pilot’s attention during overwhelming flight scenarios and prevent pilots from making common mistakes with regard to visual attention allocation...."

Source: download/file.php?id=20448 (PDF 200Kb)

[bring_it_on]

[charlielima223]

I know there is still some minor issues concerning the F-35 HMD. If they can't get all the bugs worked out wouldn't a modified JHMCS be an interim solution?

[bring_it_on]

I know there is still some minor issues concerning the F-35 HMD. If they can't get all the bugs worked out wouldn't a modified JHMCS be an interim solution?

NO. They had a back up helmet in development, that was scrapped because the collective wisdom was that the minor glitches remaining were not problematic enough (or insurmountable) to maintain 2 development programs. Bae is developing the HMD on its own for the export (non F-35) market at the moment.

[spazsinbad]

This is a long thread however starting from here on page 35 one can be up to speed on how HMDS III is travelling (from Jan 2015 - from AvWeak no less) and NOT earlier reports which have been SUPERceded: viewtopic.php?f=62&t=16223&p=284552&hilit=stages#p284552

"Improved helmet-mounted display for F-35 features multiple upgrades.

Eliminating the head-up display and relying instead on a helmet-mounted display (HMD) was only one of the bolder design decisions for the Lockheed Martin F-35. It has also proved one of the more challenging, but a system meeting the requirements is in the final stages of testing. Aircraft hardware for the third generation of the F-35 HMD is being delivered to Lockheed for production Joint Strike Fighters, with the “Gen-3” helmets expected to arrive at pilot training locations..."

[spazsinbad]

In this thread there is mention of 'colour' (color) for the HMDS displays including vHUD AFAIK (I'll find it). Here is mention of 'colour' for a new helmet for USN Seahawk MH-60S Helos - perhaps this is a path for eventual 'color' and movement for HMDS? Here: viewtopic.php?f=62&t=16223&p=221812&hilit=color#p221812
&
viewtopic.php?f=62&t=16223&p=206021&hilit=color#p206021

Perhaps I have misremembered from this article: viewtopic.php?f=62&t=16223&p=274230&hilit=colour#p274230 Anyhoo I have seen only the green symbology in pics of vHUD.

Elbit Systems Integrates Color Helmet Display & Tracking Systems on US Navy MH-60S Seahawk
02 Apr 2015 NavyRecognition

"Elbit Systems announced today that its wholly-owned U.S. subsidiary, Elbit Systems of America, LLC, was awarded a contract from Science Applications International Corporation (SAIC) to install, integrate and support flight evaluations of the Elbit Color Helmet Display and Tracking System (CHDTS) on MH-60S Seahawk test aircraft for the U.S. Navy....

...The program is part of the U.S. Navy's MH-60 Sierra's Improved Targeting System for the Seahawks' Armed Helicopter Weapon Kits. The CHDTS will provide the pilots with both night and day capability to see color flight instrument symbols on their helmet mounted display (HMD) modules. Additionally, the line of sight tracking system enables the pilots to interact with the flight navigation system, improves pilot and copilot situational awareness and can also be used to control pilot or copilot slewable sensor systems. The system also presents a continuously calculated weapon impact symbol for the pilot display, thus increasing system accuracy in the employment of installed weapon systems....

..."The CHDTS will provide the pilots with improved situational awareness through enhanced optics and symbology displayed directly on their HMDs, allowing the pilots to keep their eyes up and out."

Pic: http://www.navyrecognition.com/images/s ... _CHDTS.jpg

"Elbit Color Helmet Display and Tracking System (CHDTS). The program is part of the US Navy’s MH-60 Sierra’s improved targeting system for the Seahawks’ armed helicopter weapon kits. Picture: Elbit"

Source: http://www.navyrecognition.com/index.ph ... ahawk.html

[spazsinbad]

In this thread on page 26 also is more confirmation how the HMDS is tailored to individuals with 'inserts' that allow the HMDS to be standard - but unique temporarily - and can be changed again for another individual: viewtopic.php?f=62&t=16223&p=260571&hilit=color#p260571

F-35: New fighter creates new culture for 21st Century and beyond
24 Sep 2013 Rich Lamance, Air Force News Service

“...When we get our helmet fit, there is actually a complicated scan process that takes an image of our heads and provides a laser cut-out foam insert for the helmet that is molded to our heads. Then there’s ear cups that close the helmet around our head and a custom nape strap in the back that basically locks the helmet down on our heads. There’s very little, if any, motion in the helmet when we move our head around. Very well balanced, a very well fit and it feels great wearing the helmet. It’s very specific to each individual pilot.”

Source: http://www.af.mil/News/ArticleDisplay/t ... eyond.aspx

[spazsinbad]

One part of the main HMDS MAY NOT be interchangeable (along with the custom inner pads etc) is the DISPLAY VISOR. See earlier post of this article for more: viewtopic.php?f=62&t=16223&p=279865&hilit=Mchale+contoured#p279865 & viewtopic.php?f=62&t=16223&p=286207&hilit=Mchale+contoured#p286207

Getting fit for your jet fighter helmet
30 Sep 2014 John McHale

"...Once the measurements are made they begin assembling the helmet. This process includes custom-milling each helmet liner so the helmet sits comfortably on the pilot’s head while maintaining stability under high gravity (G) maneuvers so the optics continue to match up to the individual’s field of view, explains McKillip. “They custom fit the pads in the helmet based on head size,” Gunther says.

Once the helmet is assembled, the pilot comes in for a final fitting on the second day. During this time the optics are aligned to the pilot’s pupils and the display visor is custom contoured – a process that must be done precisely so the pilot has a single focused image at infinity, says McKillip. Many would assume that people’s eyes are aligned, but that is not the case, Gunther says...." [Mine are cross-eyed & painless]

Source: http://mil-embedded.com/articles/gettin ... er-helmet/

[Dragon029]

That's to be expected; most helmet visors are cut to properly fit the pilot's nose ridge and relative position of the oxygen mask. I can't remember what a visor for a JHMCS costs, but it's a drop in the bucket compared to the rest of the helmet.

[spazsinbad]

"...most helmet visors are cut to properly fit the pilot's nose ridge..." aaahh the wonduhs of mod tech eh. Yes I do think the Display Visor would be relatively INexpensive - anyone know? Buehler? Anyone?

[spazsinbad]

Another HELLMUTT story.

Check Out The $400,000 Helmet That Sees Through Walls
27 May 2015 Rob Szczerba

"...what should you expect when your helmet costs $400,000? Well, if you expected it to be magically able to see through walls, you’d be right. It won’t actually give you Superman’s X-Ray vision, but it’s pretty close.

This “super-helmet” is actually the F-35 Gen III Helmet Mounted Display System (HMDS) and it was created to work in synch with one of the most complex weapons systems ever created: the F-35 Lightning II....

...When you build the most impressive (and most expensive) fighter jet in history you need an equally advanced helmet for the pilot, and that technology doesn’t come cheap. The cost of each individual helmet is estimated to be more than $400,000.

Rockwell Collins was subcontracted to build the helmet, which in addition to projecting on the visor critical data, such as altitude, airspeed, targeting, heading, and warnings, also allows the pilot to see what lies beyond the exterior shell of the plane....

...Some of the interesting features of the helmet include:
Biocular, 30×40 degree-wide field of view with 100 percent overlap

Virtual heads-up display

Look through aircraft capability via DAS imagery

High accuracy tracking with auto-bore sighting

Active noise reduction (ANR)

Digital night vision sensor

Ejection capability to 550 KEAS (knots equivalent airspeed, roughly 633 mph)

Lightweight and well balanced

Custom helmet liner for precise fit and comfort

Multiple interpupillary distance (IPD) settings

Video recording

Picture in picture (compatible with eyeglasses and laser eye protection (LEP) devices)..."

Source: http://www.forbes.com/sites/robertszcze ... ugh-walls/

[spazsinbad]

‘We Can Be That Bridge’ Rockwell Collins CEO on the company's reach into the high-tech sector
06-19 Jul 2015 AvWeak

"...QUESTION: There have been a few years of challenges in getting the F-35 helmet set up. It seems things are turning around. Can you give us an update of where you are, and what was the lesson from all of this?

ANSWER: This is the first time we'd ever put an integrated head-up display in the helmet. The dynamics of that, and how the mind reacted to vibrations, surprised us and created some unplanned jittering of the display. We ended up putting a micro-[inertial measurement unit] on the pilot's head. We measure the vibration, and we filter it out with software so the mind doesn't see the different shaking of the display. We've worked through that. We also marked improvement in the brightness and the green-glow effects. So I feel pretty good about the latest generation, and we're making good progress on the cost reduction...."

Source: AVIATION WEEK & SPACETECHNOLOGY/JULY 6-19, 2015

[spazsinbad]

Over on another thread today 'lamoey' viewtopic.php?f=22&t=27549&p=295420&hilit=Cenciotti#p295420 posted pointers to the Cenciotti HMDS II article below. The video is well worth watching to hear pilot talk about the helmet in context. NavAv pilots will welcome the forward night view for flat deck landings at night which has been pointed out a zillion times on this thread. USAF? Dunno. Pilot does not mention vHUD for looking behind but as he says 'should not be necessary' or words to that effect. Also I have read that F-35 missions are more likely to be conducted at night because of all the cool doodads seemingly ignored by this USAF dood.

F-35 pilot talking about the 400K USD flight helmet: “It’s cool but I don’t really use it that often”
13 Jul 2015 David Cenciotti - The Aviationist

VIMEO VIDEO: https://vimeo.com/124614167

Source: http://theaviationist.com/2015/07/13/f- ... ht-helmet/