Helmet-mounted displays

[neptune]

Repeated from a "MADL" thread earlier: http://www.f-16.net/f-16_forum_viewtopic-t-23840.html

Lt General Christopher C. Bogdan Program Executive Officer F-35 SASC Written Testimony 24 Apr 2013
[quote.... ..The goggle-based helmet development will continue until we see demonstrated improvement in all of the risk areas of the original helmet and until the government has secured a price agreement with the prime contractor showing significant cost reduction in the original helmet...."

http://www.armed-services.senate.gov/st ... -24-13.pdf (180Kb)[/quote]

Point of clarification; a recent post indicated that the alternative helment was being discontinued and that the issue with the night vision was resolved. These appear to conflict and this statement from yesteday 24 Apr 2013 is very similar to status from 1 year ago.

Question; what is the current status on 25 Apr 2013 of the HMDS for the F-35 program?

[spazsinbad]

I think it is clear that Gen. Bogdan has made a decision to continue with the alternate helmet development which is not ideal by any measure as a night helmet either. IF the HMDS II night vision camera is able to be improved then along with the other improvements and a reduction in price overall for HMDS II it will be good to go? The earlier reports did not include the recent decision (rather than speculation about the decision?). Bogdan says what is at the time - now. However he speculates positively into the future (given these caveats).

"...Additional work still needs to be done to ensure that the program has a night vision camera that is effective for operations as our testing indicated that the current night vision camera is unsuitable for operational use. As risk reduction, the program continues to fund development of a night vision goggle-based alternative helmet solution. The goggle-based helmet development will continue until we see demonstrated improvement in all of the risk areas of the original helmet...."

[neurotech]

I think it is clear that Gen. Bogdan has made a decision to continue with the alternate helmet development which is not ideal by any measure as a night helmet either. IF the HMDS II night vision camera is able to be improved then along with the other improvements and a reduction in price overall for HMDS II it will be good to go? The earlier reports did not include the recent decision (rather than speculation about the decision?). Bogdan says what is at the time - now. However he speculates positively into the future (given these caveats).

"...Additional work still needs to be done to ensure that the program has a night vision camera that is effective for operations as our testing indicated that the current night vision camera is unsuitable for operational use. As risk reduction, the program continues to fund development of a night vision goggle-based alternative helmet solution. The goggle-based helmet development will continue until we see demonstrated improvement in all of the risk areas of the original helmet...."

Is developing two helmets the ideal scenario... probably not. Is it a smart choice given the conditions, without a doubt, Yes! I trust Gen. Bogdan to make smart choices, not necessarily the cheapest choice.

The F136 debate was another example. Sec. Gates was fully confident the F135 wasn't going to be a failure, (eg. A repeat of the TF30 F-14 experience) and had to conserve flight test time. Skip over the political irritation, the decision wasn't really about money, but was about flight test time and resources. The helmet choice is 1 jet and maybe 200 extra flight hours, so its justified from a risk reduction point of view.

I'm familiar enough with the technology involved to know that there is compromises and choices. To dispel a common myth; It isn't the helmet display itself thats the problem. Its the integration with other systems. The accelerometer is very sensitive, and the firmware doesn't always filter out vibration to give accurate positioning. As hinted above, the camera sensors are also a big part of it, and quite honestly, they under spec'd them originally and are playing catch up, these are the smallest, most sensitive cameras ever fielded in an aircraft. They have upgraded them at least twice, and there is another option available but at significant R&D cost.

[mor10]

My undestanding of what Gen. Bogdan said is that all issues except the night vision have been solved. There were some posts here a while back about an improved night vision camera that would be tested on the helmet. Anybody heard any news about the progress of this?

[spazsinbad]

Here is the thing...

Federal Register /Vol. 78, No. 75 /Thursday, April 18, 2013 /Notices 23227

"...(3) The F–35 Helmet Mounted Display System (HMDS) is Secret and contains technology representing the latest state-of-the-art in several areas. Information on performance and inherent vulnerabilities is Secret. Software (object code) is Secret. Sensitive elements include: HMDS consists of the Display Management Computer-Helmet, a helmet shell/display module, a quick disconnect integrated as part of the ejection seat, helmet trackers and tracker processing, day- and nightvision camera functions, and dedicated system/graphics processing. The HMDS provides a fully sunlight readable, biocular display presentation of aircraft information projected onto the pilot’s helmet visor. The use of a night vision camera integrated into the helmet eliminates the need for separate Night Vision Goggles (NVG). The camera video is integrated with EO and IR imaging inputs and displayed on the pilot’s visor to provide a comprehensive night operational capability...."

http://regulations.justia.com/regulatio ... 09094.html

[spazsinbad]

Cleveland Aviation Society February 2012 Newsletter, Pages 4-6

"Sit Back Red Leader. Article supplied by Chris Graham.
From the cockpit of the F-35 Joint strike Fighter, a featureless seascape stretches to the horizon, broken only by the Queen Elizabeth-class aircraft carrier steaming dead ahead. Just above my knees are several iPad-like touch screens, each one displaying flight data, weapon selections and a blur of other information. That barrage of data extends to the canopy, too. Flickering symbols are projected over the vessel in the water and other planes in the air. The aircraft is even talking to me: a well-spoken lady [Nagging Nadia] interrupts my concentration with advice such as "pull up" and, more worryingly, "wing tank empty".

Sensibly, as I've never flown a plane before, the RAF has not entrusted me with control of one of the precious few F-35s yet built (each costing at least £42m - before extras). Instead I'm sitting in BAE Systems' new simulator, a full-scale, multi-million-pound replica of the F-35 cockpit that will be used to train RAF and Royal Navy pilots. It sits in a spherical room at BAE's Warton site in Lancashire, where Britain's cutting-edge military technology is developed behind closed doors and the protection of the Official Secrets Act. I am here to get a bird's-eye view of what 21st-century warfare will look like. And, from where I am sitting, it looks like nothing so much as a computer game. The F-35 will be the most advanced aircraft in the RAF fleet when it comes into service later this decade (flight testing by British pilots is under way in America). But it is not just the performance of the aircraft that is cutting-edge.

The technology that the pilots will be wearing is also more advanced than anything the aerospace industry has produced before. The pilot's helmet, known as the Gen II will make today's systems look positively obsolete. It will give 36o-degree vision, helping to target weapons systems at a glance and identify friend or foe by using animated icons that would not look out of place in a video game. And all of this information will be projected onto the visor of the helmet. It is a development foreseen in Firefox, the novel that was filmed with Clint Eastwood in 1982. In it, Eastwood steals a Russian fighter equipped with weapons that can be fired by thought control. The F-35 does not go quite that far - yet - but it can do the next best thing and respond to voice commands. The hope is that the helmet will make the fighter easier to control and allow the pilot both to concentrate more on flying and react more quickly. The helmet I am wearing works on the same principle, but for the moment the data are instead being projected onto the screen of the simulator (to work accurately, the actual F-35 helmets must be specially built and fitted to each pilot - an expense that the RAF understandably balked at for my fleeting test). Even so, the results are startling.

The system works by stitching together the view from several cameras dotted around the plane to form a complete picture of the outside world. Inside the cockpit are sensors that monitor the position of the helmet. As the pilot moves his or her head, this hardware, called the helmet-mounted display system (HMDS), calculates where he is looking and feeds in the appropriate radar, visual and other imagery. If he sees an enemy target, he can lock on to it using voice commands. A similar system called the helmet mounted symbology system (HMSS) is already used in the RAF's Typhoon, aka the Eurofighter, but there the data are projected onto the cockpit window in head-up-display style. With the F-35, electronics in the helmet display the data directly onto the inside of the visor, so the pilot can scan the land and sky in any direction, not just through the canopy. If, for example, he looks down, he doesn't see his legs but instead is fed images from the camera on the underside of the aircraft, allowing him to see the ground below. But that is not all.

Conventional weapons systems mean that pilots have to point the aircraft in roughly the direction that they want to fire - because they need to get the enemy in their field of view in order to engage the weapons systems. The new helmet and 360-degree sensors allow the pilot to identify an enemy aircraft and engage it, even if it is directly on his tail, without having to violently turn the aircraft. My flight instructor, Steve Long, a former RAF squadron leader and now a BAE Systems test pilot, says: "This simulator is the most realistic I’ve ever been in. The visuals are higher fidelity than any I've seen before; the way it 'flies' is identical to the real airplane, and the way the throttle and stick feel as they move is indistinguishable from the real thing. The only differences are the lack of engine noise, the sounds· and feel of the air conditioning system, and the vibrations of the real thing." I ask if my years of Xbox gaming might finally prove handy. "Definitely.

The 'hands on throttle and stick' controls of a modern cockpit aren't far removed from the control pad of a game console. If you're good at one, it certainly can't hurt at the other." This isn't strictly true. I attempt to land on the carrier, hit the deck and bounce right back into the sky. The tail hook has missed all four of the arrester cables that are supposed to catch the F-35 and pull it to a stop. The screen goes blank - all that is missing is a Game Over message flashing in front of me. Long looks a little crestfallen. I say that I blame the helmet: surely it can help land a plane?

CUSTOM FIT. Each Gen II helmet for the F-35 is custom-made to fit an individual pilot, because the helmet and visor need to remain firmly in place as the flyer moves his head.

SENSORS. Dotted over the helmet's surface is a grid of LEDs. These are tracked by sensors in the cockpit, so the aircraft's computers know exactly where the pilot is looking at all times.

SMART VISOR. The Gen II is the first to use a helmet-mounted display system (HMDS). This moves the head-up display from the cockpit canopy onto the inner surface of the visor. It means that wherever the pilot's head turns, the display remains in his line of sight. What the pilot sees in his visor is a combination of camera feeds and night vision, infrared and radar imagery, together with the identity of any aircraft sharing the skies: It also allows him to "look" through the body of the aircraft at ground targets.

THE FIREFOX FACTOR. Using the HMDS, the pilot can designate an unknown aircraft as hostile by looking at it and issuing a voice command. The F-35 can then suggest a manoeuvre that will result in a "shoot point". Weapons, too, can be fired by voice command alone."

http://homepage.ntlworld.com/phillip.ch ... y-2012.pdf (192Kb)

[count_to_10]

Why is he using a RAF simulator to try and land a 'C on the QE?
Wouldn't a 'B be easier and more appropriate?

[spazsinbad]

Perhaps the date of the article needs to be highlighted. dunnit. At that time the RN/RAF were going F-35C 'Cats'n'Flaps' mode (I kid thee not). But swift turnaround when that turnaround from B to C proved too expensive and back to the B. Problem solved.

The F-35 Lightning II Joint Strike Fighter Standard Note: SN06278 Last updated: 12 April 2013

"..."The [UK] Government announced in May 2012 that after reviewing the costs, risks and technical feasibility of adapting the Carriers to the F-35C, it was reverting to the F-35B STOVL variant...."

http://www.google.com.au/url?sa=t&rct=j ... 5796,d.aGc (PDF 0.26Mb)

[popcorn]

Why is he using a RAF simulator to try and land a 'C on the QE?
Wouldn't a 'B be easier and more appropriate?

Article dated back in the day when CATOBAR was the choice. Now, with the B, he just needs to push a button.

[quicksilver]

...The helmet choice is 1 jet and maybe 200 extra flight hours, so its justified from a risk reduction point of view...

That is just not true. The cost of the alternate path helmet development is substantial -- as in multiple 10s of millions of dollars and several years of work (in parallel with the efforts to correct the POR helmet).

The justification for the effort was that the only other alternative was the addition of a conventional HUD -- which would have extended SDD even further, cost hundreds of millions of dollars in development, and imposed astronomical retrofit costs.

[neurotech]

...The helmet choice is 1 jet and maybe 200 extra flight hours, so its justified from a risk reduction point of view...

That is just not true. The cost of the alternate path helmet development is substantial -- as in multiple 10s of millions of dollars and several years of work (in parallel with the efforts to correct the POR helmet).

The justification for the effort was that the only other alternative was the addition of a conventional HUD -- which would have extended SDD even further, cost hundreds of millions of dollars in development, and imposed astronomical retrofit costs.

Your claim is misleading at best, but basically wrong on the cost and timeframe. I didn't specify hypothetical cost in my previous comment. And I also didn't specify time frame. Can you point to a GAO or JPO or other DoD document to justify your position? You have no justification for your arrogance, and you are mistaken in the cost directly related to the helmet. In future I recommend you loose the attitude.

There is a F-35 dedicated to helmet testing, and flight testing continues, and that is confirmed by JPO. It is also misleading to claim a particular retrofit cost as significant when there is many other concurrency related flixes planned, on various systems, depending on the aircraft batch.

The current issues are not related to the helmet as much as other systems that interface with the helmet. The changes and flight testing have cleared some of the issues, and the night vision acuity limitations is a camera problem, not a helmet problem.

[spazsinbad]

At end of page 19 of this thread 'quicksilver' [ http://www.f-16.net/index.php?name=PNph ... &start=270 ] has a good explanation about the 'latency' issues. Not having seen this sort of thing (HUDs, HMDS) I have to imagine a lot. One thing that occured to me was that during night flying it will be a good idea to NOT move ones head too rapidly nor abruptly due to the Spatial Disorientation issues (SD) that may not always be overcome by relying on what can be seen via HMDS II - OR - the time lag from starting to become disorientated is measurable, until the view via HMDS II mitigates the disorientation for the pilot at night. As mentioned in another thread moving your head slowly at night was one way to alleviate potential SD issues (whereas usually during the day there is no problem) in the A4G or any aircraft in that era.

Perhaps ensuring that the F-35 pilot with current or future HMDS II moves their heads slowly at night (how slow will be probably individual) will also mitigate any HMDS II night vision issues (along with any current fixes/updated cameras in future) etc. The pilots may well do this 'slow head moving' already.

[neurotech]

I'm aware of the concerns over latency, especially for night vision modes. I agree with the suggestion that Spatial Disorientation can be exacerbated by latency.This is not the fault of the helmet itself, but the hardware/software that drives the displays, which is separate from the display itself.

One of the differences between the two helmets is the sensors which determine pilot head position, and this was why the HMDS II jitter caused so many problems. Recent reports indicate the jitter situation is resolved, but the acuitiy issues at night, when dependent on the cameras for night vision display remain a problem, but that will be fixed given due time, money and testing.

I know a few fighter pilots in F/A-18s who consider NVG flying the most dangerous and high-risk non-combat operation, even more risky that the night carrier landing because for carrier landings, you have LSOs to guide them in. Colliding with their wingman, or the ground, with NVGs seems to happen every few years. Assuming all goes to plan, the HMDS coupled to EODAS system will improve Situational Awareness, but not remove the risk of spatial disorientation. Quickly moving the head around can create SD even in normal daylight conditions when flying at unusual attitudes.

http://www.f-16.net/index.php?name=PNph ... 141#234141
I agree with Gums assessment in the post above Quicksilver' mentioned. 150ms latency is too long for Night Vision display.

[quicksilver]

...The helmet choice is 1 jet and maybe 200 extra flight hours, so its justified from a risk reduction point of view...

That is just not true. The cost of the alternate path helmet development is substantial -- as in multiple 10s of millions of dollars and several years of work (in parallel with the efforts to correct the POR helmet).

The justification for the effort was that the only other alternative was the addition of a conventional HUD -- which would have extended SDD even further, cost hundreds of millions of dollars in development, and imposed astronomical retrofit costs.

Your claim is misleading at best, but basically wrong on the cost and timeframe. I didn't specify hypothetical cost in my previous comment. And I also didn't specify time frame. Can you point to a GAO or JPO or other DoD document to justify your position? You have no justification for your arrogance, and you are mistaken in the cost directly related to the helmet. In future I recommend you loose the attitude.

There is a F-35 dedicated to helmet testing, and flight testing continues, and that is confirmed by JPO. It is also misleading to claim a particular retrofit cost as significant when there is many other concurrency related flixes planned, on various systems, depending on the aircraft batch.

The current issues are not related to the helmet as much as other systems that interface with the helmet. The changes and flight testing have cleared some of the issues, and the night vision acuity limitations is a camera problem, not a helmet problem.

If you are unfamiliar with 'alternate path helmet', then you are unfamiliar with the history and current status of HMDS in F-35. My post above stands. Do some research.

[quicksilver]

Try this on for size...

Lockheed Weighs Alternate F-35 Helmet Display
Apr 21, 2011
By Graham Warwick@AviationWeek

Lockheed Martin plans to select the supplier of an alternate helmet-mounted display (HMD) for the F-35 Joint Strike Fighter by the end of this quarter.

Development issues with the primary Vision Systems International (VSI) visor-projected helmet-mounted display have led to the decision to pursue an alternate HMD to provide a night-vision capability. The F-35 does not have a head-up display (HUD), and an HMD capable of day and night operation is a critical requirement (Aerospace DAILY, March 4).

“To ensure we have a viable combat capability, we will have two paths for the HMD,” says Eric Branyan, F-35 deputy program manager, adding that the alternate helmet will use off-the-shelf Anvis-9 night vision goggles and provide both virtual-HUD and targeting information.

In parallel, work will continue on resolving display resolution issues with VSI’s Gen 2 HMD, which uses less-mature electron-bombarded active pixel sensor (Ebaps) technology. VSI says it is looking at more advanced versions of the Ebaps night-vision camera.

A request for proposals was issued in mid-March. “We will select a supplier late in the second quarter and begin to go down the path to develop the alternate helmet,” Branyan says. “We will continue down that path as far as we have to. It’s smart program management.”

“They are looking for an interim solution,” says Paul Cooke, defense avionics business development manager for BAE Systems, which is proposing a version of its visor-projected HMD in production for the Eurofighter Typhoon. BAE was developing an HMD for the F-35 up to around 2005, when VSI was selected.