Sensor Fusion

[eloise]

What actually is different between F-35's sensor fusion and sensor fusion of Typhoon , Rafale , Gripen ?

[eloise]

I saw this picture before , but to be honest , i dont quite get their point ? isnt it also fused the data ?

[popcorn]

AFAIK, it speaks to the granularity of data being fed into the fusion engine. 4Gens correlate tracks ie. processed data, selecting the most robust, for presentation to the pilot and discarding the others. The F-35 fusion engine OTOH correlates and processes raw unbiased sensor data streams from onboard and offboard sensors/data links, to build the most accurate accurate and comprehensive COP of the battlespace. It also speaks to a tighter integration of the avionics courtesy of those multi-million lines of software code.

4.5Gens build on the 4Gen capabilities and likely fuse data wih finer granularity. But apparently this still falls far short of what the 5Gens are capable of. Recall the RAF Wing Commander Townsend, an experienced Typhoon pilot who got his Saul-on-the -road-to-Damascus epiphany during his first encounter with F-35 sensor fusion.



viewtopic.php?f=61&t=28108..Question: How would characterize the role of the F-35 compared to the other elements in the evolving RAF air combat force?
Group Captain Townsend: The F-35 is not a multi-role fighter. Multi-role, in current thinking, would be a sequential series of tasks. The F-35 is doing a number of missions simultaneously. The concept of mission simultaneity is really important.

The airplane has the ability to do things without the pilot asking it to do it. Automatically conducting, particularly, ISR whilst it’s conducting an OCA mission or an attack mission in a very different way than platforms have done business in the past. This is something that other operators are working in the package alongside F-35 need to understand.

That the F-35 operator won’t be going through sequential thought process. He will be thinking about the battle space in a broader sense, a much different way than a Typhoon operator would be thinking about the battle space.

I think there is another step change and difference in the way in which the information is displayed to the pilot which is important and is extremely intuitive. I’ll give you an example. I commanded a Typhoon squadron for two years.

Very early on this job with F-35, I was lucky enough to fly the F-35 simulator. and the different way in which F35 displays information compared to Typhoon is eye-catching. In fact, I asked for the simulator to be stopped because I was taken aback by the information being displayed to me. There was just so much data available at my fingertips, but displayed in a really different sense in Typhoon.

So very, very quickly, I knew a great deal about the entity being targeted – sensor fusion at work. I think it’s a very different way of displaying information that any other fast jet has done before.

Knowing what my wingman is seeing and my wingman knowing what I am seeing, and my ability to communicate what I want to have achieved by my formation, by my package, which all may be by the air wing that’s air-borne at the time. This airplane changes the game in a way which we can conduct that sort of business.

[spazsinbad]

Thanks 'popcorn' - I was going to ask 'eloise' how the SLDinfo came/was read? As I recall there were SLDinfo SLIDES in one package and then another package had explanations for said slides that would be helpful for 'eloise' to ponder. Whilst LM 'White Paper' PDFs explained more from the interface designer Mike Skaff viewpoint. As for the 'other aircraft slides' it just looks like mumbo-jumbo to me.

[eloise]

AFAIK, it speaks to the granularity of data being fed into the fusion engine. 4Gens correlate tracks ie. processed data, selecting the most robust, for presentation to the pilot and discarding the others. The F-35 fusion engine OTOH correlates and processes raw unbiased sensor data streams from onboard and offboard sensors/data links, to build the most accurate accurate and comprehensive COP of the battlespace. It also speaks to a tighter integration of the avionics courtesy of those multi-million lines of software code.

4.5Gens build on the 4Gen capabilities and likely fuse data wih finer granularity. But apparently this still falls far short of what the 5Gens are capable of. Recall the RAF Wing Commander Townsend, an experienced Typhoon pilot who got his Saul-on-the -road-to-Damascus epiphany during his first encounter with F-35 sensor fusion.

so what you saying is the 4 gen sensor fusion will only display the information from the sensor that seem to have most information while the F-35 computer will try to use information from all sensor to track the target ?

anyway here is Rafale sensor fusion according to dassault

1 – “Multi-sensor data fusion”

Implementation of the “multi-sensor data fusion” into the RAFALE translates into accurate, reliable and strong tracks, uncluttered displays, reduced pilot workload, quicker pilot response, and eventually into increased situational awareness.

It is a full automated process carried out in three steps:

Establishing consolidated track files and refining primary information provided by the sensors,
Overcoming individual sensor limitations related to wavelength / frequency, field of regard, angular and distance resolution, etc, by sharing track information received from all the sensors,
Assessing the confidence level of consolidated tracks, suppressing redundant track symbols and decluttering the displays.

Here is Typhoon sensor fusion

"The sensor fusion process produces a unique track of a single target which may be reported by several sensors simultaneously, each one providing a subset of target attributes which are compiled to produce an as complete as possible view of the target," Friemer says. Algorithms weigh the reliability of each report before merging them to produce a fused target identity and priority.

[charlielima223]

I'm am still a bit "hazy" about "sensor fusion" for 4.5gen aircraft. I get it with the F-35 but what I get hazy about is when it comes to F-22. (To me) The Raptor's avionics and sensors are 2nd to the Lightning but, before I used to hear and read a lot about the Raptor's integrated avionics (to me now is just called sensor fusion) and how it was such a game changer. I remember reading and hearing about its debut appearance at Northern Edge Alaska. How the combination of stealth and integrated avionics put the Raptor head and shoulders above the rest. So is the Raptor just a gen 4.5++ sensor fusion/integrated avionics wrapped up in a stealth platform?

[popcorn]

From the Rafale info ....Assessing the confidence level of consolidated tracks, suppressing redundant track symbols and decluttering the displays. ..I wonder why there would be redundant track symbols to begin with? Seems like it chose among several tracks the one with the highest confidence. Sounds like fusion at the track level. AFAIK F-35 fusion engine generates a single unified track from scratch.


The Typhoon brochure description seems to point to the use of more granular sensor input similar to descriptions of F-35 Sensor Fusion. This must be weighed against first hand testimony from Typhoon vet Group Captain Townsend of his mind-blowing first experience with the JSF implementation of sensor fusion. Brochures are nice but the proof of the pudding is in the eating.

[SnakeHandler]

I'm am still a bit "hazy" about "sensor fusion" for 4.5gen aircraft. I get it with the F-35 but what I get hazy about is when it comes to F-22. (To me) The Raptor's avionics and sensors are 2nd to the Lightning but, before I used to hear and read a lot about the Raptor's integrated avionics (to me now is just called sensor fusion) and how it was such a game changer. I remember reading and hearing about its debut appearance at Northern Edge Alaska. How the combination of stealth and integrated avionics put the Raptor head and shoulders above the rest. So is the Raptor just a gen 4.5++ sensor fusion/integrated avionics wrapped up in a stealth platform?

Yep. Even the F-117 was a Frankenstein collection of many different parts of aircraft underneath the skin. Hornet engines (non afterburning), Eagle landing gear, etc. Stealth is only skin deep, you have to love her for what she is on the inside and how she treats you.

All sensor fusion entails is taking all the information you receive from all the different places you can receive it from and putting it together into one coherent picture. "Integrated Avionics" means the same thing to a lesser degree so the pilot still has to process a lot of information from different displays. I still have to look at six different displays in the Viper to collect all the info available to me but the F-35 driver will only need to stare at one drool bucket.

[popcorn]

SLD has a whole series of white papers and slide shows that are very helpful in exploring the impact of sensor fusion as implemented in US 5Gen designs. The analogy between living in a MS-DOS world and todays GUI environment captures some of it. Jeeves doing the housekeeping while the pilot does the cool stuff and gets the girl.

http://www.sldinfo.com/whitepapers/flyi ... cture-cop/


The COP gives us a visual vice verbal approach to generating synergy.
It’s synergy in a picture rather than words.

The focus is on time – which is a precious commodity in tactical employment.
Rather than pilots spending time constructing a mental picture of the battlespace, the F-35 constructs this picture for them and presents it graphically.

The next generation LIVES in a graphical world, and they tend to process tasks graphically.

[eloise]

SLD has a whole series of white papers and slide shows that are very helpful in exploring the impact of sensor fusion as implemented in US 5Gen designs. The analogy between living in a MS-DOS world and todays GUI environment captures some of it. Jeeves doing the housekeeping while the pilot does the cool stuff and gets the girl.

http://www.sldinfo.com/whitepapers/flyi ... cture-cop/


The COP gives us a visual vice verbal approach to generating synergy.
It’s synergy in a picture rather than words.

The focus is on time – which is a precious commodity in tactical employment.
Rather than pilots spending time constructing a mental picture of the battlespace, the F-35 constructs this picture for them and presents it graphically.

The next generation LIVES in a graphical world, and they tend to process tasks graphically.

Thanks for the link so basically what i understand from reading these are :

Rafale sensor fusion : when multiple sensor tracking/detecting a targets , it only use data from sensor with the highest amount of information and ignore everything else
Typhoon sensor fusion : when multiple sensor detecting/ tracking a target , all information from all sensors that is tracking that target will be used to get more accurate track
F-35 sensor fusion : when multiple sensor detecting/ tracking a target , all information from all sensors that is tracking that target will be used to get more accurate track , also difference sensors can also communicate with each otherz to cue , rely track , when a sensor detect something , it will ask others sensors to look at that direction for more information .

[popcorn]

SLD has a whole series of white papers and slide shows that are very helpful in exploring the impact of sensor fusion as implemented in US 5Gen designs. The analogy between living in a MS-DOS world and todays GUI environment captures some of it. Jeeves doing the housekeeping while the pilot does the cool stuff and gets the girl.

http://www.sldinfo.com/whitepapers/flyi ... cture-cop/


The COP gives us a visual vice verbal approach to generating synergy.
It’s synergy in a picture rather than words.

The focus is on time – which is a precious commodity in tactical employment.
Rather than pilots spending time constructing a mental picture of the battlespace, the F-35 constructs this picture for them and presents it graphically.

The next generation LIVES in a graphical world, and they tend to process tasks graphically.

Thanks for the link so basically what i understand from reading these are :

Rafale sensor fusion : when multiple sensor tracking/detecting a targets , it only use data from sensor with the highest amount of information and ignore everything else
Typhoon sensor fusion : when multiple sensor detecting/ tracking a target , all information from all sensors that is tracking that target will be used to get more accurate track
F-35 sensor fusion : when multiple sensor detecting/ tracking a target , all information from all sensors that is tracking that target will be used to get more accurate track , also difference sensors can also communicate with each otherz to cue , rely track , when a sensor detect something , it will ask others sensors to look at that direction for more information .

Yup, to my level of comprehension. All those millions of lines of code and expansive threat libraries are intended to shorten the OODA loop.
The other dimension that is often not appreciated is the impact of tightly linked fusion engnes communicating via stealthy data link that generate a unified Common Operating Picture (COP) of the battlespace. Not only the quantity and quality of displayed information, F-35 pilots see the same tactical picture ie. god's-eye view displayed on their panoramic cockpit display. Game-changing. Probably what dazzled the RAF Group Commander.

[popcorn]

From the CODE ONE thread, more corroboration of the "mind blowing" experience the F-35 bestows, this time fro a Tornado pilot Wing Commander Jim Beck. Bad guys beware, fusion engine allows for wormhole capability..


http://www.codeonemagazine.com/article.html?item_id=182
The system is clearly impressing Beck, who is a former Tornado pilot. “I simply cannot explain to you how good this sensor suite is,” he said. “It is mind-blowing. We don't actually even need to carry a weapon, albeit we can. I can track targets, identify them all, after having turned [nose] cold [away from the targets], then datalink that information to my Typhoons. The Typhoon pilots can then carry their ordnance to bear against the targets.

“So, I’ve identified everything at distances that no one thought previously possible,” Beck continued. “I’ve shared that data with other assets. I can lead them all into the fight. We are very focused on getting value for money and we can do a lot more by blending our assets.

“This jet isn’t just about the weapons — it’s a game-changing capability. The Tornado GR.4 can't just stroll into a double digit SAM MEZ [Missile Engagement Zone]. In the F-35 I can generate a wormhole in the airspace and lead everyone through it. There isn’t another platform around that can do that. This isn’t all about height and supercruise speed — it’s the ability to not be seen,” added Beck.

Waterfall added: “The F-35 is providing the pilot with all the needed information; it is largely irrelevant where that information has come from because the aircraft is manipulating all of the sensors available and taking the best of those sensors, correlating the information and presenting it to the pilot.”

Beck noted: “We can never be explicit about the true capabilities of this jet, we've got to hold our cards close because otherwise people will try to reverse engineer it.

[hornetfinn]

This is my understanding regarding sensor fusion implementations in 4++ gen fighters and 5th gen fighters. AFAIK, all 4++ gen fighters have very similar fusion implementations in principle but exact details might of course be quite different.

First difference is that 4++ gen sensor fusion is what is called track correlation. This means every sensor has to first create a track and then track information is sent to sensor fusion engine where different tracks are correlated and a single track is created by the fusion engine. For example target range might be taken from radar track, angular information (elevation, azimuth) from IRST track and ID information combined from radar (radar NCTR methods) and RWR (radar emissions from target direction) tracks. In 5th gen fighters (at least in F-22 and F-35) the sensors don't create their own tracks at all but rather all the sensor data is fed to sensor fusion engine. Thus sensor fusion engine has access to all the information the sensors generate. Sensor fusion creates a single track from all that data and can use data from sensors that is unavailable in track correlation systems. For example radar (or IRST system) might get infrequent detections which would not be enough to generate or maintain a track. It also generates more information from targets in shorter time as there as time is not wasted in generating tracks with every sensor before sensor fusion gets to work. Downside of 5th gen sensor fusion is that it requires much better network inside the aircraft (from sensors to fusion engine computer) and much more computing power to crunch the data in real time.

Second main difference is that 5th gen sensor fusion is much more autonomous and can truly automatically cue and task all the sensors. This lowers pilot workload and improves situational awareness a lot in complex situations. It also shortens reaction time a lot and can actually do sensor fusion against much larger number of simultaneous targets. For example if RWR detects something in 3 different directions simultaneously, sensor fusion engine can command radar, IRST and IFF systems to probe for more information in very quick order. In 4++ gen systems the RWR would first have to get more information before giving info to pilot who might then have to control all the sensors to do the same. This would take a lot more time and the situation might change drastically during that time.

Third difference is what data can be used for sensor fusion. In 4++ gen systems the sensor fusion uses only data which has been got from sensors (data link is basically a sensor). In 5th gen systems the sensor fusion can (potentially) use much more diverse set of information. It might be able to use things like threat image and dimension data, geographical and spatial data (terrain features, buildings), ATO&ACO level info, weather data etc. It's basically just a matter of having information sources available and software to factor in the information. I doubt all this is currently done or even planned yet. IMO, F-35 sensor fusion in 2045 will have vastly more capabilities than it currently does.

Basically F-35 sensor fusion does things much faster, much more automatically and against many more targets than 4++ gen sensor fusion. That's the reason for the "God's eye of the battlefield" quotes regarding it.

[popcorn]

The Navy's CEC implements sensor fusion (raw radar data) in a similar manner to create a unified track for the AEGIS Combat System.


The Cooperative Engagement Capability*

A revolutionary approach to air defense has been extensively evaluated recently. The approach is a new Cooperative EngagementCapability (CEC) that allows combat systems to share unfiltered sensor measurements data associated with tracks with rapid timing and precision to enable the battlegroup units to operate as one..

The CEC is based on the approach of taking full advantage of the diversities provided by each combatant
at a different location with different sensor and weapons frequencies and features. This approach requires
sharing measurements from every sensor (unfiltered range, bearing, elevation, and, if available, Doppler
updates) among all units while retaining the critical data characteristics of accuracy and timeliness. For
effective use, the data must be integrated into each unit’s combat system so that it can use the data as if it were generatedonboard that unit. Thus, the battle force of units networked in this way can operate as a
single, distributed, theater defensive system...

SUMMARY

The CEC was developed in response to the need to maintain and extend Fleet air defense againstadvanced, next-generation threats as well as to complement advances in sensor and weapon systems. By networking at the measurement level, each unit can view the theater air situation through the collective sensors of the combatants, and units are no longer limited in knowledge of air targets and in missile intercept range by the performance limits of their own sensors. The result is a quantum improvementin which advanced threats may be composite-tracked and engaged using remote data by networked units that would otherwise not have been able to track or engage them. In a 1994 U.S. News & World Report article,1 Rear Admiral Philip Coady,Jr.,Director of Navy Surface Warfare, observed about CEC that “the composite picture is more than the sum
of the parts.”

[popcorn]

Once in a while we get tantalizing glimpses of game-changing capabilities from pilots who have actually undergone the F-35 experience. One can understand the priority given to 5th-to-4th interoperability.. a hundred ti es better? OK, the guy may be exercising poetic license but the message is clear enough.


http://airheadsfly.com/2015/08/25/dutch ... tegration/
Situational awareness in the F-35 is considerably more impressive thanks to the aircraft’s sensitive sensors. Smaal: “Even while I was still preparing for take off, I could already share this information with my formation. On another occasion, four F-16s had a hard time beating a flight of four A-4 Skyhawks. We added just one F-35 to the flight and suddenly became a hundred times more effective.”