Hi,
I am considering building a system around P3D and want to certify the system as an ATD. The FAA requires a "compatibility statement" from the manufacturer of the software.
Per the FAA:
To do so, the manufacturer should obtain a “compatibility statement” from the software developer, which may, at the FAA’s discretion, be used to satisfy this requirement. The following is an example compatibility statement:
“This is to certify that <Name of Software Company or Developer> has demonstrated that the operating system and/or functional kernel(s) <Software part number and version/revision>, is fully compatible with <Name of ATD Manufacturer, Make and Model> and can assure that the communications/transport data latency is not greater than 300 milliseconds all analog and digital input signals meet the performance criteria established for software performance by the ATD manufacturer.”
My question is: Will Lockheed provide this statement? If so, what process do you anticipate?
In general, I am sure that LM knows, based on its minimum computer performance specifications, the latency from a standard USB device (such as a yoke) to the software "reacting" is some specified time (it has to be much less than 300 mS) because when I fly the sim, it responds seemingly instantly - much faster than a real airplane (I'm a flight instructor). Right now, I'm using it as a Cockpit Procedures Trainer (no FAA approval required) and it is just amazing.
Thank you
FAA AATD Compatibilty Statement
-
Legacy Support
- Posts: 474
- Joined: Mon Aug 16, 2010 7:22 pm
Hi,
I have been looking into this for a while and will discuss with a Program Manager on the best way forward and get back to this forum with a step-plan. I might also post some documentation up.
In order for us to do this, we must be in a position to test the system and review the data collected to ensure that the compatibility is confirmed. The manufacturer must prove to us (Lockheed) that the latency is not greater than 300 ms etc. This typically means you will have your own custom hardware and software monitoring application that will measure the latency of whatever system it is under test and provide that data. This will be the same data that you would provide the FAA inspector. Also note that the FAA inspectors love pulling USB cables out and ensuring that the system responds properly and indicates that there is a transport/comms failure flagged and presented to the operator. If they plug it into another USB port, then it must also respond accordingly.
In order for a simulator manufacturer to get the capability statement, they will need to engage with Lockheed Martin to provide the necessary test data for confirmation. It is likely that someone would have to be on-site to do this, so you should budget for that.
Regards,
John
I have been looking into this for a while and will discuss with a Program Manager on the best way forward and get back to this forum with a step-plan. I might also post some documentation up.
In order for us to do this, we must be in a position to test the system and review the data collected to ensure that the compatibility is confirmed. The manufacturer must prove to us (Lockheed) that the latency is not greater than 300 ms etc. This typically means you will have your own custom hardware and software monitoring application that will measure the latency of whatever system it is under test and provide that data. This will be the same data that you would provide the FAA inspector. Also note that the FAA inspectors love pulling USB cables out and ensuring that the system responds properly and indicates that there is a transport/comms failure flagged and presented to the operator. If they plug it into another USB port, then it must also respond accordingly.
In order for a simulator manufacturer to get the capability statement, they will need to engage with Lockheed Martin to provide the necessary test data for confirmation. It is likely that someone would have to be on-site to do this, so you should budget for that.
Regards,
John
Thanks for the response. Let me offer some ideas and comments to consider. First, I do have a concept on how to measure system latency; however, I was hoping that there would be an easier approach. Here's what I mean: Prepar3D, if I consider it a "blackbox" closed system, should have quantifiable data from Lockheed on system latency. Obviously, the "time" it takes for the system (P3D) to output a response is a function of hardware (PC) processing which Lockheed has no control but can be measured based P3D's minimum PC system requirements. In other words, based on your minimum computer requirements, can P3D take an input from an USB device (assuming standard device such as USB 2.0) and deliver a response within the 300 mS ATD requirement? I would say it would be very easy for Lockheed to measure this since you are directly dealing with the P3D source code. For example, using a PC at the specified minimum performance requirements, a step input is made into an axis at time0, at some "recognizable" threshold, a change is noted at time1. The difference in time is the latency. This could be published. I believe that ESP published a conformity statement for someone based around this concept already. I have it somewhere.
I believe USB sends its data at about 250 Hz (I could be wrong) but my guess is that it is fast. In our CPT, there are no accessible USB connectors to "pull" so I'm not sure what an inspector would "pull" to get a notification. I think that such is an irrelevant test. Say I have a pitch potentiometer connected to to USB controller which is faithfully reporting to the PC the pitch control position and then the wiper randomly fails and physically disconnects. The input now floats to some level but the USB just keeps reporting, but no "notification" would be given by the "monitoring system" - it's just a pitch change not a loss of a USB device.
I'm not an expert on the AC, but I can only find the requirement for start-up tests and no requirement beyond the start-up test. After running, the following applies:
"Remain in the approved configuration during the training session. Authorized ATD instruction may not proceed after a malfunction of the ATD system has occurred. The operator must correct the ATD malfunction and repeat the start-up test described in paragraph c of this section before resuming authorized instruction."
As far as I can tell, the AC does not require that the system report the error after start-up. The key words are "...during the training..." and that if a malfunction occurs you can't continue until it's fixed. I also can't find a requirement in the AC that swapping USB ports must be accommodated. I can think of many sims that have no accessible USB cables. I'd be asking the inspector to show me where that requirement exists.
From a customer position, it doesn't seem that the customer should prove to Lockheed that its software has latency less than 300 mS. I'd like to suggest that Lockheed measure and publish the latency based on P3D's minimum PC requirements. I's also like to suggest that your team look at enhancing P3D to have a self-test function at start-up similar to X-plane Professional. This would solve certification barriers, add a dimension to your product line, and minimize a need and costs to send out people an ATD manufacturer to validate test data. It seems that what is said requires that the ATD manufacturer now has to prove the compliance to both Lockheed and the FAA if they want to use P3D. Those are big barriers.
I love the product but I need an easy solution for qualification. Please feel free to correct any misunderstandings that I have or errors in my analysis.
I believe USB sends its data at about 250 Hz (I could be wrong) but my guess is that it is fast. In our CPT, there are no accessible USB connectors to "pull" so I'm not sure what an inspector would "pull" to get a notification. I think that such is an irrelevant test. Say I have a pitch potentiometer connected to to USB controller which is faithfully reporting to the PC the pitch control position and then the wiper randomly fails and physically disconnects. The input now floats to some level but the USB just keeps reporting, but no "notification" would be given by the "monitoring system" - it's just a pitch change not a loss of a USB device.
I'm not an expert on the AC, but I can only find the requirement for start-up tests and no requirement beyond the start-up test. After running, the following applies:
"Remain in the approved configuration during the training session. Authorized ATD instruction may not proceed after a malfunction of the ATD system has occurred. The operator must correct the ATD malfunction and repeat the start-up test described in paragraph c of this section before resuming authorized instruction."
As far as I can tell, the AC does not require that the system report the error after start-up. The key words are "...during the training..." and that if a malfunction occurs you can't continue until it's fixed. I also can't find a requirement in the AC that swapping USB ports must be accommodated. I can think of many sims that have no accessible USB cables. I'd be asking the inspector to show me where that requirement exists.
From a customer position, it doesn't seem that the customer should prove to Lockheed that its software has latency less than 300 mS. I'd like to suggest that Lockheed measure and publish the latency based on P3D's minimum PC requirements. I's also like to suggest that your team look at enhancing P3D to have a self-test function at start-up similar to X-plane Professional. This would solve certification barriers, add a dimension to your product line, and minimize a need and costs to send out people an ATD manufacturer to validate test data. It seems that what is said requires that the ATD manufacturer now has to prove the compliance to both Lockheed and the FAA if they want to use P3D. Those are big barriers.
I love the product but I need an easy solution for qualification. Please feel free to correct any misunderstandings that I have or errors in my analysis.
-
Legacy Support
- Posts: 474
- Joined: Mon Aug 16, 2010 7:22 pm
Hi,
Thanks for the idea, it sounds a lot simpler than what I had in mind! I was not aware of the ESP letter, so if you could send me a PM and if you have a copy that you could email, I can put together a similar statement after confirmation with the dev staff for developers to use. I have it on authority from a recent test, that the FAA inspector asked for controllers to be plugged into different ports and for them to be pulled out to ensure that the test was being done and not fudged. I guess it comes down to individual interpretations.
Regards,
John
Thanks for the idea, it sounds a lot simpler than what I had in mind! I was not aware of the ESP letter, so if you could send me a PM and if you have a copy that you could email, I can put together a similar statement after confirmation with the dev staff for developers to use. I have it on authority from a recent test, that the FAA inspector asked for controllers to be plugged into different ports and for them to be pulled out to ensure that the test was being done and not fudged. I guess it comes down to individual interpretations.
Regards,
John
