How do T&M companies validate the performance of their products?

Jonathan Borrill, director of marketing EMEA for Anritsu, explained that companies can take, in general, two approaches. "The first one is to work against recognised standard sources, which are stable. The other, which is more common, is to work with a reference customer." According to Borrill, most of the work companies such as Anritsu undertake at the leading edge is for particular requirements. "For example, we typically would have a customer designing a device with a noise floor of -120dB, but no device with which to measure it. That means we have to work together." When the work is related to standards, Anritsu will work with lead customers. "The problem with standards, however, is they are often written by a group of people who haven't been through all the details. That means engineers will sit down and interpret the standard in one way, whilst customers will interpret it differently. The question could be 'how do you know it's 300Mbit/s?'. Both sides could have that 300Mbit/s but is it the same?" When the work is non standard, instruments can often be developed in conjunction with a customer who has a particular need to make that measurement. "It could be a non standard dynamic range," Borrill offered, "and the customer could be a university or research institute looking to develop a new technology." Whichever route is taken, there will be the need for the customer, or customers, to make a commitment with resources. "If we come with a new system, it won't be fully debugged, for example. The customer will therefore have to do some verification. So they will be using the new test equipment, but they will also be helping with product development. "In return," Borrill continued, "they will get first access to a device which might give them a market advantage or they may get to define how the product works. But there will still be customers who prefer to buy leading edge equipment 'off the shelf'." Resources, in some circumstances, might just mean manpower. In others, it might mean hard cash. "If a company doesn't have the resources, it might have to wait to buy something off the shelf; a product that works. Others want to push the boundaries, so they will put in the resources because they perceive a benefit – for example, early access to a spectrum analyser with a new range." Trying to match your development back to a national standard might not always work. "When Anritsu started to become involved in mobile phone testing," Borrill recalled, "there were requests to measure error vector magnitude. We looked for national standards, but there weren't any." Whether it's working with lead customers or research institutes, the projects will feed back into Anritsu's own product development process. "We will make a call on whether it's a 'one off' or whether there might be wider application for such a device," Borrill said. "Is it what's coming next, for example, and if so, should we do it? "We'll also determine whether we've seen similar things from other companies. If it's a market trend, it's simple; go away, look at the product and determine the current limitations – is it a platform issue, a component issue and so on. So we might need to develop a new architecture or new components. The front end may need to be redesigned or we might need a new A/D converter with better dynamic range or noise performance." Anritsu can, in some cases, provide the required performance by hand building an instrument. "We may specify the noise floor measurement to ±2dB," Borrill pointed out, "but test the device to ±1.3dB. Some instruments may perform to ±0.7dB and we can hand select components for a particular application." An example of this approach is the development of a vector network analyser (VNA) module for RF measurement. "It has a new architecture to open up access to new specifications," Borrill claimed. "The traditional approach is to down convert; the front end may be at 100G, but the VNA may have a limit of 70G. We changed that by putting the VNA in the head and measuring directly." In some respects, it's down to particular parameters. Anritsu has worked with a standards institute that was interested in greater measurement stability. "We provided a system," Borrill noted, "and this institute set up the evaluation environment, providing us with certification of the performance." The company has also worked with University of St Andrews to develop instrumentation that provided the ability to measure a noise floor of -160dB. "Its millimetre wave group was doing work on volcanoes and atmospheric sensing using a free space optical system," he said. "The work had a theoretical noise floor of -160dB, but it couldn't be measured. So we got together. Traditional test equipment could measure a noise floor of -115dB, but the target was so far off that we needed a new concept." Borrill said that when users are chasing fractions of a dB, the measurement can be verified using standard references, calibrated by bodies such as NPL. "The challenge is when you get into new areas; there's no simple solution. There isn't a standard." Verification of that noise floor was a challenge. "We had nobody who could measure the noise floor, St Andrews had nothing with which to measure it. We needed a third party to verify the measurement." As designers push the boundaries, measurement technologies are needed from other domains. The St Andrews millimetre wave group, for example, develops instruments for applications in the range from 70 to 300GHz. "There's standard RF technology," Borrill noted, "and standard optical technology. St Andrews was bang in the middle, so we tried to bridge the gap by bringing elements from both sides into the equation, while bringing new technology to market." If users want custom measurements from a standard product, both sides need to understand what needs to be optimised. "Frequency and amplitude are what they are," Borrill pointed out, "so we have to work out what parameters can be tweaked. If the customer wants to measure frequency, can we optimise bandwidth or sensitivity? "Absolute speed is quoted all the time, but it's an area where there isn't a standard because it depends on each instrument. We can optimise our instruments to give best performance, but if you can get two or three instruments on their best settings, which one do you choose? While you can say what the sweep time, bandwidth and resolution are, there are too many variables," he concluded.