High-dimensional Empirical Linear Prediction (HELP) was a technique developed by engineers at the National Institute of Standards and Technology (NIST) for reducing time and cost of quality assurance tests. The basic question is: for a product with thousands of characteristics, can we be sure that all the characteristics meet the standard without exhaustively measuring all of them? The answer is yes. Since these characteristics are usually determined by only a few components in the product, they are related to each other and do not have to be measured one by one. Generally the components cannot be directly measured without destroying the product. Therefore we need to measure some of the measurable characteristics, and to save time and cost of measuring the rest of them we will predict their value based on the measurements we have. Also, since the product is usually very complex, it is difficult and time-consuming to build a model relating the characteristics to each other, and small modeling error can destroy the reliability of such a testing procedure. In comes HELP. To overcome the above-mentioned difficulties, the engineers proposed finding a model of the relationship among the characteristics empirically using factor analysis and then build testing procedures based on the empirical model.
HELP technique has potential applications in many other scientific areas. The basic mathematical setup for HELP is: knowing that a high-dimensional vector (say temperature at thousands of different locations over the world, or economic indexes of hundreds of different regions) actually falls into a lower dimension space (because they are related to each other by a few hidden common factors), how can we predict the rest of the vector based on measurements on a subvector of it.
My research work in this area provided theoretical foundation of HELP, and proposed new confidence and prediction intervals procedures that controls the errors of HELP. I am interested in further improving and generalizing the HELP technique to other areas of applications.