One of the major challenges of learning predictive models for complex tasks is the right hyperparameter and model selection strategy, as state-of-the-art approaches such as grid-search and random sampling require many runs of the learning algorithm, and therefore are usually conducted on large compute clusters rather than resource-restricted platforms such as robots, cars or mobile phones for instance. Therefore, autonomous hyperparameter learning strategies that are able to take into account observations of past hyperparameter performances on related problems have to be developed, enabling learning systems to learn in a fraction of the time it takes today.
In this project, we aim to design a hyperparameter recommendation model that is based on a priori characteristics of a new learning problem and few meta observations and is able to recommend further hyperparameter configurations to test, requiring only a fraction of runs of the learning algorithm, but still delivering comparable performance. This will be accomplished by learning latent features characterizing datasets and models by means of a factorization model in a way to directly correlate hyperparameter performance with these characteristics. Furthermore, active learning strategies will be employed to choose hyperparameter combinations to such an extent that the hyperparameter recommendation model demands even less runs of the learning algorithm.
HyLAP will run for two years from September 2014 - August 2016.