>claimed that by introducing a second kernel to be used on the training data only (e.g. some creative and not necessarily well formalizable annotations by human annotators, ranging from mundane things to assigning poetic qualities to training samples) one can make the error inversely proportional to the number of training samples even when classes overlap with respect to the main

Не в этом ли и состоит регуляционная теория (см. Плохо поставленные задачи, Тихонов А.Н., Арсенин В.Я, 1974)?