Loss Functions¶

This part of the package provides a description and mathematical background of the implemented loss functions. Every loss function can be supplied to salsa subroutines either directly (see salsa()) or passed within SALSAModel. In the definitions below $l(y,p)$ stands for the loss loss function evaluated at the true label $y$ and a prediction $p$ .

HINGE()¶: Defines an implementation of the Hinge Loss function, i.e. $l(y,p) = \max(0,1 - yp)$ .

LOGISTIC()¶: Defines an implementation of the Logistic Loss function, i.e. $l(y,p) = \log(1 + \exp(-yp))$ .

LEAST_SQUARES()¶: Defines an implementation of the Least Squares Loss function, i.e. $l(y,p) = \frac{1}{2}(p - y)^2$ .

SQUARED_HINGE()¶: Defines an implementation of the Squared Hinge Loss function, i.e. $l(y,p) = \max(0,1 - yp)^2$ .

PINBALL()¶

Defines an implementation of the Pinball (Quantile) Loss function, i.e.

$l(y,p) = \left\lbrace\begin{array}{ll} 1 - yp, & \rm{if} \hspace{1mm} yp \leq 1, \\ \tau(yp - 1), & \rm{otherwise} \\ \end{array}\right.$

If PINBALL loss is selected $\tau$ parameter will be tuned by the build-in cross-validation routines.

MODIFIED_HUBER()¶: Defines an implementation of the Modified Huber Loss function, i.e.

$l(y,p) = \left\{\begin{array}{ll} -4yp, & \rm{if} \hspace{1mm} yp < -1 \\ \max(0,1 - yp)^2, & \rm{otherwise} \\ \end{array}\right.$

loss_derivative(type)¶

Defines a derivative of the loss function. One can pass any type of the loss function, e.g. HINGE or an entire algorithm, for instance RK_MEANS().

Parameters:	type – type of the loss function, e.g. `HINGE` or an entire algorithm
Returns:	`Function` which calculates a derivative at the current iterate $w_t$ , subsample $\mathcal{A}_t$ and label $y_t$