Skip to content

Loss Functions¤

mse_loss ¤

mse_loss(input, target)

Mean squared error between predictions and targets.

Source code in minitorch/loss.py 
6
7
8
def mse_loss(input, target):
    """Mean squared error between predictions and targets."""
    return ((input - target) ** 2).mean()

cross_entropy_loss ¤

cross_entropy_loss(input, target)

Softmax cross-entropy. target may be class indices or one-hot rows.

Source code in minitorch/loss.py 
17
18
19
20
21
22
23
24
25
26
27
def cross_entropy_loss(input, target):
    """Softmax cross-entropy. `target` may be class indices or one-hot rows."""
    if target.data.ndim == 1 or (target.data.ndim == 2 and target.data.shape[1] == 1):
        labels = target.data.flatten().astype(np.int64)
        one_hot = np.zeros((input.data.shape[0], input.data.shape[1]), dtype=np.float32)
        one_hot[np.arange(len(labels)), labels] = 1.0
        target = Tensor(one_hot, requires_grad=False)

    log_probs = F.log_softmax(input, axis=1)
    N = input.data.shape[0]
    return (-log_probs * target).sum() / N

bce_loss ¤

bce_loss(input, target)

Binary cross-entropy. Input should be probabilities.

Source code in minitorch/loss.py 
11
12
13
14
def bce_loss(input, target):
    """Binary cross-entropy. Input should be probabilities."""
    p = input.clamp(1e-7, 1 - 1e-7)
    return -(target * p.log() + (1.0 - target) * (1.0 - p).log()).mean()