In this post, you will learn about when to use **categorical cross entropy loss function** when training **neural network** using Python **Keras**. Generally speaking, the loss function is used to compute the quantity that the the model should seek to minimize during training. For regression models, the commonly used loss function used is mean squared error function while for classification models predicting the probability, the loss function most commonly used is cross entropy. In this post, you will learn about different types of cross entropy loss function which is used to train the Keras neural network model.

Cross entropy loss function is an optimization function which is used in case of training a classification model which classifies the data by predicting the probability of whether the data belongs to one class or the other class. One of the examples where Cross entropy loss function is used is Logistic Regression. Check my post on the related topic – Cross entropy loss function explained with Python examples.

When fitting a neural network for classification, Keras provide the following three different types of cross entropy loss function:

**binary_crossentropy**: Used as a loss function for binary classification model. The binary_crossentropy function computes the cross-entropy loss between true labels and predicted labels.**categorical_crossentropy**: Used as a loss function for multi-class classification model where there are two or more output labels. The output label is assigned one-hot category encoding value in form of 0s and 1. The output label, if present in integer form, is converted into categorical encoding using keras.utils to_categorical method.**sparse_categorical_crossentropy**: Used as a loss function for multi-class classification model where the output label is assigned integer value (0, 1, 2, 3…). This loss function is mathematically same as the categorical_crossentropy.

Here is how the loss function is set as one of the above in order to configure neural network. Pay attention to the parameter, **loss,** which is assigned the value of **binary_crossentropy** for learning parameters of the binary classification neural network model.

```
network.compile(optimizer=optimizers.RMSprop(lr=0.01),
loss='binary_crossentropy',
metrics=['accuracy'])
```

When loss function to be used is **categorical_crossentropy, **the Keras network configuration code would look like the following:

```
network.compile(optimizer=optimizers.RMSprop(lr=0.01),
loss='categorical_crossentropy',
metrics=['accuracy'])
```

You may want to check different kinds of loss functions which can be used with Keras neural network on this page – **Keras Loss Functions**.

- First Principles Understanding based on Physics - April 13, 2021
- Precision & Recall Explained using Covid-19 Example - April 11, 2021
- Moving Average Method for Time-series forecasting - April 4, 2021

## Leave a Reply