🔥 Jadwal & Hasil: deep learning/how to calculate the number of parameters in cnn

Calculating number of parameters in Convolutional Neural Networks (CNNs) is important for understanding model complexity, computational requirements and potential overfitting. Parameters in CNNs are primarily weights and biases learned during training.

Steps to Calculate Number of Parameter in CNN

To calculate the total number of parameters in a 2D convolutional neural network which includes convolutional, fully connected and batch normalization layers while excluding pooling layers as they contribute zero parameters. There steps are:

1. Convolutional Layer

Each convolutional layer has filters or kernels that scan the input image or feature map. The number of parameters depends on:

Filter size (height and width)
Number of input channels
Number of filters (output channels)
Bias term (optional, usually 1 per filter)

Parameters= (k_w \times k_h \times C_{in} +1)\times C_{out}

Where:

k_w = Filter width
k_h = Filter height
C_{in} = number of input channels
C_{out}= number of filters (output channels)

Here the "+ 1" accounts for the bias term for each filter.

Let's consider an example: if a layer has 32 filters of size 3x3 and input with 3 channels (RGB image). Then its parameters will be: (3×3×3+1)×32=(27+1)×32=28×32=896 parameters

2. Fully Connected (Dense) Layers

For a fully connected layer, the number of parameters is given by the number of input units times the number of output units, plus one bias term for each output unit.

Parameters=(\text{input units} × \text{output units})+ \text{output units}

Let's take an example: if the fully connected layer has 128 input units and 64 output units the computed parameters are: (128×64)+64=8192+64=8256 Parameters.

3. Batch Normalization Layers

For batch normalization layers each feature channel has two parameters (gamma and beta).

Parameters=2×\text{num features}

Let's take an example: Suppose that batch normalization layer is applied to an output of 64 channels then parameters can be computed as: 2 x 64 =128 parameters.

4. Pooling Layers

Pooling layers like max pooling, average pooling, etc do not have learnable parameters so they contribute 0 to the parameter count.

5. Combining All Layers

To find the total number of parameters in the CNN sum all the parameters from all the layers calculated above.

Example of Calculating the Number of Parameter in CNN

Consider a simple CNN with the following layers:

Conv layer: 16 filters, 3x3 size, 3 input channels
Conv layer: 32 filters, 3x3 size, 16 input channels
Fully connected layer: 128 input units, 64 output units
Batch normalization after each convolutional layer

Below are the parametrs calculation:

Conv Layer 1: (3×3×3+1)×16=(27+1)×16=448
Batch Norm 1: 2×16=32
Conv Layer 2: (3×3×16+1)×32=(144+1)×32=4640
Batch Norm 2: 2×32=64
Fully Connected Layer: (128×64)+64=8256
Total Parameters: 448+32+4640+64+8256=13440

So, the total number of parameters in this simple CNN example is 13,440.

Parameter Calculation for 3-D Convolutions

For a 3D convolutional layer, the number of parameters depends on the size of the filters (kernels), number of filters and the number of input channels.

\text{Parameters} = (k_d \times k_h \times k_w \times C_{in} + 1) \times C_{out}

Where:

k_d= Filter depth (size along the depth dimension)
k_h = Filter height (size along the height dimension)
k_w = Filter width (size along the width dimension)
C_{in} = number of input channels
C_{out} = number of filters (output channels)

The "+ 1" accounts for the bias term for each filter.

Example of Calculating Number of Parameter in 3-D CNN

Consider a CNN with the following layers:

Conv layer: 16 filters, size 3x3x3, 3 input channels
Conv layer: 32 filters, size 3x3x3, 16 input channels
Fully connected layer: 128 input units, 64 output units
Batch normalization after each convolutional layer