DC-GAN: Discriminator loss going up while generator loss goes down

Question

I cannot tell if this error is due to a technical mistake or hyper-parameters, but my DC-GAN's discriminator loss starts low and gradually climbs higher, slowing down around 8, whereas my generator loss goes way down. I ended it at about 60,000 epochs. Funny enough, the discriminator accuracy seems to be floating around 20-50%. Does anybody have any suggestions to fix the problem? Any help is appreciated.

Important Info

• Data Format: 472 320x224 Color PNG files.
• Optimizer: Adam(0.0002, 0.5)
• Loss: Binary Cross-Entropy

A generated image after 50,000+ epochs: (Supposed to be a sneaker on a white background)

Discriminator Model:

    def build_discriminator(self):

        img_shape = (self.img_size[0], self.img_size[1], self.channels)

        model = Sequential()

        model.add(Conv2D(32, kernel_size=self.kernel_size, strides=2, input_shape=img_shape, padding="same"))  # 192x256 -> 96x128
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))

        model.add(Conv2D(64, kernel_size=self.kernel_size, strides=2, padding="same"))  # 96x128 -> 48x64
        model.add(ZeroPadding2D(padding=((0, 1), (0, 1))))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(128, kernel_size=self.kernel_size, strides=2, padding="same"))  # 48x64 -> 24x32
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(256, kernel_size=self.kernel_size, strides=1, padding="same"))  # 24x32 -> 12x16
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))

        model.add(Conv2D(512, kernel_size=self.kernel_size, strides=1, padding="same"))  # 12x16 -> 6x8
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))

        model.add(Flatten())
        model.add(Dense(1, activation='sigmoid'))

        model.summary()

        img = Input(shape=img_shape)
        validity = model(img)

        return Model(img, validity)

Generator Model:

    def build_generator(self):

        noise_shape = (100,)

        model = Sequential()
        model.add(
            Dense(self.starting_filters * (self.img_size[0] // (2 ** self.upsample_layers))  *  (self.img_size[1] // (2 ** self.upsample_layers)),
                  activation="relu", input_shape=noise_shape))
        model.add(Reshape(((self.img_size[0] // (2 ** self.upsample_layers)),
                           (self.img_size[1] // (2 ** self.upsample_layers)),
                           self.starting_filters)))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 6x8 -> 12x16
        model.add(Conv2D(1024, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 12x16 -> 24x32
        model.add(Conv2D(512, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 24x32 -> 48x64
        model.add(Conv2D(256, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 48x64 -> 96x128
        model.add(Conv2D(128, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(UpSampling2D())  # 96x128 -> 192x256
        model.add(Conv2D(64, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(32, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("relu"))
        model.add(BatchNormalization(momentum=0.8))

        model.add(Conv2D(self.channels, kernel_size=self.kernel_size, padding="same"))
        model.add(Activation("tanh"))

        model.summary()

        noise = Input(shape=noise_shape)
        img = model(noise)

        return Model(noise, img)

Answer 1

It sounds totally understandable to me that you are having this problem. Your networks are not compensated, the Generator is much more powerful than the Discriminator, in terms of number of neurons. I would try to make the generator and discriminator symmetric each other, in terms of number of layers, their configuration, and their size, that way you assure no one is stronger than the other de facto .

Answer 2

I guess you are using your own Dataset. If not consider using the same hyperparameter as in the DCGAN paper and first reproduce their results. Then use that network for your dataset. DCGANs especially with cross-entropy loss are known to be really tricky and extremely hyperparameter sensitive to get to run (see https://arxiv.org/abs/1711.10337 ). Especially, if you did not constrain your discriminator with some kind of gradient penalties (Gradient Penalty or Spectral norm).

In GAN's using your own ideas for the hyperparameters is mostly a bad idea, unless you either really know what you are doing or have loads of GPU power to burn on a large scale search.