Transpose-CNN with batch normalization The 2019 Stack Overflow Developer Survey Results Are Inunderstanding batch normalizationhow to calculate the output shape of conv2d_transpose?Several fundamental questions about CNNUnderstand the shape of this Convolutional Neural Networkis Batch Norm a little bit stochastic by default?GANs and grayscale imagery colorizationBatch Normalization and Input Normalization in CNNHuge performance discrepancies at each run, with the same CNN architectureBatch normalization vs batch sizeWhy do most GAN (Generative Adversarial Network) implementations have symmetric discriminator and generator architectures?
How can I define good in a religion that claims no moral authority?
What does Linus Torvalds mean when he says that Git "never ever" tracks a file?
Why isn't the circumferential light around the M87 black hole's event horizon symmetric?
Cooking pasta in a water boiler
How would you translate 「腰掛で仕事をする」
Does a dangling wire really electrocute me if I'm standing in water?
Proving Trigonometric “Definitions”
Match Roman Numerals
How to display lines in a file like ls displays files in a directory?
How to support a colleague who finds meetings extremely tiring?
Can there be female White Walkers?
Pokemon Turn Based battle (Python)
What to do when moving next to a bird sanctuary with a loosely-domesticated cat?
Omit the same coordinate parameters in drawing line in tikz
Why was M87 targeted for the Event Horizon Telescope instead of Sagittarius A*?
Landlord wants to switch my lease to a "Land contract" to "get back at the city"
Ubuntu Server install with full GUI
Is an up-to-date browser secure on an out-of-date OS?
Why don't hard Brexiteers insist on a hard border to prevent illegal immigration after Brexit?
Keeping a retro style to sci-fi spaceships?
What could be the right powersource for 15 seconds lifespan disposable giant chainsaw?
Multiply Two Integer Polynomials
The difference between dialogue marks
How do I free up internal storage if I don't have any apps downloaded?
Transpose-CNN with batch normalization
The 2019 Stack Overflow Developer Survey Results Are Inunderstanding batch normalizationhow to calculate the output shape of conv2d_transpose?Several fundamental questions about CNNUnderstand the shape of this Convolutional Neural Networkis Batch Norm a little bit stochastic by default?GANs and grayscale imagery colorizationBatch Normalization and Input Normalization in CNNHuge performance discrepancies at each run, with the same CNN architectureBatch normalization vs batch sizeWhy do most GAN (Generative Adversarial Network) implementations have symmetric discriminator and generator architectures?
$begingroup$
I am new to GAN (generative adversarial networks) and am trying to implement a Transpose-CNN (T-CNN) in Matlab (which is used as the generator in the GAN).
I was able to build a Transpose-CNN with out batch normalization and use it to generate a particular digit, e.g. '4' from the MNIST database.
I then tried to add the batch normalization layer to the Transpose-CNN afterwards. Then, the model(T-CNN) is going to crash, e.g. loss is becoming larger and larger. When I decrease the learning rate for batch-normalization layer a lot but keep the learning rate of transpose-convolutional layer same as before, the model (T-CNN)is still working.
I doubled checked my batch-normalization code which seems right (not 100% sure yet).
I am wondering if the learning rate of batch-normalization layer should not be the same as other layers, e.g. fully-connected, transpose-convolutional layers?
Is there a standard testbench for testing T-CNN or batch-normalization layer?
Sorry for describing the questions in such a weird way.
machine-learning cnn gan batch-normalization
edited Mar 29 at 21:42
Ethan
701625
asked Mar 29 at 18:35
EdisonEdison
115
$endgroup$
add a comment |
$begingroup$
I am new to GAN (generative adversarial networks) and am trying to implement a Transpose-CNN (T-CNN) in Matlab (which is used as the generator in the GAN).
I was able to build a Transpose-CNN with out batch normalization and use it to generate a particular digit, e.g. '4' from the MNIST database.
I then tried to add the batch normalization layer to the Transpose-CNN afterwards. Then, the model(T-CNN) is going to crash, e.g. loss is becoming larger and larger. When I decrease the learning rate for batch-normalization layer a lot but keep the learning rate of transpose-convolutional layer same as before, the model (T-CNN)is still working.
I doubled checked my batch-normalization code which seems right (not 100% sure yet).
I am wondering if the learning rate of batch-normalization layer should not be the same as other layers, e.g. fully-connected, transpose-convolutional layers?
Is there a standard testbench for testing T-CNN or batch-normalization layer?
Sorry for describing the questions in such a weird way.
machine-learning cnn gan batch-normalization
edited Mar 29 at 21:42
Ethan
701625
asked Mar 29 at 18:35
EdisonEdison
115
$endgroup$
add a comment |
$begingroup$
I am new to GAN (generative adversarial networks) and am trying to implement a Transpose-CNN (T-CNN) in Matlab (which is used as the generator in the GAN).
I was able to build a Transpose-CNN with out batch normalization and use it to generate a particular digit, e.g. '4' from the MNIST database.
I then tried to add the batch normalization layer to the Transpose-CNN afterwards. Then, the model(T-CNN) is going to crash, e.g. loss is becoming larger and larger. When I decrease the learning rate for batch-normalization layer a lot but keep the learning rate of transpose-convolutional layer same as before, the model (T-CNN)is still working.
I doubled checked my batch-normalization code which seems right (not 100% sure yet).
I am wondering if the learning rate of batch-normalization layer should not be the same as other layers, e.g. fully-connected, transpose-convolutional layers?
Is there a standard testbench for testing T-CNN or batch-normalization layer?
Sorry for describing the questions in such a weird way.
machine-learning cnn gan batch-normalization
edited Mar 29 at 21:42
Ethan
701625
asked Mar 29 at 18:35
EdisonEdison
115
$endgroup$
I am new to GAN (generative adversarial networks) and am trying to implement a Transpose-CNN (T-CNN) in Matlab (which is used as the generator in the GAN).
I was able to build a Transpose-CNN with out batch normalization and use it to generate a particular digit, e.g. '4' from the MNIST database.
I then tried to add the batch normalization layer to the Transpose-CNN afterwards. Then, the model(T-CNN) is going to crash, e.g. loss is becoming larger and larger. When I decrease the learning rate for batch-normalization layer a lot but keep the learning rate of transpose-convolutional layer same as before, the model (T-CNN)is still working.
I doubled checked my batch-normalization code which seems right (not 100% sure yet).
I am wondering if the learning rate of batch-normalization layer should not be the same as other layers, e.g. fully-connected, transpose-convolutional layers?
Is there a standard testbench for testing T-CNN or batch-normalization layer?
Sorry for describing the questions in such a weird way.
machine-learning cnn gan batch-normalization
machine-learning cnn gan batch-normalization
edited Mar 29 at 21:42
Ethan
701625
asked Mar 29 at 18:35
EdisonEdison
115
edited Mar 29 at 21:42
Ethan
701625
asked Mar 29 at 18:35
EdisonEdison
115
edited Mar 29 at 21:42
Ethan
701625
edited Mar 29 at 21:42
Ethan
701625
edited Mar 29 at 21:42
Ethan
701625
701625
asked Mar 29 at 18:35
EdisonEdison
115
asked Mar 29 at 18:35
EdisonEdison
115
asked Mar 29 at 18:35
EdisonEdison
115
115
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48231%2ftranspose-cnn-with-batch-normalization%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48231%2ftranspose-cnn-with-batch-normalization%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
