What are the criteria for updating bias values in back propagation?Trying to figure out how to set weights for convolutional networksBack-propagation through max pooling layersWhy is the learning rate for the bias usually twice as large as the the LR for the weights?How to update bias in CNN?Back Propagation Using MATLABUpdating the weights of the filters in a CNNBack Propagation in time for tf.nn.dynamic_rnn for sequential input (from batch)How does backpropagation differ from reverse-mode autodiffWhat are the possible values of a filter in a CNN?CNN Back Propagation without Sigmoid Derivative

Single Colour Mastermind Problem

If Earth is tilted, why is Polaris always above the same spot?

Will tsunami waves travel forever if there was no land?

Where did the extra Pym particles come from in Endgame?

Please, smoke with good manners

Why does Bran Stark feel that Jon Snow "needs to know" about his lineage?

In the time of the mishna, were there Jewish cities without courts?

Minimum value of 4 digit number divided by sum of its digits

Confusion about capacitors

How to set printing options as reverse order as default on 18.04

Is GOCE a satellite or aircraft?

Phrase for the opposite of "foolproof"

Pawn Sacrifice Justification

Why does nature favour the Laplacian?

Binary Numbers Magic Trick

A question regarding using the definite article

Why does processed meat contain preservatives, while canned fish needs not?

Python "triplet" dictionary?

Are some sounds more pleasing to the ear, like ㄴ and ㅁ?

Why do computer-science majors learn calculus?

How to set the font color of quantity objects (Version 11.3 vs version 12)?

How to back up a running remote server?

What's the metal clinking sound at the end of credits in Avengers: Endgame?

How to stop co-workers from teasing me because I know Russian?



What are the criteria for updating bias values in back propagation?


Trying to figure out how to set weights for convolutional networksBack-propagation through max pooling layersWhy is the learning rate for the bias usually twice as large as the the LR for the weights?How to update bias in CNN?Back Propagation Using MATLABUpdating the weights of the filters in a CNNBack Propagation in time for tf.nn.dynamic_rnn for sequential input (from batch)How does backpropagation differ from reverse-mode autodiffWhat are the possible values of a filter in a CNN?CNN Back Propagation without Sigmoid Derivative













0












$begingroup$


During back propagation, the algorithm can modify the weight values or bias values to reduce the loss.



How does the algorithm decide whether it has to modify the weight values or bias values to reduce the loss?



Does it modify the weight values in one pass and bias values in another pass?



Thanks!






deep-learning cnn backpropagation







share|improve this question









$endgroup$
















    0












    $begingroup$


    During back propagation, the algorithm can modify the weight values or bias values to reduce the loss.



    How does the algorithm decide whether it has to modify the weight values or bias values to reduce the loss?



    Does it modify the weight values in one pass and bias values in another pass?



    Thanks!






    deep-learning cnn backpropagation







    share|improve this question









    $endgroup$














      0












      0








      0





      $begingroup$


      During back propagation, the algorithm can modify the weight values or bias values to reduce the loss.



      How does the algorithm decide whether it has to modify the weight values or bias values to reduce the loss?



      Does it modify the weight values in one pass and bias values in another pass?



      Thanks!






      deep-learning cnn backpropagation







      share|improve this question









      $endgroup$




      During back propagation, the algorithm can modify the weight values or bias values to reduce the loss.



      How does the algorithm decide whether it has to modify the weight values or bias values to reduce the loss?



      Does it modify the weight values in one pass and bias values in another pass?



      Thanks!






      deep-learning cnn backpropagation




      deep-learning cnn backpropagation






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Apr 9 at 4:12









      MaanuMaanu

      1031




      1031




















          1 Answer
          1






          active

          oldest

          votes


















          1












          $begingroup$

          Actually, weight values and bias values are updated simultaneously in each pass of backpropagation. That’s because the orientation of loss gradient vector is determined by the partial derivatives of all weights and biases with respect to the loss function. So if in each pass, you want to move in the correct direction towards the minimun of loss function, you must update both weights and biases at the same time and in the correct orientation.






          share|improve this answer









          $endgroup$













            Your Answer








            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "557"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );













            draft saved

            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48932%2fwhat-are-the-criteria-for-updating-bias-values-in-back-propagation%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1












            $begingroup$

            Actually, weight values and bias values are updated simultaneously in each pass of backpropagation. That’s because the orientation of loss gradient vector is determined by the partial derivatives of all weights and biases with respect to the loss function. So if in each pass, you want to move in the correct direction towards the minimun of loss function, you must update both weights and biases at the same time and in the correct orientation.






            share|improve this answer









            $endgroup$

















              1












              $begingroup$

              Actually, weight values and bias values are updated simultaneously in each pass of backpropagation. That’s because the orientation of loss gradient vector is determined by the partial derivatives of all weights and biases with respect to the loss function. So if in each pass, you want to move in the correct direction towards the minimun of loss function, you must update both weights and biases at the same time and in the correct orientation.






              share|improve this answer









              $endgroup$















                1












                1








                1





                $begingroup$

                Actually, weight values and bias values are updated simultaneously in each pass of backpropagation. That’s because the orientation of loss gradient vector is determined by the partial derivatives of all weights and biases with respect to the loss function. So if in each pass, you want to move in the correct direction towards the minimun of loss function, you must update both weights and biases at the same time and in the correct orientation.






                share|improve this answer









                $endgroup$



                Actually, weight values and bias values are updated simultaneously in each pass of backpropagation. That’s because the orientation of loss gradient vector is determined by the partial derivatives of all weights and biases with respect to the loss function. So if in each pass, you want to move in the correct direction towards the minimun of loss function, you must update both weights and biases at the same time and in the correct orientation.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Apr 9 at 5:27









                pythinkerpythinker

                8641314




                8641314



























                    draft saved

                    draft discarded
















































                    Thanks for contributing an answer to Data Science Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …


                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.

                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48932%2fwhat-are-the-criteria-for-updating-bias-values-in-back-propagation%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    Marja Vauras Lähteet | Aiheesta muualla | NavigointivalikkoMarja Vauras Turun yliopiston tutkimusportaalissaInfobox OKSuomalaisen Tiedeakatemian varsinaiset jäsenetKasvatustieteiden tiedekunnan dekaanit ja muu johtoMarja VaurasKoulutusvienti on kestävyys- ja ketteryyslaji (2.5.2017)laajentamallaWorldCat Identities0000 0001 0855 9405n86069603utb201588738523620927

                    Image
                    Vuonna 1953 syntyneetSuomalaiset kasvatustieteilijätSuomalaiset professoritTurun yliopiston professorit 9. maaliskuuta1953TurkuTurun yliopistonSuomalaisen tiedeakatemian Marja Vauras Henkilötiedot Syntynyt 9. maaliskuuta 1953 (ikä 66) Turku Kansallisuus Suomi Ammatti kasvatustieteen professori Koulutus ja ura Instituutti Turun yliopisto Aiheesta muualla Marja Vauras Turun yliopiston tutkimusportaalissa Infobox OK Marja Merja Sinikka Vauras (s. 9. maaliskuuta 1953 Turku) on suomalainen filosofian tohtori, ja Turun yliopiston kasvatustieteen professori. [1] [2] Hän on toiminut myös Turun yliopiston Kasvatustieteellisen tiedekunnan dekaanina. [3] Suomalaisen tiedeakatemian jäsen Vauras on ollut vuodesta 2013. [2] Lähteet | ↑ Suomen professorit 1640–2007 , Professoriliitto, 2008 (ISBN  978-952-99281-1-8 ja 978-952-99281-2-5 , viitattu 7 huhtikuuta 2019 ) ↑ a b Suomalaisen Tiedeakatemian varsinaiset jäsenet pä...
                    Read more

                    Which is better: GPT or RelGAN for text generation?2019 Community Moderator ElectionWhat is the difference between TextGAN and LM for text generation?GANs (generative adversarial networks) possible for text as well?Generator loss not decreasing- text to image synthesisChoosing a right algorithm for template-based text generationHow should I format input and output for text generation with LSTMsGumbel Softmax vs Vanilla Softmax for GAN trainingWhich neural network to choose for classification from text/speech?NLP text autoencoder that generates text in poetic meterWhat is the interpretation of the expectation notation in the GAN formulation?What is the difference between TextGAN and LM for text generation?How to prepare the data for text generation task

                    Image
                    Neighboring nodes in the network Is it canonical bit space? Why do I get two different answers for this counting problem? Python: return float 1.0 as int 1 but float 1.5 as float 1.5 Drawing a Grid/Tikz image Why is it a bad idea to hire a hitman to eliminate most corrupt politicians? Why is this clock signal connected to a capacitor to gnd? Blender 2.8 I can't see vertices, edges or faces in edit mode How to say in German "enjoying home comforts" Is there an expression that means doing something right before you will need it rather than doing it in case you might need it? Can a virus destroy the BIOS of a modern computer? Saudi Arabia Transit Visa Why "Having chlorophyll without photosynthesis is actually very dangerous" and "like living with a bomb"? What is the word for reserving something for yourself before others do? Why do bosons tend to occupy the same state? Is the Joker left-handed? What method can I use to des...
                    Read more

                    Is this part of the description of the Archfey warlock's Misty Escape feature redundant?When is entropic ward considered “used”?How does the reaction timing work for Wrath of the Storm? Can it potentially prevent the damage from the triggering attack?Does the Dark Arts Archlich warlock patrons's Arcane Invisibility activate every time you cast a level 1+ spell?When attacking while invisible, when exactly does invisibility break?Can I cast Hellish Rebuke on my turn?Do I have to “pre-cast” a reaction spell in order for it to be triggered?What happens if a Player Misty Escapes into an Invisible CreatureCan a reaction interrupt multiattack?Does the Fiend-patron warlock's Hurl Through Hell feature dispel effects that require the target to be on the same plane as the caster?What are you allowed to do while using the Warlock's Eldritch Master feature?

                    Image
                    How to Reset Passwords on Multiple Websites Easily? How can a function with a hole (removable discontinuity) equal a function with no hole? What is the opposite of 'gravitas'? Return the Closest Prime Number Term for the "extreme-extension" version of a straw man fallacy? How to check is there any negative term in a large list? How does it work when somebody invests in my business? How do I rename a Linux host without needing to reboot for the rename to take effect? Avoiding estate tax by giving multiple gifts Purchasing a ticket for someone else in another country? Energy of the particles in the particle accelerator How can I get through very long and very dry, but also very useful technical documents when learning a new tool? How did Doctor Strange see the winning outcome in Avengers: Infinity War? How to create a 32-bit integer from eight (8) 4-bit integers? What can we do to stop prior company from asking us questions? How to be diplo...
                    Read more