Journaling is helping with navigating out of local optima

Credit: Raschka, S. *Single-Layer Neural Networks and Gradient Descent*, *Sebastian Raschka, PhD*. Available at: https://sebastianraschka.com/Articles/2015_singlelayer_neurons.html (Accessed: 27 February 2023).

I wrote about not trusting your gut, since it leads you into local optima, rather than into the global optimum.

While reflecting upon this, I realized that this is quite similar to gradient descent, where you risk navigating into a local optimum as well. To solve this, you have to choose a better learning rate, that isn’t taking forever to converge and is less likely to find a local optimum.

I think that journaling is a method of achieving this. You squint at the learning landscape and learn how it looks like, before going down a path. You know how far away you are from the global optimum. In a way, you increase your learning rate, leading to larger jumps and jumps into unusual directions. This seems to be the power of metacognition for me.

If we take the example from the zettel about not trusting your gut, the pianist would criticially reflect her fears and choose a larger learning rate, namely she would still give concerts, in order to get used to her fear.

Source:

Links:
Don’t (always) trust your gut