The learning rate impacts the generalization when training neural network. Did you know it can occur in convex settings too? Check our last preprint!