The documentation page:https://www.arduino.cc/en/Tutorial/Debounce has an error in thecode.

The variablebuttonState at line 40 is uninitialized which will cause undefined behaviour the first time line 75if (reading != buttonState) is executed.

Usually (based on testing)buttonState is not equal toreading which results in detection of a button press on startup that did not happen. Sincereading isHIGH when the button is not pressed, the correct initial value forbuttonState is alsoHIGH. When changing this, the example functions correctly and the LED is on after the program started.