i just realized that there’s a little note in the pause screen saying “visual novel mode”, does this mean that there is another mode or was this an accident? (free-roam mode?)

That's the choice you make on a new game. The great majority of the content is available on "Visual Novel Mode", but there's some content only available in the other modes, such as the harem route or other content that is tied to gameplay elements.