Bacause the tokenizer only returns pointers to the beginning and the end of the token, calculating line numbers/column offsets is more complicated than needed.

Feature or enhancement

Instead of the tokenizer returning a token type and settings pointers, we wanna return the token type (remains as is) and then set astruct token that has the following information:

1. Pointers to beginning and end
2. Location information (lineno,col_offset, etc.)
3. Level (the level in theparenstack)

This way the parser will have a much easier job of setting line numbers & column offsets in the generated AST numbers and will make some of our work on the f-strings parsing easier.