- Notifications
You must be signed in to change notification settings - Fork2
a "zero dependency" implementation of a lexical analyzer that tokenize a program using only official "re" package. you can use this repo for your "compiler design" course.
License
danialkeimasi/python-compiler-regex-based-scanner
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
this repo contains implementation of a "zero dependency" implementation of a lexical analyzer that tokenize a program using only official "re" package.
Python 3.8
you can just put your rules in to the compiler_cli.py file and use it for yourself.
this is how you can compile your code using compiler_cli.py
PS: you need to installtyper to run the compiler_cli.
python compiler_cli.py path/to/file
you can also skip using compiler_cli.py if you want and write a script like this:
fromscannerimportScanner,UnknownTokenErrorrules= [ ('IF_KW',r'if'), ('ELSE_KW',r'else'), ('FOR_KW',r'for'), ('CONST_STR',r'".*?"|\'.*?\''), ('CONST_NUMBER',r'\d+'), ('PLUS_OP',r'\+'), ('MINUS_OP',r'\-'), ('MULTIPLY_OP',r'\*'), ('DIVIDE_OP',r'\/'), ('LP',r'\('), ('LCB',r'\{'), ('RP',r'\)'), ('RCB',r'\}'), ('EQUAL_OP',r'=='), ('ASSIGNMENT_OP',r'='), ('SEMICOLON',r';'), ('IDENTIFIER',r'[a-zA-Z_]\w*'),]scanner=Scanner(rules,"i = 5 + 3;")try:fortokeninscanner.token_generator():print(token)exceptUnknownTokenErroraserror:print(error)
- Danial Keimasi
About
a "zero dependency" implementation of a lexical analyzer that tokenize a program using only official "re" package. you can use this repo for your "compiler design" course.
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.