- Notifications
You must be signed in to change notification settings - Fork2
a "zero dependency" implementation of a lexical analyzer that tokenize a program using only official "re" package. you can use this repo for your "compiler design" course.
License
danialkeimasi/python-compiler-regex-based-scanner
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
this repo contains implementation of a "zero dependency" implementation of a lexical analyzer that tokenize a program using only official "re" package.
Python 3.8
you can just put your rules in to the compiler_cli.py file and use it for yourself.
this is how you can compile your code using compiler_cli.py
PS: you need to installtyper to run the compiler_cli.
python compiler_cli.py path/to/file
you can also skip using compiler_cli.py if you want and write a script like this:
fromscannerimportScanner,UnknownTokenErrorrules= [ ('IF_KW',r'if'), ('ELSE_KW',r'else'), ('FOR_KW',r'for'), ('CONST_STR',r'".*?"|\'.*?\''), ('CONST_NUMBER',r'\d+'), ('PLUS_OP',r'\+'), ('MINUS_OP',r'\-'), ('MULTIPLY_OP',r'\*'), ('DIVIDE_OP',r'\/'), ('LP',r'\('), ('LCB',r'\{'), ('RP',r'\)'), ('RCB',r'\}'), ('EQUAL_OP',r'=='), ('ASSIGNMENT_OP',r'='), ('SEMICOLON',r';'), ('IDENTIFIER',r'[a-zA-Z_]\w*'),]scanner=Scanner(rules,"i = 5 + 3;")try:fortokeninscanner.token_generator():print(token)exceptUnknownTokenErroraserror:print(error)
- Danial Keimasi
About
a "zero dependency" implementation of a lexical analyzer that tokenize a program using only official "re" package. you can use this repo for your "compiler design" course.
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Releases
Packages0
Uh oh!
There was an error while loading.Please reload this page.