Repository files navigation

• How to read Python syntax errors
• How to fix syntax errors
  • Misplaced, missing, or mismatched punctuation
  • Misspelled, misplaced, or missing Python keywords
  • Illegal characters in variable names
  • Incorrect indentation
  • Incorrect use of the assignment operator (=)

This article examines how to read and fix Python syntax errors with thehelp of practical web scraping examples.

For a detailed explanation, see ourblog post.

When you get an error message, Python tries to point to the root causeof the error. Sometimes, the message tells exactly what’s the problem,but other times it’s unclear and even confusing. This happens becausePython locates the first place where it couldn’t understand the syntax;therefore, it might show an error in a code line that goes after theactual error.

Knowing how to read Python error messages goes a long way to save bothtime and effort. So let’s examine a Python web scraping code sample thatraises two syntax errors:

prices= {"price1":9.99,"price2":13.48"price3":10.99,"price4":15.01}price_found=Falseforkeyvalueinprices.items():if10<=value<=14.99:print(key+":",value)price_found=Trueifnotprice_found:print("There are no prices between $10 and $14.99")

In this example, we have a dictionary of differentprices. We use afor loop to find and print the prices between $10 and $14.99. Theprice_found variable uses a boolean value to determine whether sucha price was found in the dictionary.

When executed, Python points to the first invalid syntax error it cameupon, even though there are two more errors along the way. The firsterror message looks like this:

Information in the yellow box helps us determine the location of theerror, and the green box includes more details about the error itself.The full message can be separated into four main elements:

1. The path directory andname of the file where the error occurred;

2. The line number and thefaulty code line where the error was first encountered;

3. The carets (^) that pinpoint the place of the error;

4. The error message determines the error type, followed by additional information that may help fix the problem.

The code sample produced a syntax error found in the first line of code– theprices dictionary. The carets indicate that the error occurredbetween“price2”: 13.48 and“price3”: 10.99, and the invalidsyntax message says that perhaps we forgot to add a comma between theitems in our dictionary. That’s exactly it! The Python interpretersuggested the correct solution, so let’s update the code:

prices= {"price1":9.99,"price2":13.48,"price3":10.99,"price4":15.01}

Now, rerun the code to see what’s the second syntax error:

This time, the carets fail to pinpoint the exact location of the error,and theSyntaxError message doesn’t include additional informationabout the possible solution. In such cases, the rule of thumb would beto examine the code that comes just before the carets. In the codesample, the syntax error is raised because there’s a missing commabetween the variableskey andvalue in thefor loop. Thesyntactically correct code ine should look like this:

forkey,valueinprices.items():

1. Ensure that parentheses(), brackets[], and braces{} are properly closed. When left unclosed, the Python interpreter treats everything following the first parenthesis, bracket, or brace as a single statement. Take a look at this web scraping code sample that sends parsing instructions to ourweb scraping tool:

payload= {"source":"universal","url":"https://sandbox.oxylabs.io/products/1","parse":True,"parsing_instructions": {"title": {"_fns": [                {"_fn":"css_one","_args": ["h2"]                }            ]        }    }# Error messageFile"<stdin>",line1payload= {^SyntaxError:'{'wasneverclosed

At first glance, it may look like thepayload was closed with braces, but the Python interpreter raises a syntax error that says otherwise. You can fix the error by closing thepayload:

payload= {"source":"universal","url":"https://sandbox.oxylabs.io/products/1","parse":True,"parsing_instructions": {"title": {"_fns": [                {"_fn":"css_one","_args": ["h2"]                }            ]        }    }}# Add the missing brace

2. Make sure you close a string with proper quotes. For example, if you started your string with a single quote ‘, then use a single quote again at the end of your string. The below code snippet illustrates this:

list_of_URLs= ('https://example.com/1',    'https://example.com/2",    'https://example.com/3)print(list_of_URLs)# Error messageFile"<stdin>",line3    'https://example.com/2",^SyntaxError:unterminatedstringliteral (detectedatline3)

This example has two errors, but as you can see, the interpreter showsonly the first syntax error. It pinpoints the issue precisely, which isthe use of a single quote at the start, and a double quote at the end toclose the string.

The second error is in the third example URL, which isn’t closed with aquotation mark at all. The syntactically correct version would look likethis:

list_of_URLs= ('https://example.com/1','https://example.com/2','https://example.com/3')print(list_of_URLs)

When the string content itself contains quotation marks, use single‘, double“, and/or triple‘’’ quotes to specify where thestring starts and ends. For instance:

print("In this example, there's a "quotewithin'a quote'", which we separate with double and single quotes.")# Error messageFile"<stdin>",line1print("In this example, there's a "quotewithin'a quote'", which we separate with double and single quotes.")^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^SyntaxError:invalidsyntax.Perhapsyouforgotacomma?

The interpreter shows where the error occurred, and you can see that thecarets end within the second double quotation mark. To fix the syntaxerror, you can wrap the whole string in triple quotes (either’’’ or“””):

print("""In this example, there's a "quote within 'a quote'", which we specify with double and single quotes.""")

3. When passing multiple arguments or values, make sure to separate them with commas. Consider the following web scraping example that encapsulates HTTP headers in theheaders dictionary:

headers= {'Accept':'text/html','Accept-Encoding':'gzip, deflate, br','Accept-Language':'en-US, en;q=0.9''Connection':'keep-alive'}# Error messageFile"<stdin>",line5'Connection':'keep-alive'^SyntaxError:invalidsyntax

Again, the interpreter fails to show precisely where the issue is, butas a rule of thumb, you can expect the actual invalid syntax error to bebefore where the caret points. You can fix the error by adding themissing comma after the‘Accept-Language’ argument:

headers= {'Accept':'text/html','Accept-Encoding':'gzip, deflate, br','Accept-Language':'en-US, en;q=0.9','Connection':'keep-alive'}

4. Don’t forget to add a colon: at the end of a function or a compound statement, likeif,for,while,def, etc. Let’s see an example of web scraping:

defextract_product_data()forurlinproduct_urlsresponse=requests.get(url,headers=headers)soup=BeautifulSoup(response.content,'html.parser')title=soup.find("h1").textprice=soup.find("span", {"itemprop":"price"}).textproduct_data.append({"title":title,"price":price,        })# Error messageFile"<stdin>",line1defextract_product_data()^SyntaxError:expected':'

This time, the interpreter shows the exact place where the erroroccurred and hints as to what could be done to fix the issue. In theabove example, thedef function and thefor loop are missing acolon, so we can update our code:

defextract_product_data():forurlinproduct_urls:

1. Make sure you’re not using the reserved Python keywords to name variables and functions. If you’re unsure whether a word is or isn’t a Python keyword, check it with thekeyword module in Python or look it up in thereserved keywords list. Many IDEs, like PyCharm and VS Code, highlight the reserved keywords, which is extremely helpful. The code snippet below uses the reserved keyword `pass` to hold the password value, which causes the syntax error message:

user='username1'pass='password1'# Error messageFile"<stdin>",line2pass='password1'^SyntaxError:invalidsyntax

2. Ensure that you haven’t misspelled a Python keyword. For instance:

importtimefromrequestsimpotrSession# Error messageFile"<stdin>",line2fromrequestsimpotrSession^^^^^^SyntaxError:invalidsyntax

This code sample tries to import theSession object from therequests library. However, the Python keywordimport is misspelledasimpotr, which raises an invalid syntax error.

3. Placing a Python keyword where it shouldn't be will also raise an error. Make sure that the Python keyword is used in the correct syntactical order and follows the rules specific to that keyword. Consider the following example:

importtimeimportSessionfromrequests# Error messageFile"<stdin>",line2importSessionfromrequests^^^^SyntaxError:invalidsyntax

Here, we see an invalid syntax error because the Python keywordfromdoesn’t follow the correct syntactical order. The fixed code should looklike this:

importtimefromrequestsimportSession

Python variables have to follow certain naming conventions:

1. You can’t use blank spaces in variable names. The best solution is to use the underscore character. For example, if you want a variable named “two words”, it should be written astwo_words,twowords,TwoWords,twoWords, orTwowords.

2. Variables are case-sensitive, meaningexample1 andExample1 are two different variables. Take this into account when creating variables and calling them later in your code.

3. Don’t start a variable with a number. Python will give you a syntax error:

response1=requests.get(url)2response=requests.post(url)# Error messageFile"<stdin>",line22response=requests.post(url)^SyntaxError:invaliddecimalliteral

As you can see, the interpreter allows using numbers in variable namesbut not when the variable names start with a number.

4. Variable names can only use letters, numbers, and underscores. Any other characters used in the name will produce a syntax error.

1. Remember that certain Python commands, like compound statements and functions, require indentation to define the scope of the command. So, ensure that such commands in your code are indented properly. For instance:

prices= (16.99,13.68,24.98,14.99)defprint_price():forpriceinprices:ifprice<15:print(price)print_price()# Error message 1File"<stdin>",line6ifprice<15:^IndentationError:expectedanindentedblockafter'for'statementonline5# Error message 2File"<stdin>",line7print(price)^IndentationError:expectedanindentedblockafter'if'statementonline6

The first error message indicates that theif statement requires anindented block. After fixing that and running the code, we encounter thesecond error message that tells us theprint statement is outsidetheif statement and requires another indent. Fix the code with thecorrect indentation:

prices= (16.99,13.68,24.98,14.99)defprint_price():forpriceinprices:ifprice<15:print(price)print_price()

2. Use consistent indentation marks: either all spaces or all tabs. Don’t mix them up, as it can reduce the readability of your code, in turn making it difficult to find the incorrect indentation just by looking at the code. Most Python IDEs highlight indentation errors before running the code, so you can reformat the file automatically to fix the indentation. Let’s take the above code sample and fix the first error message by adding a single space in front of theif statement:

prices= (16.99,13.68,24.98,14.99)defprint_price():forpriceinprices:ifprice<15:print(price)print_price()

The code works without errors and prints the correct result. However,you can see how the mix of spaces and tabs makes the code a littleharder to read. Using this method can bring about unnecessary syntaxerrors when they can be avoided by sticking to either spaces or tabsthroughout the code.

1. Ensure you aren’t assigning values to functions or literals with the assignment operator=. You can only assign values to variables. Here’s an overview of some examples:

price=10.98type(price)=float# Error messageFile"<stdin>",line2    type(price)=float^^^^^^^^^^^SyntaxError:cannotassigntofunctioncallhere.Maybeyoumeant'=='insteadof'='?"price"=10.98# Error messageFile"<stdin>",line1"price"=10.98^^^^^^^SyntaxError:cannotassigntoliteralhere.Maybeyoumeant'=='insteadof'='?

In the first code sample, we want to check whether the value10.98is a float type. The Python interpreter raises an error since theassignment operator can’t be used to assign a value to a function. Thecorrect way to accomplish this is with one the following code samples:

price=10.98print(type(price))# orprice=10.98is_float=type(price)==floatprint(is_float)

2. Assign values in a dictionary with a colon: and not an assignment operator=. Let’s take a previous code sample and modify it to incorrectly use the assignment operator instead of colons:

headers= {'Accept'='text/html','Accept-Encoding'='gzip, deflate, br','Accept-Language'='en-US, en;q=0.9','Connection'='keep-alive'}# Error messageFile"<stdin>",line2'Accept'='text/html',^^^^^^^^SyntaxError:cannotassigntoliteralhere.Maybeyoumeant'=='insteadof'='?

3. Use== when comparing objects based on their values. For instance:

price_1=200.99price_2=200.98compare= (price_1=price_2)print(compare)# Error messageFile"<stdin>",line4compare= (price_1=price_2)^^^^^^^^^^^^^^^^^SyntaxError:invalidsyntax.Maybeyoumeant'=='or':='insteadof'='?

You can fix the issue by using the double equal sign== betweenprice_1 andprice_2 instead of=, which will print thecorrect result.

Check out ourblog post to find out more about Python syntax errors. There, you’ll find an explanation of syntax errors, their common causes, and some tips for avoiding them.