Repository files navigation

PySonar2 is a type inferencer and indexer library for Python, designed for analysing large code bases.The resulting index can be used to build code browsers and code search engines.

PySonar2 has a sophisticated type system and whole-project interprocedural analysis to infer typesand locate definitions. It handles first-class functions (closures) and control flow correctly.As a result, the accuracy of the index often outperforms Python IDEs such as PyCharm.

PySonar2 uses advanced analysis techniques similar to control-flow analysis (k-CFA, CFA2, etc), butwith a much simpler theory and without their drawbacks. It is both highly accurate and highly performant,thus it became the choice of several large scale code index infrastructures,

Major users include:

• Google
• Sourcegraph
• Insight.io (now part of Elastic)

mvn package -DskipTests

To have a feel of what PySonar2 produce, you can build a simple code browser of the Python 2.7standard library with the following command line:

java -jar target/pysonar-<version>.jar /usr/lib/python2.7 ./html

This may take a few minutes depending on your machine. You should find some interactive HTML filesinside thehtml directory after this process. You can move your mouse on the variables and clickon them to jump to definitions etc.

Note that this is just a simple demo program based on the library. PySonar2 is not meant to be anend-user tool. It is mainly designed as a library for Python IDEs, developer tools and code searchengines, so its interface may not be as appealing as an end-user tool.

If you have problems with it, please feel free to contact me.

• Python 2.7.x
• Python 3.x
• Java 8+
• maven

PySonar2 uses CPython's built-inast package to parse Python code, so please make sure you havepython orpython3 installed and pointed to by thePATH environment variable. If you have themin different names, please make symbol links.

PYTHONPATH environment variable is used for locating the Python standard libraries. It isimportant to point it to the correct Python library, for example

export PYTHONPATH=/usr/lib/python2.7

If this is not set up correctly, references to library code will not be found.

You are welcome to make code contributions.

Because of the highly complex and unpublished theory behind PySonar2, things may go wrong easilywith even an innocent-looking change. If you hope to contribute to PySonar2, please discuss with mefirst before making significant changes, otherwise I may not be able to review your changes.

For basic verification, you can run the unit tests. PySonar2 has a basic test framework. You can runthe tests using this command:

mvn test

If you modify the code or tests, you need to generate new expected results. Run these command lines:

mvn package -DskipTestsjava -classpath target/pysonar-<version>.jar org.yinwang.pysonar.TestInference -generate tests

To write new tests, you just need to write relevant Python code demonstrating your change, put theminto a directory namedtests/testname.test(test directory name must end with ".test"). Please lookat thetests directory for examples.

Please don't expect the tests to catch all bugs. Be very careful :)

Apache 2.0 License. See LICENSE file.