Repository files navigation

Unobtrusive literate programming experience for Python pragmatists

illiterate is a Python module that helps you applysome of the literate programming paradigmwithout requiring a meta-programming language (likenoweb) or any preprocessing step to actuallyget your code up and running.

It works kind of opposite to how literate programming suggests, in the sense that you write code-firstand embed documentation into your code. I know, purist literate programmers will hate this but hey, it's a compromise.

If you've never heard about literate programming before, then I suggest you to read at least theWikipedia entryand then we can continue discussing.It is a fascinating topic, and there aremany resources out there.

Back already? Ok, so here is illiterate's take on this matter.

Ideally, for literate programming to work, you would code in a meta-language mixing prose, code, and macros.This is great if everyone that will ever write code in your project is willing to indulge in literate programming.Sadly, I have found that more often that not, this is not the case.Hence, even though it is awesome, literate programming has some major practical drawbacks that, at this moment,make it impossible for many people to apply it widely, including:

• Poor support from editors and lack of tooling, which is not just a matter of syntax highlighting. The very feature that makesliterate programming extra powerful, i.e., macros, makes it almost impossible for any semantic analysis to work, so forgetabout intellisense, smart completion, or interactive linting.
• A hard entry curve, since unfortunately people in the 21st century still learns to code the "old" way, that is,code-first. Introducing someone into literate programming is hard because it takes some time to grok it and understand the benefits.
• It's hard to incrementally switch to it. If you already have a somewhat large program written in the "traditional" way,it's very hard to port it to the literate programming paradigm incrementally.

All these reasons make it, at least for me, almost impossible to apply pure literate programming in anything morethan toy projects. However, I do love the paradigm, and I do think it makes you a better programmer, and makes your codeeasier to maintain and understand. I wanted a way to introduce as much of literate programming as possible into thetraditional programming paradigms, but still being able to use the same tools, introducing literate programmingidiosyncrasies incrementally into existingcodebases but "flying under the radar" as much as possible, so detractors don't complain.

Hence,illiterate was born. It is called that way in part because is kind of a twist on the literate programmingparadigm, and also because it is supposed to help us illiterates to write more literate code.

Glad you asked. The idea is to encourage a more literate codebase while introducing as few changes as possible.Specifically, you should not need to use new tools, editor extensions, or preprocessors. Code written using theilliterate style looks exactly like regular code, but hopefully, a bit better.

Everything stems from these key principles:

• Documentation for a codebase should be written as prose, and it should be enjoyable to read it top to bottom.It should not be simply a list of modules and methods with few-line descriptions; rather, it should be a cohesivepiece of literature that clearly explains the authors' intents for any small details and how everything fits into the biggerpicture.
• Documentation should be as close as possible to real code, ideally right next to it, instead of in externalmarkdown files that can easily get out of sync. Furthermore, there should be some automated wayto ensure that documentation is up to date, ideally with embedded unit tests that fail if documentation is wrong.
• Documentation should be written both for users of your code and future developers, and it should beeasy for anyone to dive as deep as they want. This means that users only interested in calling your high-levelAPI can easily understand how to use it, while collaborators or anyone wanting to understand how everything worksshould also be able to follow all the details.

To achieve these objectives, illiterate proposes really only one major paradigm shift:

Your code is the documentation.

That's it. You will simply write all the documentation for your code right inside your code, as comments and as Python docstrings,according that what is more convenient for each case. But keep this in mind,all your code will be published as-is for documentation purposes.

Now you are forced to think about your code in terms of: "well, this will be read by users at some point, so I better make it as publishable as possible".This means that you can no longer simply put some throw-away code in some forsaken_tmp.py file. That file will appear in the documentation, so it better be publishable.

The only thing that illiterate does is taking your Python repository and turn it into Markdown files.It will parse all your code, and output nicely formatted Markdown versions of each.py file. It is up to you that what is writen in those.py files is something worth publishing as documentation.

To use it, you simply run:

python -m illiterate build [src] [output]

Where[src] is the folder that is the root of your project's code (i.e., the top-level folder with an__init__.py inside), and[output] is where you want the markdown files. You can add--copy from:to to copy verbatim some files into the output folder.I do this for copying theReadme.md into anindex.md which becomes the homepage.

For example, in this project, standing on the root folder (where this Readme is located), you would run the following (🤓 yeah, it is kind if Inception-ish):

python -m illiterate build illiterate docs --copy Readme.md:index.md

This will take all the code inilliterate, convert it to Markdown, and drop it inside thedocs folder.It will also copy theReadme.md file intodocs/index.md.

The same procedure can be obtained using configuration files. The configuration files have the following structure:

inline:true# true if we want to process the comments within code blocksoutput:docs# The address of the output foldersources:# List of files to be processed or copiedsample_module.file:sample_module/file.py# An input is made up of the address of the output file without suffix and using periods as a separator and the address of the input file.

The configuration is usually in a file calledilliterate.yml somehow mimicking mkdocs.

Although these configuration files can be done completely by hand, we recommend using the commandpython -m illiterate preset init to create it. This command uses the same parameters as thepython -m illiterate build command but instead of parsing the code directly it creates the appropriate configuration file.As long as we have a configuration file to start with, we can modify it and include other details to suit our purpose as our code progresses. Then we can make illiterate work according to that setting using the commandpython -m illiterate preset build to build the Markdowns.

What you do with those Markdowns is up to you. In this project, I usemkdocs for documentation.If you havemkdocs, then make sure to have yourmkdocs.yml correctly configured so that it renders those freshly created markdowns.You can also see themkdocs.yml in this repository to get an idea of how that configuration looks, but beware, I'm using some customthemes and other stuff you might or might not want.

You can mix illiterate with regular markdown simply by hand-crafting all the documentation you want in pure Markdown and then conveniently designing yourmkdocs.yml.

This project is quite small, but it is a self-fulfilling prophecy. The remainingdocumentation has been written withilliterate. Just keep reading and you'll see for yourself what does this mean.