- Notifications
You must be signed in to change notification settings - Fork7
Python one-liners: Awk-like one-liners for python
License
riolet/poline
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
poline lets you do awk-like one liners in python.
For example, the following will graph the number of connections for each hosts.
netstat -an| grep ESTABLISHED| pol"|url(_4).hostname""counter(_)"":x, c: Cols(17,40).f(x,'*' * c)"
Example output:
216.58.193.78 *****74.125.199.189 ****198.252.206.25 ***74.125.199.188 **192.30.253.125 **127.0.0.1 **216.58.193.67 *The equivalent awk version can be found oncommandlinefu.
poline is compatible with Python 2.7 to Python 3.6.
You can easily install poline is via pip:
pip install polineor
pip<version> install polineWhere<version> is the version of python you wish to use. For example, if you want to use poline with Python 3.6 installed, you should use
pip3.6 install polinepol [-h] [-F SEPARATOR] [-s] [-q] expression1 [expression2]positional arguments: expression python expressionoptional arguments: -h, --help show this help message and exit -F SEPARATOR, --separator SEPARATOR split each line by SEPARATOR -s, --split split each line -q, --quiet don't implicitly print resultspoline storesstdin in the variable_ (underscore) in the form of a generator of strings.
You can see what's inside_ with:
ls -la| pol"x for x in _"'total 20''drwxr-xr-x 2 default root 104 Apr 2 23:34 .''drwxr-xr-x 10 default root 230 Apr 2 23:34 ..''-rw-r--r-- 1 default root 0 Mar 31 14:33 __init__.py''-rw-r--r-- 1 default root 1740 Apr 2 23:31 _com_collections.py''-rw-r--r-- 1 default root 4239 Apr 2 23:33 core.py''-rw-r--r-- 1 default root 750 Apr 2 23:34 fields.py''-rw-r--r-- 1 default root 3387 Apr 2 23:31 utilfuncs.py'
If you callpol with-s option, you the_ is returned in the form of a generator of lists.
ls -la| pol -s"pformat(x,width=100) for x in _"['total','20']['drwxr-xr-x','2','default','root','104','Apr','2','23:34','.']['drwxr-xr-x','10','default','root','230','Apr','2','23:34','..']['-rw-r--r--','1','default','root','0','Mar','31','14:33','__init__.py']['-rw-r--r--','1','default','root','1740','Apr','2','23:31','_com_collections.py']['-rw-r--r--','1','default','root','4239','Apr','2','23:33','core.py']['-rw-r--r--','1','default','root','750','Apr','2','23:34','fields.py']['-rw-r--r--','1','default','root','3387','Apr','2','23:31','utilfuncs.py']
Each list item is of the classField, which is derived fromString.Field has a number of useful addition methods:
| Method | Explanation |
|---|---|
| i() | returns int value of the field |
| f() | returns int value of the field |
| h() | returns field as number of bytes in a human readable format.See bytesize() |
You can chain expressions as if you were chaining commands on bash using pipes.
netstat -an | grep ESTABLISHED | pol "|url(_4).hostname" "counter(_)" ":x, c: Cols(17,40,None).f(x, get([' '.join(l[1:]) for l in sh(['whois', x],s=T) if 'OrgName' in l[0]], 0), '*' * c)"|epxression
%FS%expression
Works on the last result item by item (or line by line if the last result wasstdin)The split() is called|epxression and split(FS) is called with%FS%expression.The list generated by the split is available in__.Individual fields are available as_0,_1,_2 etc.
ps aux| pol"|Cols(10,None).f(_0,_10)"USER COMMANDdefault bashdefault psdefault /tmp/poline/poline_venv/bin/python
:*args:expressionWorks on the last result item by item (or line by line if the last result wasstdin)Receives the n-tuple *args.
Example
ps aux | pol "skip(_)" "|(_0,_10)" ":user,cmd:Cols(50,None).f(next(sh('id',user)),cmd)"uid=1001(default) gid=0(root) groups=0(root) bashuid=1001(default) gid=0(root) groups=0(root) psuid=1001(default) gid=0(root) groups=0(root) /tmp/poline/poline_venv/bin/pythonWe are in the process of adding utility functions to poline. Contributions are most welcome.
Returns the valuex as a barchart. Ifp is true, x is intepreted as a percentage, with 100% beingw wide.
df -B1 | pol "|Cols(20,10,10,10,5,None,10).f(_0,_1.h(),_2.h(),_3.h(),_4, barchart(_2.i()/_1.f(),p=True) if _1.isdigit() else ' '*10,_5)"Filesystem 1B-blocks Used Available Use% Mounted/dev/mapper/docker-8 9.99 G 8.06 G 1.93 G 81% ▓▓▓▓▓▓▓▓░░/tmpfs 31.37 G 0.00 B 31.37 G 0% ░░░░░░░░░░/devtmpfs 31.37 G 0.00 B 31.37 G 0% ░░░░░░░░░░/sys/fs/cg/dev/sda3 211.08 G 26.49 G 173.84 G 14% ▓░░░░░░░░░/etc/hostsshm 64.00 M 0.00 B 64.00 M 0% ░░░░░░░░░░/dev/shmReturns the number of bytesx in a human readable string with a 'B', 'K', 'M', 'G', 'T', 'P' prefix.
If x is a string, bytesize tries to convert it to a float, or returns the string as is.
u specifies that bytes are in a different unit, for example u='K' if the size is in 1-K blocks.
s specifies that byte size should be strict.
Example:
$ pol "bytesize(972693249)"927.63 MSorts a list by descending order, and returns a 2-tuple with element and number of appearances in l:
[(<element>, <count>), ...]Example:
$ pol -q "pprint(counter('information'), width=20)"[('i', 2), ('n', 2), ('o', 2), ('f', 1), ('r', 1), ('m', 1), ('a', 1), ('t', 1)]getith element from listl if theith element exists, or return value d
>>>get([1,2,3,4],1)2>>>get([1,2,3,4],4,0)0
Executes shell command and arguments specified inargs, returns stdout in the form of a list of lists.
Recognized kwargs:F SEPARATOR split each line by SEPARATORs True|False split each line
Example:
The following displays the inode of each file usingstat
Python>=3.6
$ ls | pol "f'{l:20.20}\t%s' % [i[3] for i in sh(['stat',l]) if 'Inode:' in i[2]][0] for l in _"Python>=2.7
$ ls | pol "[columns(20,None).format(l,i[3]) for i in sh('stat',l,s=True) if 'Inode:' in i[2]][0] for l in _"LICENSE 360621Makefile 360653pol 360606pol.c 360637pol.o 360599README.md 360623N.B. The syntax ofsh has changed, but old syntax is still supported for the sake of backward compatibility.
As well, the popular shell commandscp,df,docker,du,find,git,history,ln,ls,lsof,mv,netstat,nmcli,ps,rm are all functions. For example
They behave as if the command is being passed tosh above.
pol "ls('-lah')"total 20Kdrwxr-xr-x 2 default root 104 Apr 2 23:34 .drwxr-xr-x 10 default root 230 Apr 2 23:34 ..-rw-r--r-- 1 default root 0 Mar 31 14:33 __init__.py-rw-r--r-- 1 default root 1.7K Apr 2 23:31 _com_collections.py-rw-r--r-- 1 default root 4.2K Apr 2 23:33 core.py-rw-r--r-- 1 default root 750 Apr 2 23:34 fields.py-rw-r--r-- 1 default root 3.4K Apr 2 23:31 utilfuncs.pySkip the firstn items from the Iteratoriter.
pol "ls('-lah')" "skip(_,n=3)"-rw-r--r-- 1 default root 0 Mar 31 14:33 __init__.py-rw-r--r-- 1 default root 1.7K Apr 2 23:31 _com_collections.py-rw-r--r-- 1 default root 4.2K Apr 2 23:33 core.py-rw-r--r-- 1 default root 750 Apr 2 23:34 fields.py-rw-r--r-- 1 default root 3.4K Apr 2 23:31 utilfuncs.pyCols is used to manage formatted output in the form of columns
The constructor generators columns by unpacking *args. For example:Cols(10,10,20,None) generates a format string as'{10.10}\t{10.10}\t{20.20}\t{}'
Unpacks*args to pass toString.format(). If format string has not been set, sets the format string as{}\t{}
history | pol "|_1" "counter(_)" ":x,c:Cols(5,None).f(x,c)"About
Python one-liners: Awk-like one-liners for python