Repository files navigation

AutoPy is a simple, cross-platform GUI automation library for Python. Itincludes functions for controlling the keyboard and mouse, finding colors andbitmaps on-screen, and displaying alerts.

Currently supported on macOS, Windows, and X11 with the XTest extension.

• Python 3.8 and onwards (for newer releases).
• Rust 1.23.0-nightly 2019-02-06 or later (unless using a binary wheeldistribution).
• macOS 10.6 and up.
• Windows 7 and up.
• X11 with the XTest extension.

First, see if a binary wheel is available for your machine by running:

$ pip install -U autopy

If that fails, installrustup and then run:

$ rustup default nightly-2019-10-05$ pip install -U setuptools-rust$ pip install -U autopy

Another option is to build from the latest source on the GitHub repository:

$ git clone git://github.com/autopilot-rs/autopy-rs.git$ cd autopy$ python -m venv .env && source .env/bin/activate$ maturin develop

Additional possibly outdated instructions for installing from source on Windowsare availablehere.

The following is the source for a "hello world" script in autopy. Running thiscode will cause an alert dialog to appear on every major platform:

importautopydefhello_world():autopy.alert.alert("Hello, world")hello_world()

AutoPy includes a number of functions for controlling the mouse. For a fulllist, consult theAPIReference. E.g.,to immediately "teleport" the mouse to the top left corner of the screen:

>>> import autopy>>> autopy.mouse.move(0, 0)

To move the mouse a bit more realistically, we could use:

>>> import autopy>>> autopy.mouse.smooth_move(0, 0)

Even better, we could write our own function to move the mouse across the screenas a sine wave:

importautopyimportmathimporttimeimportrandomimportsysTWO_PI=math.pi*2.0defsine_mouse_wave():"""    Moves the mouse in a sine wave from the left edge of    the screen to the right.    """width,height=autopy.screen.size()height/=2height-=10# Stay in the screen bounds.forxinrange(int(width)):y=int(height*math.sin((TWO_PI*x)/width)+height)autopy.mouse.move(x,y)time.sleep(random.uniform(0.001,0.003))sine_mouse_wave()

The following will enter the keys from the string "Hello, world!" in thecurrently focused input at 100 WPM:

importautopyautopy.key.type_string("Hello, world!",wpm=100)

Alternatively, individual keys can be entered using the following:

importautopyautopy.key.tap(autopy.key.Code.TAB, [autopy.key.Modifier.META])autopy.key.tap("w", [autopy.key.Modifier.META])

All of autopy's bitmap routines can be found in the moduleautopy.bitmap. Auseful way to explore autopy is to use Python's built-inhelp() function, forexample inhelp(autopy.bitmap.Bitmap). AutoPy's functions are documented withdescriptive docstrings, so this should show a nice overview.

>>> import autopy>>> autopy.bitmap.capture_screen()<Bitmap object at 0x12278>

This takes a screenshot of the main screen, copies it to a bitmap, displays itsmemory address, and then immediately destroys it. Let's do something moreuseful, like look at its pixel data:

>>> import autopy>>> autopy.bitmap.capture_screen().get_color(0, 0)15921906

AutoPy uses a coordinate system with its origin starting at the top-left, sothis should return the color of pixel at the top-left corner of the screen. Thenumber shown looks a bit unrecognizable, but we can format it with Python'sbuilt-inhex function:

>>> import autopy>>> hex(autopy.bitmap.capture_screen().get_color(0, 0))'0xF2F2F2'

Alternatively, we can use:

>>> import autopy>>> autopy.color.hex_to_rgb(autopy.screen.get_color(0, 0))(242, 242, 242)

which converts that hex value to a tuple of(r, g, b) values. (Note thatautopy.screen.get_color(), used here, is merely a more convenient andefficient version ofautopy.bitmap.capture_screen().get_color().)

To save the screen capture to a file, we can use:

>>> import autopy>>> autopy.bitmap.capture_screen().save('screengrab.png')

The filetype is either parsed automatically from the filename, or given as anoptional parameter. Currently only jpeg and png files are supported.

>>> import autopy>>> autopy.bitmap.Bitmap.open('needle.png')<Bitmap object at 0x1001d5378>

Aside from analyzing a bitmap's pixel data, the main use for loading a bitmap isfinding it on the screen or inside another bitmap. For example, the followingprints the coordinates of the first image found in a bitmap (scanned from leftto right, top to bottom):

importautopydeffind_image_example():needle=autopy.bitmap.Bitmap.open('needle.png')haystack=autopy.bitmap.Bitmap.open('haystack.png')pos=haystack.find_bitmap(needle)ifpos:print("Found needle at: %s"%str(pos))find_image_example()

It's also possible to do a bounded search by passing a tuple((x, y), (width, height)):

haystack.find_bitmap(needle,rect=((10,10), (100,100)))

• AutoPyDriverServer, AutoPythrough WebDriver or a webdriver-compatible server.
• guibot, A tool for GUI automation usinga variety of computer vision and desktop control backends.
• spynner, Programmatic web browsingmodule with AJAX support for Python.
• SUMO, An open source, highly portable,microscopic and continuous road traffic simulation package designed to handlelarge road networks.

Hope you enjoy using autopy! For a more in depth overview, see theAPIReference.

If you are interested in this project, please consider contributing. Here are afew ways you can help:

• Report issues.
• Fix bugs andsubmit pull requests.
• Write, clarify, or fix documentation.
• Suggest or add new features.

This project is licensed under either theApache-2.0 orMIT license, at your option.

Unless you explicitly state otherwise, any contribution intentionally submittedfor inclusion in the work by you, as defined in the Apache-2.0 license, shall bedual licensed as above, without any additional terms or conditions.