- Notifications
You must be signed in to change notification settings - Fork299
Use your tablet as graphic tablet/touch screen on your computer.
License
H-M-H/Weylus
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Weylus turns your tablet or smart phone into a graphic tablet/touch screen for your computer!
Weylus in action withXournal++:
- Control your mouse with your tablet
- Mirror your screen to your tablet
- Send keyboard input using physical keyboards
- Hardware accelerated video encoding
The above features are available on all Operating Systems but Weylus works best on Linux. Additionalfeatures on Linux are:
- Support for a stylus/pen (supports pressure and tilt)
- Multi-touch: Try it with software that supports multi-touch, like Krita, and see for yourself!
- Capturing specific windows and only drawing to them
- Faster screen mirroring
- Tablet as second screen
Just grab the latest release for your OS from thereleases page and install it on your computer. No appsexcept a modern browser (Firefox 80+, iOS/iPadOS 13+) are required on your tablet.If you runLinux make sure to follow the instructions describedhere to enable uinput for featureslike pressure sensitivity and multitouch!
AUR packages for Weylus are available here:
- From source:weylus
- Prebuilt binary:weylus-bin
Start Weylus, preferably set an access code in the access code box and press the Start button. Thiswill start a webserver running on your computer. To control your computer with your tablet you needto open the urlhttp://<address of your computer>:<port set in the menu, default is 1701>, ifpossible Weylus will display to you the url you need to open and show a QR code with the encodedaddress. If you have a firewall running make sure to open a TCP port for the webserver (1701 bydefault) and the websocket connection (9001 by default).
On many Linux distributions this is done with ufw:
sudo ufw allow 1701/tcpsudo ufw allow 9001/tcpPlease only run Weylus in networks you trust as there is no encryption to enable minimal latencies.
You may want to add a bookmark to your home screen on your tablet as this enables running Weylus infull screen mode (on iOS/iPadOS this needs to be done with Safari). If you are not on iOS/iPadOSthere is a button to toggle full screen mode.
Weylus supports keyboard input for physical keyboards, so if you have a Bluetooth keyboard, justconnect it to your tablet and start typing. Due to technical limitations onscreen keyboards are notsupported.
Weylus provides some features to make automation as convenient as possible. There is a command-lineinterface;--no-gui for example starts Weylus in headless mode without a gui. For more options seeweylus --help. If you want to run a specific script e.g., once a client connects to your computeryou can do so by parsing the log Weylus generates. You may want to enable more verbose logging bysetting the environment variableWEYLUS_LOG_LEVEL toDEBUG orTRACE as well asWEYLUS_LOG_JSON totrue to enable easily parseable JSON logging.
Weylus uses theuinput interface to simulate input events on Linux.To enable stylus andmulti-touch support/dev/uinput needs to be writable by Weylus. To make/dev/uinputpermanently writable by your user, run:
sudo groupadd -r uinputsudo usermod -aG uinput$USERecho'KERNEL=="uinput", MODE="0660", GROUP="uinput", OPTIONS+="static_node=uinput"' \| sudo tee /etc/udev/rules.d/60-weylus.rules
Then, either reboot, or run
sudo udevadm control --reloadsudo udevadm trigger
then log out and log in again. To undo this, run:
sudo rm /etc/udev/rules.d/60-weylus.rules
This allows your user to synthesize input events system-wide, even when another user is logged in.Therefore, untrusted users should not be added to the uinput group.
Weylus offers experimental support for Wayland. Installingpipewire andxdg-desktop-portal aswell as one of:
xdg-desktop-portal-gtkfor GNOMExdg-desktop-portal-kdefor KDExdg-desktop-portal-wlrfor wlroots-based compositors like Swayis required.
There are still some things that do not work:
- input mapping for windows
- displaying proper window names
- capturing the cursor
On Linux Weylus supports hardware accelerated video encoding through the Video Acceleration API(VAAPI) or Nvidia's NVENC. By default hardware acceleration is disabled as quality and stability ofthe hardware encoded video stream varies widely among different hardware and sufficient quality cannot be guaranteed. If VAAPI is used it is possible to select a specific driver by setting theenvironment variableLIBVA_DRIVER_NAME. You can find possible values with the commandls /usr/lib/dri/ | sed -n 's/^\(\S*\)_drv_video.so$/\1/p'. On some distributions the drivers maynot reside in/usr/lib/dri but for example in/usr/lib/x86_64-linux-gnu/dri and may not be foundby Weylus. To force Weylus to search another directory for drivers, the environment variableLIBVA_DRIVERS_PATH can be set.Additionally you can specify the VAAPI device to use by settingWEYLUS_VAAPI_DEVICE; by defaultdevices can be found in/dev/dri. On some systems this is not optional and this variable must beset. If VAAPI doesn't work out of the box for you, have a look into/dev/dri, often settingWEYLUS_VAAPI_DEVICE=/dev/dri/renderD129 is already the solution. Note that you may need to installthe driver(s) first.
Nvidias NVENC is very fast but delivers a video stream of noticeably lower quality (at least on myGeForce GTX 1050 Mobile GPU) but more recent GPUs should provide higher quality. For this to worknvidia drivers need to be installed.
There are a few possibilities to use Weylus to turn your tablet into a second screen.
Intel's drivers support creating virtual outputs that can be configured via xrandr.
But first a word of warning: The following configuration may break starting the X server. This meansyou might end up without a graphical login or X may get stuck and just display a black screen. Somake sure you know what you are doing or are at least able to recover from a broken X server.
You will need to install thexf86-video-intel driver and create the file/etc/X11/xorg.conf.d/20-intel.conf with the following contents:
Section "Device" Identifier "intelgpu0" Driver "intel" # this adds two virtual monitors / devices Option "VirtualHeads" "2" # if your screen is flickering one of the following options might help # Option "TripleBuffer" "true" # Option "TearFree" "true" # Option "DRI" "false"EndSectionAfter a rebootxrandr will show two additional monitorsVIRTUAL1 andVIRTUAL2 and can be usedto configure them. To activateVIRTUAL1 with a screen size of 1112x834 and a refresh rate of 60fps the following commands can be used:
># this generates all input parameters xrandr needs>#from a given screen resolution and refresh rate>gtf 1112 834 60 # 1112x834 @ 60.00 Hz (GTF) hsync: 51.78 kHz; pclk: 75.81 MHz Modeline "1112x834_60.00" 75.81 1112 1168 1288 1464 834 835 838 863 -HSync +Vsync># setup the monitor>xrandr --newmode"1112x834_60.00" 75.81 1112 1168 1288 1464 834 835 838 863 -HSync +Vsync>xrandr --addmode VIRTUAL1 1112x834_60.00>xrandr --output VIRTUAL1 --mode 1112x834_60.00># check if everything is in order>xrandr
Now you should be able to configure this monitor in your system setting like a regular secondmonitor and for example set its position relative to your primary monitor.
After setting up the virtual monitor start Weylus and select it in the capture menu. You may want toenable displaying the cursor in this case. That is it!
Weylus detects if you use multiple monitors and you can select the one you want to mirror. So if youwant to use Weylus as a second screen you could just buy another monitor. Obviously this ispointless as if you already bought that monitor, there is no need to use Weylus! This is where socalledHDMI/Displayport/VGA Dummy Plugs come in handy. These are small devices that pretend tobe a monitor but only cost a fraction of the price of an actual monitor.
Once you have bought one and plugged it into your computer you can configure an additional screenjust like you would do with an actual one and then use Weylus to mirror this virtual screen.
The following is untested/incomplete, feel free to do more research and open a pull request toexpand documentation on this!
- On Wayland with sway there is
create_outputwhich can be used tocreate headlessoutputs, unfortunately it is not documented howto actually do that:swaywm/sway#5553 - On Wayland with GNOME recently there has been added an option tocreate virtual monitors withmutter
By default Weylus comes without encryption and should only be run on networks you trust. If this isnot the case it's strongly advised to set up a TLS proxy. One option is to usehitch, an example script that sets up encryption is located atweylus_tls.sh.But any TLS proxy should work just fine.
Note that the mentioned script works by creating a self-signed certificate. This means your browserwill most likely display a scary looking but completely unfounded message telling you how incrediblydangerous it is to trust the certificate you yourself just created; this can be safely ignored!
In case you are using Firefox: There is abugthat prevents users from accepting self-signed certificates for websocket connections. A workaroundis to directly open the websocket connection via the URL bar and accept the certificate there. Afteraccepting the connection will of course fail as the browser expects https and not wss as protocol.
Sadly this solution is anything but frictionless and I am unhappy with the current state of affairs.This is also another reason why encryption is not enabled by default, self-signed certificates arejust too painful to handle nowadays. I'd gladly welcome any proposals to improve the situation!
Weylus needs some permissions to work properly, make sure you enable:
- Incoming connections
- Screen capturing
- Controlling your desktop
Weylus can make use of the Videotoolbox framework on macOS for hardware acceleration. In my teststhe video quality has been considerably worse than that using software encoding and thusVideotoolbox is disabled by default.
Weylus can make use of Nvidias NVENC as well as Microsoft's MediaFoundation for hardware acceleratedvideo encoding. Due to widely varying quality it is disabled by default.
To build Weylus you need to install Rust, Typescript, make, git, a C compiler, nasm and bash.cargo build builds the project. By default Weylus is build in debug mode, if you want a release build runcargo build --release. On Linux some additional dependencies are required to build Weylus. OnDebian or Ubuntu they can be installed via:
apt-get install -y libx11-dev libxext-dev libxft-dev libxinerama-dev libxcursor-dev libxrender-dev \libxfixes-dev libxtst-dev libxrandr-dev libxcomposite-dev libxi-dev libxv-dev autoconf libtool-bin \nvidia-cuda-dev pkg-config libdrm-dev libpango1.0-dev libgstreamer1.0-dev \libgstreamer-plugins-base1.0-dev libdbus-1-dev
On Fedora, they can be installed via:
sudo dnf install libXext-devel libXft-devel libXinerama-devel libXcursor-devel libXrender-devel \libXfixes-devel libXtst-devel libXrandr-devel libXcomposite-devel libXi-devel libXv-devel autoconf libtool \pkg-config libdrm-devel pango-devel gstreamer1-devel \gstreamer1-plugins-base-devel dbus-devel nasm npm
After npm is installed, typescript must be installed by:
sudo npm install typescript -g
Note that building for the first time may take a while as by default ffmpeg needs to be built. OnWindows only msvc is supported as C compiler; it is, however, possible to cross compile on Linux forWindows using minGW.
In case you do not want to build ffmpeg and libx264 via the supplied build script you can create thedirectorydeps/dist yourself and copy static ffmpeg libraries built with support for libx264 and astatic version of libx264 intodeps/dist/lib. Additionaldeps/dist/include needs to be filledwith ffmpeg's include header files. For hardware acceleration to work ffmpeg needs to be built withadditional flags depending on your OS: Consult the variableFFMPEG_EXTRA_ARGS indeps/build.shfor details. Furthermore, for VAAPI on Linux a static version of libva is required as well.
The build script will only try to build ffmpeg if the directorydeps/dist does not exist.
Alternatively passing--features ffmpeg-system to cargo will build Weylus using the system'sversion of ffmpeg. This is disabled by default for compatibility reasons, on newer systems thisshould not pose a problem and using the system libraries is advised.
It is also possible to build the Linux version inside a docker container. The Dockerfile used islocated atdocker/Dockerfile. This is also how the official release is built.Building works likethis:
docker run -it hhmhh/weylus_build bashroot@f02164dbfa18:/#git clone https://github.com/H-M-H/WeylusCloning into 'Weylus'...remote: Enumerating objects: 10, done.remote: Counting objects: 100% (10/10), done.remote: Compressing objects: 100% (7/7), done.remote: Total 827 (delta 1), reused 6 (delta 0), pack-reused 817Receiving objects: 100% (827/827), 5.38 MiB | 7.12 MiB/s, done.Resolving deltas: 100% (431/431), done.root@f02164dbfa18:/#cd Weylus/root@f02164dbfa18:/Weylus#cargo deb Compiling ...
Once the build is finished you can for example copy the binary from the container to your filesystem like this:
docker cp f02164dbfa18:/Weylus/target/release/weylus~/some/path/weylusThe .deb is located at/Weylus/target/debian/. Please note that the container ID will most likelynot bef02164dbfa18 if you run this yourself, replace it accordingly.
Modern browsers expose so calledPointerEvents that can convey notonly mouse but additionally stylus/pen and touch information. Weylus sets up a webserver with thecorresponding javascript code to capture these events. The events are sent back to the server usingwebsockets.Weylus then processes these events using either the generic OS independent backend, which onlysupports controlling the mouse or on Linux the uinput backend can be used. It makes use of theuinput Linux kernel module which supports creating a wide range of input devices including mouse,stylus and touch input devices.
Either the generic backend is used which is less efficient and only captures the whole screen or onLinux xlib is used to connect to the X-server and do the necessary work of getting windowinformation and capturing the window/screen. To make things fast the "MIT-SHM - The MIT SharedMemory Extension" is used to create shared memory images usingXShmCreateImage. If Wayland insteadof X11 is running, PipeWire and GStreamer is used to capture the screen. The images captured arethen encoded to a video stream using ffmpeg. Fragmented MP4 is used as container format to enablebrowsers to play the stream via the Media Source Extensions API. The video codec used is H.264 asthis is widely supported and allows very fast encoding as opposed to formats like AV1. To minimizedependencies ffmpeg is statically linked into Weylus.
Q: Why does the page not load on my tablet and instead I get a timeout?
A: There probably is some kind of firewall running, make sure the ports Weylus uses are opened.
Q: Why do I get the errorERROR Failed to create uinput device: CError: code...?
A: uinput is probably misconfigured, have you made sure to follow all instructions and logged outand in again? You may also be running a very old kernel that does not support the required features.In that case try to upgrade your system or use a newer one.
Q: Why is the "Capture" drop down empty and the screen not mirrored?
A: It is possible that only the port for the webserver but not the websocket has been opened, checkthat both ports have been opened.
Q: Why can I not select any windows in the "Capture" drop down and only see the whole screen.
A: If you are running Weylus on MacOS or Windows this feature is unfortunately not implemented. OnLinux it is possible that your window manager does not supportExtended Window Manager Hints or that youneed to activate them first, like for XMonad.
Q: Do I have to follow the instructions to setup Weylus as second screen too?
A: No, this is strictly optional.
Q: Why am I unable to connect my tablet to the URL displayed by Weylus?
A: It is possible that your computer and WiFi connected tablet are on different networks, make surethey are on the same network.
Q: Why does this not run on Firefox for Android?
A: Actually it does, just make sure Firefox version 80+ is installed.
Q: Why does this not run under Chrome on my iPad?
A: Chrome lacks some features for video streaming on iPadOS/iOS, try Firefox or Safari.
Q: Why won't my cursor move in osu! ?
A: Try disabling raw input.
Q: Can I use Weylus even if there is no WiFi?
A: Probably yes! Most tablets permit setting up a WiFi hotspot that can be used to connect yourcomputer and tablet. Alternatively there is USB tethering too, which can be used to setup a peer topeer connection between your tablet and computer over USB. Another method for Android devices is tosetup a socket connection withadb:
adb reverse tcp:1701 tcp:1701adb reverse tcp:9001 tcp:9001
Like that you can connect from your Android device to Weylus with the URL:http://127.0.0.1:1701.
Weylus only requires that your devicesare connected via the Internet Protocol and that doesn't necessarily imply WiFi.
About
Use your tablet as graphic tablet/touch screen on your computer.
