Notice: This project is looking for a maintainer

/* * Wolnościowiec / WebProxy * ------------------------ * *   Web Proxy passing through all traffic on port 80 *   A part of an anarchist portal - wolnosciowiec.net * *   Wolnościowiec is a project to integrate the movement *   of people who strive to build a society based on *   solidarity, freedom, equality with a respect for *   individual and cooperation of each other. * *   We support human rights, animal rights, feminism, *   anti-capitalism (taking over the production by workers), *   anti-racism, and internationalism. We negate *   the political fight and politicians at all. * *   http://wolnosciowiec.net/en * *   License: LGPLv3 */

Installation

# you can also create the "config.custom.php" with `<?php return ['apiKey' => 'your-api-key'];` to have the key stored permanently without having to pass it through shellexport WW_TOKEN="your-api-key-here" composer installphp -S 0.0.0.0:8081 ./web/index.php

<?phpreturn [    'externalProxyProviders' => 'FreeProxyListProvider', // use http://free-proxy-list.net as a provider    'connectionTimeout'      => 10,    'apiKey'                 => 'something',        // cache stored in the filesystem    'cache'                  => new \Doctrine\Common\Cache\FilesystemCache(__DIR__ . '/var/cache'),    'cacheTtl'               => 360, // cache live time, refresh every eg. 360 seconds (the list of external proxy addresses is cached)        // turn off the cache    // 'cache'               => new \Doctrine\Common\Cache\VoidCache(),        // fixtures, example: Detect Facebook captcha and return a 500 response, convert all 404 to 500 error codes    'fixtures'               => 'FacebookCaptchaTo500,NotFoundTo500',        //    // Feature: Content processor    // When the HTML page is downloaded, then we can replace JS and CSS urls, so the will also be proxied    //    'contentProcessingEnabled' => true,        //     // Feature: External IP providers    // Use external proxies randomly to provide a huge amount of IP addresses, best option to scrap a big amount of data    // from pages such as Facebook, Google which are blocking very quickly by showing a captcha    //    'externalProxyProviders'   => 'HideMyNameProvider,FreeProxyListProvider,GatherProxyProvider,ProxyListOrgProvider',        // Wait 15 seconds for the connection    'connectionTimeout'        => 15,        //    // Feature: One-time access tokens    //   Imagine you can display an IFRAME on your page that will allow users to browse the URLs you allow    //   So, on server side you can prepare a token, encrypt it with AES + base64 and give to the user    //   then a user can view the specific URL through the proxy using this token    //    //   Token format: {"url": "http://some-allowed-url", "expires": "2017-05-05 10:20:30", "process": true, "stripHeaders": "X-Frame-Options"}    //   GET parameter to pass token: __wp_one_time_token    //   @see Implementation at https://github.com/Wolnosciowiec/news-feed-provider/blob/master/src/WebProxyBundle/Service/OneTimeViewUrlGenerator.php    //    'encryptionKey' => 'some-key',    'oneTimeTokenStaticFilesLifeTime' => '+2 minutes',        //    // Feature: Chromium/PhantomJS prerenderer    //   Use an external service - Wolnościowiec Prerenderer to send requests using a real browser like Chromium or PhantomJS    //    'prerendererUrl'           => 'http://my-prerenderer-host',    'prerendererEnabled'       => true];

External providers list

# fetch the list of proxy IP addresses from providers selected in configuration# and verify all proxy addresses one-by-one to make sure that everything is fresh*/8 * * * * php ./bin/rebuild-proxy-list

How to use

Example request

GET / HTTP/1.1ww-target-url: http://facebook.com/ZSP-Związek-Wielobranżowy-Warszawa-290681631074873ww-token: your-api-key-hereww-no-external-proxy: false

Example request through Chromium/PhantomJS + external proxy

GET /__webproxy/render HTTP/1.1Host: webproxy.localhostww-token: your-api-key-hereww-url: https://facebook.comww-process-output: false

Example request with Chromium/PhantomJS without external proxy

GET /__webproxy/render HTTP/1.1Host: webproxy.localhostww-token: your-api-key-hereww-url: https://facebook.comww-process-output: falseww-no-external-proxy: true

Example request to get only external proxy details

GET /__webproxy/get-ip HTTP/1.1Host: webproxy.localhostww-token: your-api-key-here

Deployment

sudo docker build . -t webproxy sudo docker run -p 7001:80 webproxy:latestcurl http://localhost:7001

sudo docker run -p 7001:80 wolnosciowiec/web-proxy:latest

CURL example

$headers = [/* ... */];$headers[] = 'ww-token: my-proxy-token'$headers[] = 'ww-target-url: http://google.com';curl_setopt($curlHandle, CURLOPT_URL, 'https://proxy-address');curl_setopt($curlHandle, CURLOPT_HTTPHEADER, $headers);curl_setopt($curlHandle, CURLOPT_CONNECTTIMEOUT, 15);curl_setopt($curlHandle, CURLOPT_TIMEOUT, 15);curl_setopt($curlHandle, CURLOPT_PROXY, '');

Fixtures

export WW_FIXTURES="FacebookCaptchaTo500,SomethingElse"

return [    'fixtures' => 'FacebookCaptchaTo500',];

Special endpoints

ProxySelector-------------  Returns the IP address with port of a proxy which normally would be used to redirect the traffic  Token is required to use the endpoint.    Useful when need to render a page using eg. Chromium, so the browser could be spawn with proper arguments.  See: https://github.com/Wolnosciowiec/frontend-prerendererGET /__webproxy/get-ip

Renderer--------  Renders the page with Chromium/PhantomJS using an external service Wolnościowiec Prerenderer.  See: https://github.com/Wolnosciowiec/frontend-prerenderer  GET /__webproxy/render HTTP/1.1Host: webproxy.localhostww-token: your-api-key-hereww-url: https://facebook.comww-process-output: false