Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

Simple server for fetching static web pages using Firefox Marionette

NotificationsYou must be signed in to change notification settings

raohwork/prpr

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Simple application fetching static HTML via Firefox Marionette.

It is designed most for internal using. YOU HAVE TO PREVENT SECURITY ISSUE YOURSELF.

Prpr accepts following environmental variables:

  • SECRET: shared secret. Default to"" which disables authenticating.
  • FIREFOX: path to Firefox binary. Default to"firefox".
  • DEBUG_FIREFOX: disable--headless mode.
  • FIREFOX_OPTS: extra command line arguments passed to Firefox.
  • BIND: binding address passed toListenAndServe. Default to":9801".
  • QUEUE_SIZE: max concurrent loading pages. Default to1.

It always passes--marionette and--safe-mode to Firefox.

Extra configuration for Docker

The docker imageronmi/prpr will create new profile directory in /tmp each run,which might waste disk space if you start/stop it frequently.

You can mount a volume somewhere and pass it in envvarFIREFOX_PROFILE

docker run -d --name prpr \  -v"$(pwd)/data:/fxprofile" \  -e"FIREFOX_PROFILE=/fxprofile" \  -p"127.0.0.1:9801:9801" \  ronmi/prpr

How it fetches HTML

  1. Allocate a free browser window (blocked if none free)
  2. Switch to allocated window
  3. Navigate to the url
  4. Wait a second
  5. Every second, check if the HTML element presents
  6. If it does exists, run a piece of JS to fetch HTML and return
  7. Returns error after 10 tries

It is much more complex in real code as prpr now supports concurrent loading.

Usage

wget \  --post-data='uri=http://google.com&wait=queryString&secret=mysecret' \  http://127.0.0.1:9801/grab
  • uri: URL you want to fetch from.
  • wait: CSS-selector to wait for.
  • secret: your shared secret.

It returns HTML code if success, 400 if shared secret mismatch and 500 when failed.

WARNING

  • To keep code simple, prpr just waits 10 seconds before actually connect to firefox. IT DOES NOT SUPPORT RECONNECTING.
  • For best security, DO NOT EXPOSE IT TO PUBLIC NETWORK.

MAKE SURE YOU KNOW WHAT YOU ARE FETCHING!

License

WTFPL

About

Simple server for fetching static web pages using Firefox Marionette

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

[8]ページ先頭

©2009-2025 Movatter.jp