Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
This repository was archived by the owner on Aug 21, 2024. It is now read-only.

Aggregate job opportunities for homeless service providers so that people can search for a job in the non-profit sector that fits their skillset.

License

NotificationsYou must be signed in to change notification settings

hackforla/jobs-for-hope

Repository files navigation

Aggregate job opportunities with homeless service providers so that people can search for a job that fits their skillset.

Prerequisites

  1. Git for working with Github source code
  2. Node and npm for running the web app
  3. Python 2 and pip for running the web scraper

Complete beginner instructions

click here to show

Windows

show

  1. Install chocolatey (https://chocolatey.org/install)
    1. Install chocolatey gui (optional)
      choco install chocolateygui -y
  2. Install git, chromium, chromedriver, vscode, python2, dbeaver, nodejs
    choco install git chromium chromedriver vscode python2 dbeaver nodejs -y
  3. Install postgresql and set postgres user password
    choco install postgresql --params '/Password:password' --params-global -y

Windows 10

show

  1. Enable Windows Subsystem for Linux (reference:https://docs.microsoft.com/en-us/windows/wsl/install-win10https://lifehacker.com/how-to-get-started-with-the-windows-sybsystem-for-linux-1828952698)
    1. In the search bar, type "turn windows features on or off" and choose the correct item
    2. Scroll down and check the box for Windows Subsystem for Linux
    3. Windows will restart to complete the installation
  2. Install Ubuntu Linux
    1. Open the Microsoft Store and search for "Run Linux on Windows"
    2. Install and launch Ubuntu
    3. Set up a new linux user account when running for the first time
    4. Update and upgrade all packages
      1. In a terminal, run (you will need to type in your user password when running sudo)
        sudo apt update && sudo apt upgrade -y
  3. Install ChomeDriver
    1. Install chocolatey (https://chocolatey.org/install)
    2. Install ChomeDriver
      choco install chromedriver -y
    3. Add a filechromedriver to the project directory with this content
      #!/bin/shchromedriver.exe "$@"
  4. Continue to Linux instructions

Linux

show

  1. Install Homebrew on Linux (reference:https://docs.brew.sh/Homebrew-on-Linux)
    1. Open a Linux terminal
    2. Install dependencies
      • Debian-based (Ubuntu)
        sudo apt install build-essential curl file git
      • Fedora-based
        sudo yum groupinstall 'Development Tools' && sudo yum install curl file git
    3. Install homebrew
      sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"
  2. Install packagesbrew install python@2 postgresql sudo apt install chromium-chromedriver -y

macOS

show

  1. Install homebrew (https://brew.sh/)
  2. Install packagesbrew install git python@2 postgresql brew cask install chromedriver

Common Tools for all OSes

show

  1. Install Visual Studios Code (https://code.visualstudio.com/)

    1. Install Prettier - Code formatter extension
  2. Install DBeaver (https://dbeaver.io/), Community Edition

  3. Install nvm, node, and npm (reference:https://gist.github.com/d2s/372b5943bce17b964a79)

    1. Install nvm
      curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash
    2. Install the latest LTS Node.js (https://nodejs.org/en/)
      nvm install v10.15.3
  4. Download and install Chrome (https://www.google.com/chrome/)

  5. Setup Postgresql (reference:https://github.com/michaeltreat/Windows-Subsystem-For-Linux-Setup-Guide/blob/master/readmes/installs/PostgreSQL.md)

    1. Start the postgres service

      1. Start a terminal app
        1. linux (and WSL)
          1. Start the service
            sudo service postgresql start
          2. Setup the postgres user (linux)
            1. Start a terminal app
            2. Set the password
              sudo passwd postgres
            3. Type in the password and confirmation
            4. Close the terminal
          3. Connect to postgres
            1. Start a terminal app
            2. Switch to the postgres user and start the psql prompt
              sudo -u postgres psql
            3. If the above doesn't work, do this instead
              su - postgrespsql
          4. Troubleshooting postgres on WSL (reference:microsoft/WSL#3863)
            1. Append this at the end of/etc/postgresql/10/main/postgresql.conf
              data_sync_retry = true
        2. macOS
          1. Start the service
            brew services start postgresql
          2. Connect to postgres
            1. Start a terminal app
            2. enter the psql prompt
              psql postgres
    2. Create the database (reference:https://www.techrepublic.com/blog/diy-it-guy/diy-a-postgresql-database-server-setup-anyone-can-handle/)

      1. Start the psql prompt
      2. Issue the command
        create database jobsforhope;
      3. Create the user
        1. Start the psql prompt
        2. Issue the command
          create user jobsforhope;
        3. Check that the user was created
          \du
      4. Grant user privilege 1. Start the psql prompt 1. Issue the commandgrant all privileges on database jobsforhope to jobsforhope;

Full-Stack React/Node Application Installation

  1. Clone this repo to your local drive.

    details

    1. Start a terminal app, such as Ubuntu for Windows Subsystem for Linux
    2. Create a src directory in the user's home directory and go in it
      cd && mkdir src && cd src
    3. Clone the repository
      git clone https://github.com/hackforla/jobs-for-hope

  2. Change to the jobs-for-hope directory:

    cd jobs-for-hope
  3. Install the node server npm depedencies:

    npm install
  4. Obtain the.env file from the slack channel and place it in this directory. It contains private info (i.e., the production database connection string) that we cannot put in this public GitHub repo.

  5. Change to the client directory:

    cd client
  6. Install the client (React) dependencies:

    npm install

To Run the React/Node Application

  1. Runnpm start from the jobs-for-hope directory to start the node server.
  2. Runnpm start from the jobs-for-hope/client directory to start the react app and open the browser.

Backend Python Scraper Setup

  1. Virtualenv (optional, but will be useful if you eventually want to have multiple python projects)

    show

    1. Set up virtualenv

      1. Install virtualenv using pip

        pip install virtualenv virtualenvwrapper
      2. cd into project directory

        cd jobs-for-hope/backend
      3. create the virtualenv

        virtualenv venv
        Alternative for systems where python 2.7 is not the default

        1. Specify the python location when creating the virtualenv

          virtualenv -p /usr/local/bin/path/to/python2.7 venv # use this if the system default is python3

      alternative setup using virtualenvwrapper

      1. Install virtualenv using pip

        pip install virtualenv virtualenvwrapper
      2. Create directory to hold virtual environments

        mkdir $HOME/.virtualenvs
      3. Find out where virtualenvwrapper.sh is located for next step

        which virtualenvwrapper.sh
      4. Make.bash_profile call.bashrc (reference:http://www.joshstaiger.org/archives/2005/07/bash_profile_vs.html)

        1. Add this to.bash_profile
          if [ -f ~/.bashrc ]; then  source ~/.bashrcfi
      5. Add to.bashrc

        export WORKON_HOME=$HOME/.virtualenvsexport PATH="/path/to/virtualenvwrapper:$PATH"source virtualenvwrapper.sh
      6. Start a new terminal session or call.bashrc

        source ~/.bashrc        # activate virtualenvwrapper.sh, just for the first time
      7. Create the virtualenv

        mkvirtualenv jobs-for-hope
        Alternative for systems where python 2.7 is not the default

        1. Specify the python location when creating the virtualenvmkvirtualenv -p /usr/local/bin/path/to/python2.7 jobs-for-hope # use this if the system default is python3

    2. Activate the virtualenv

      source venv/bin/activate
      for virtualenvwrapper

      workon  // list the existing virtual environments, blank if none is createdworkon jobs-for-hope  // activate virtual environment, not needed when first creating the virtualenv

    3. Do work and run python within the virtualenv

    4. Deactivate the virtualenv

      deactivate // switch back to system python

    1. All python commands should be run inside the jobs-for-hope virtualenv if you choose to set it up. You will have to make sure the virtualenv is activated before following the steps.
  2. Install project dependencies

    cd backendpip install -r ./requirements.txt
  3. Setup for the scrapers

    1. Download thedatabase.ini file from the slack channel into thebackend directory
    2. Dump the aws database to a file (referencedatabase.ini for values)
      pg_dump -d <database> -h <host> -U <user> -C -f jobsforhope.sql
    3. restore to the local database
      psql -h localhost -U postgres -d jobsforhope -f jobsforhope.sql

To run the scraper

  1. Switch to the backend directory
    cd backend
  2. To run all the scrapers
    python scraper_runner.py
  3. To run a single scraper
    python scraper_runner.py scrapers/prototypes.py

Checking the database for scraped data

  1. Use DBeaver
  2. Add a postgres database connection with the .env file credentials from the slack channel.
  3. The jobs are in the jobs table.

About

Aggregate job opportunities for homeless service providers so that people can search for a job in the non-profit sector that fits their skillset.

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

    Packages

    No packages published

    [8]ページ先頭

    ©2009-2025 Movatter.jp