- Notifications
You must be signed in to change notification settings - Fork20
Aggregate job opportunities for homeless service providers so that people can search for a job in the non-profit sector that fits their skillset.
License
hackforla/jobs-for-hope
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Aggregate job opportunities with homeless service providers so that people can search for a job that fits their skillset.
- Git for working with Github source code
- Node and npm for running the web app
- Python 2 and pip for running the web scraper
click here to show
show
- Install chocolatey (https://chocolatey.org/install)
- Install chocolatey gui (optional)
choco install chocolateygui -y
- Install chocolatey gui (optional)
- Install git, chromium, chromedriver, vscode, python2, dbeaver, nodejs
choco install git chromium chromedriver vscode python2 dbeaver nodejs -y
- Install postgresql and set postgres user password
choco install postgresql --params '/Password:password' --params-global -y
show
- Enable Windows Subsystem for Linux (reference:https://docs.microsoft.com/en-us/windows/wsl/install-win10https://lifehacker.com/how-to-get-started-with-the-windows-sybsystem-for-linux-1828952698)
- In the search bar, type "turn windows features on or off" and choose the correct item
- Scroll down and check the box for Windows Subsystem for Linux
- Windows will restart to complete the installation
- Install Ubuntu Linux
- Open the Microsoft Store and search for "Run Linux on Windows"
- Install and launch Ubuntu
- Set up a new linux user account when running for the first time
- Update and upgrade all packages
- In a terminal, run (you will need to type in your user password when running sudo)
sudo apt update && sudo apt upgrade -y
- In a terminal, run (you will need to type in your user password when running sudo)
- Install ChomeDriver
- Install chocolatey (https://chocolatey.org/install)
- Install ChomeDriver
choco install chromedriver -y
- Add a file
chromedriver
to the project directory with this content#!/bin/shchromedriver.exe "$@"
- Continue to Linux instructions
show
- Install Homebrew on Linux (reference:https://docs.brew.sh/Homebrew-on-Linux)
- Open a Linux terminal
- Install dependencies
- Debian-based (Ubuntu)
sudo apt install build-essential curl file git
- Fedora-based
sudo yum groupinstall 'Development Tools' && sudo yum install curl file git
- Debian-based (Ubuntu)
- Install homebrew
sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"
- Install packages
brew install python@2 postgresql sudo apt install chromium-chromedriver -y
show
- Install homebrew (https://brew.sh/)
- Install packages
brew install git python@2 postgresql brew cask install chromedriver
show
Install Visual Studios Code (https://code.visualstudio.com/)
- Install Prettier - Code formatter extension
Install DBeaver (https://dbeaver.io/), Community Edition
Install nvm, node, and npm (reference:https://gist.github.com/d2s/372b5943bce17b964a79)
- Install nvm
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash
- Install the latest LTS Node.js (https://nodejs.org/en/)
nvm install v10.15.3
- Install nvm
Download and install Chrome (https://www.google.com/chrome/)
Setup Postgresql (reference:https://github.com/michaeltreat/Windows-Subsystem-For-Linux-Setup-Guide/blob/master/readmes/installs/PostgreSQL.md)
Start the postgres service
- Start a terminal app
- linux (and WSL)
- Start the service
sudo service postgresql start
- Setup the postgres user (linux)
- Start a terminal app
- Set the password
sudo passwd postgres
- Type in the password and confirmation
- Close the terminal
- Connect to postgres
- Start a terminal app
- Switch to the postgres user and start the psql prompt
sudo -u postgres psql
- If the above doesn't work, do this instead
su - postgrespsql
- Troubleshooting postgres on WSL (reference:microsoft/WSL#3863)
- Append this at the end of
/etc/postgresql/10/main/postgresql.conf
data_sync_retry = true
- Append this at the end of
- Start the service
- macOS
- Start the service
brew services start postgresql
- Connect to postgres
- Start a terminal app
- enter the psql prompt
psql postgres
- Start the service
- linux (and WSL)
- Start a terminal app
Create the database (reference:https://www.techrepublic.com/blog/diy-it-guy/diy-a-postgresql-database-server-setup-anyone-can-handle/)
- Start the psql prompt
- Issue the command
create database jobsforhope;
- Create the user
- Start the psql prompt
- Issue the command
create user jobsforhope;
- Check that the user was created
\du
- Grant user privilege 1. Start the psql prompt 1. Issue the command
grant all privileges on database jobsforhope to jobsforhope;
Clone this repo to your local drive.
details
- Start a terminal app, such as Ubuntu for Windows Subsystem for Linux
- Create a src directory in the user's home directory and go in it
cd && mkdir src && cd src
- Clone the repository
git clone https://github.com/hackforla/jobs-for-hope
Change to the jobs-for-hope directory:
cd jobs-for-hope
Install the node server npm depedencies:
npm install
Obtain the
.env
file from the slack channel and place it in this directory. It contains private info (i.e., the production database connection string) that we cannot put in this public GitHub repo.Change to the client directory:
cd client
Install the client (React) dependencies:
npm install
- Run
npm start
from the jobs-for-hope directory to start the node server. - Run
npm start
from the jobs-for-hope/client directory to start the react app and open the browser.
Virtualenv (optional, but will be useful if you eventually want to have multiple python projects)
show
Set up virtualenv
Install virtualenv using pip
pip install virtualenv virtualenvwrapper
cd into project directory
cd jobs-for-hope/backend
create the virtualenv
virtualenv venv
Alternative for systems where python 2.7 is not the default
Specify the python location when creating the virtualenv
virtualenv -p /usr/local/bin/path/to/python2.7 venv # use this if the system default is python3
alternative setup using virtualenvwrapper
Install virtualenv using pip
pip install virtualenv virtualenvwrapper
Create directory to hold virtual environments
mkdir $HOME/.virtualenvs
Find out where virtualenvwrapper.sh is located for next step
which virtualenvwrapper.sh
Make
.bash_profile
call.bashrc
(reference:http://www.joshstaiger.org/archives/2005/07/bash_profile_vs.html)- Add this to
.bash_profile
if [ -f ~/.bashrc ]; then source ~/.bashrcfi
- Add this to
Add to
.bashrc
export WORKON_HOME=$HOME/.virtualenvsexport PATH="/path/to/virtualenvwrapper:$PATH"source virtualenvwrapper.sh
Start a new terminal session or call
.bashrc
source ~/.bashrc # activate virtualenvwrapper.sh, just for the first time
Create the virtualenv
mkvirtualenv jobs-for-hope
Alternative for systems where python 2.7 is not the default
- Specify the python location when creating the virtualenv
mkvirtualenv -p /usr/local/bin/path/to/python2.7 jobs-for-hope # use this if the system default is python3
- Specify the python location when creating the virtualenv
Activate the virtualenv
source venv/bin/activate
for virtualenvwrapper
workon // list the existing virtual environments, blank if none is createdworkon jobs-for-hope // activate virtual environment, not needed when first creating the virtualenv
Do work and run python within the virtualenv
Deactivate the virtualenv
deactivate // switch back to system python
- All python commands should be run inside the jobs-for-hope virtualenv if you choose to set it up. You will have to make sure the virtualenv is activated before following the steps.
Install project dependencies
cd backendpip install -r ./requirements.txt
Setup for the scrapers
- Download the
database.ini
file from the slack channel into thebackend
directory - Dump the aws database to a file (reference
database.ini
for values)pg_dump -d <database> -h <host> -U <user> -C -f jobsforhope.sql
- restore to the local database
psql -h localhost -U postgres -d jobsforhope -f jobsforhope.sql
- Download the
- Switch to the backend directory
cd backend
- To run all the scrapers
python scraper_runner.py
- To run a single scraper
python scraper_runner.py scrapers/prototypes.py
- Use DBeaver
- Add a postgres database connection with the .env file credentials from the slack channel.
- The jobs are in the jobs table.
About
Aggregate job opportunities for homeless service providers so that people can search for a job in the non-profit sector that fits their skillset.