Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitbc37799

Browse files
committed
Merge branch 'feature/docker'
2 parents6a51783 +d74d5fb commitbc37799

File tree

6 files changed

+31
-31
lines changed

6 files changed

+31
-31
lines changed

‎.dockerignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
dev/
2+
ebooks/

‎Dockerfile‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
11
FROM python:2.7
22

3-
MAINTAINER Martin Kuchynar <matokuchy@gmail.com>
3+
WORKDIR /packtpub-crawler
44

5-
COPY requirements.txt /requirements.txt
6-
RUN pip install -r /requirements.txt
5+
COPY script script
6+
COPY config config
7+
COPY requirements.txt requirements.txt
78

8-
ADD entrypoint.sh /
9-
RUN chmod +x /entrypoint.sh
9+
RUN pip install -r requirements.txt
1010

11-
ENTRYPOINT ["/entrypoint.sh"]
12-
13-
CMD ["/bin/bash" ]
11+
CMD ["python","/packtpub-crawler/script/scheduler.py"]

‎readme.md‎renamed to ‎README.md‎

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ This crawler automates the following step:
1212
* upload files to Google Drive
1313
* store data on Firebase
1414
* notify via email
15-
* schedule daily job on Heroku or with[Docker (TODO)](https://github.com/niqdev/packtpub-crawler/issues/18)
15+
* schedule daily job on Heroku or with Docker
1616

1717
###Default command
1818
```bash
@@ -176,21 +176,32 @@ Update `script/scheduler.py` with your own preferences.
176176

177177
More info about Heroku[Scheduler](https://devcenter.heroku.com/articles/scheduler),[Clock Processes](https://devcenter.heroku.com/articles/clock-processes-python),[Add-on](https://elements.heroku.com/addons/scheduler) and[APScheduler](http://apscheduler.readthedocs.io/en/latest/userguide.html)
178178

179-
###Docker setup[TODO](https://github.com/niqdev/packtpub-crawler/issues/18)
179+
###Docker setup
180180

181-
* Install docker and docker-compose
182-
* Clone the repository`git clone https://github.com/niqdev/packtpub-crawler.git`
183-
* Create a[config](https://github.com/niqdev/packtpub-crawler/blob/master/config/prod_example.cfg) file`cp config/prod_example.cfg config/prod.cfg`
184-
* Change your Packtpub credentials in the config file
181+
Build your image
185182
```
186-
[credential]
187-
credential.email=PACKTPUB_EMAIL
188-
credential.password=PACKTPUB_PASSWORD
183+
docker build -t niqdev/packtpub-crawler:1.3.0 .
189184
```
190185

191-
Now you should be able to claim and download your first eBook
186+
Run manually
187+
```
188+
docker run \
189+
--rm \
190+
--name my-packtpub-crawler \
191+
niqdev/packtpub-crawler:1.3.0 \
192+
python script/spider.py --config config/prod.cfg --upload drive
193+
```
194+
195+
Run scheduled crawler in background
192196
```
193-
docker-compose run packtpub-crawler --config config/prod.cfg
197+
docker run \
198+
--detach \
199+
--name my-packtpub-crawler \
200+
niqdev/packtpub-crawler:1.3.0
201+
202+
# useful commands
203+
docker exec -i -t my-packtpub-crawler bash
204+
docker logs -f my-packtpub-crawler
194205
```
195206

196207
###Development (only for spidering)

‎docker-compose.yml‎

Lines changed: 0 additions & 7 deletions
This file was deleted.

‎entrypoint.sh‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎script/spider.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ def main():
1616
parser=argparse.ArgumentParser(
1717
description='Download FREE eBook every day from www.packtpub.com',
1818
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
19-
version='1.2.0')
19+
version='1.3.0')
2020

2121
parser.add_argument('-c','--config',required=True,help='configuration file')
2222
parser.add_argument('-d','--dev',action='store_true',help='only for development')

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp