Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

feat(render): add deployment of rq worker#85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
# Deploy the rq background worker to Render.com

When deploying to Render.com, it's much easier if we don't have to pass the `REDIS_URL` and the queue name directly to the command.

So instead, let's create a `settings.py` file and put our `rq` worker configuration there:

```python title="settings.py"
import os
from dotenv import load_dotenv

load_dotenv()

REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379")
QUEUES = ["emails", "default"]
```

The names of the variables are important, see [the documentation](https://python-rq.org/docs/workers/#using-a-config-file) for all the options that are currently supported.

To run the `rq` worker using this settings file use `rq worker -c settings`.

Let's add this to our repo, and then deploy the background worker to Render.com.

First create a new background worker:

![Create a new service of type background worker in Render.com](./assets/render-create-bg-worker.png)

Then, give it a name and fill in its basic settings. The default works for the most part. Make sure it's in the same region as or close to your Postgres and Redis databases:

![Filling in the Render basic worker information with its name set to 'rest-api-background-worker', environment set to 'docker', and region set to 'Frankfurt'](./assets/render-bg-worker-basic-settings.png)

Add the environment variables it needs. Although in this case it doesn't need the `DATABASE_URL`, you can add it if you will be adding other tasks that do use the database in the near future. If not, leave it out.

:::warning Internal URL
If your Redis database is with Render.com, you'd want to use the Redis database **Internal URL**, but I encountered some issues with it where the `redis` package didn't recognise the URL. Try it, but fall back to the external URL if it doesn't work.
:::

![Environment variables added in Render.com including DATABASE_URL, REDIS_URL, MAILGUN_API_KEY, and MAILGUN_DOMAIN, with their respective values](./assets/render-bg-worker-env-vars.png)

Finally, this "background worker" is just a Python program without networking capabilities. So if we leave it as is, it will actually just run our Dockerfile and the Dockerfile's `CMD` command (which starts our web application). Therefore we want to give it a custom Docker command that starts the background worker.

In that command, I'll go into the `/app` directory of the Docker container, and run the `rq` worker passing in the `settings.py` file.

The command is `/bin/bash -c cd /app && rq worker -c settings`.

This is what it looks like in Render.com:

![Screenshot showing the Docker command in Render.com](./assets/render-bg-worker-docker-command.png)
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong.Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
DATABASE_URL=
MAILGUN_API_KEY=
MAILGUN_DOMAIN=
REDIS_URL=
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
FLASK_APP=app
FLASK_DEBUG=True
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
.env
.venv
.vscode
__pycache__
data.db
*.pyc
.DS_Store
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
3.10.6
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
# CONTRIBUTING

## How to run the Dockerfile locally

```
docker run -dp 5000:5000 -w /app -v "$(pwd):/app" IMAGE_NAME sh -c "flask run"
```
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
FROM python:3.10
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --upgrade -r requirements.txt
COPY . .
CMD ["/bin/bash", "docker-entrypoint.sh"]
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
# REST APIs Recording Project

Nothing here yet!
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
import os

from flask import Flask, jsonify
from flask_smorest import Api
from flask_jwt_extended import JWTManager
from flask_migrate import Migrate
from dotenv import load_dotenv


from db import db
from blocklist import BLOCKLIST
import models

from resources.item import blp as ItemBlueprint
from resources.store import blp as StoreBlueprint
from resources.tag import blp as TagBlueprint
from resources.user import blp as UserBlueprint


def create_app(db_url=None):
app = Flask(__name__)
load_dotenv()

app.config["PROPAGATE_EXCEPTIONS"] = True
app.config["API_TITLE"] = "Stores REST API"
app.config["API_VERSION"] = "v1"
app.config["OPENAPI_VERSION"] = "3.0.3"
app.config["OPENAPI_URL_PREFIX"] = "/"
app.config["OPENAPI_SWAGGER_UI_PATH"] = "/swagger-ui"
app.config["OPENAPI_SWAGGER_UI_URL"] = "https://cdn.jsdelivr.net/npm/swagger-ui-dist/"
app.config["SQLALCHEMY_DATABASE_URI"] = db_url or os.getenv("DATABASE_URL", "sqlite:///data.db")
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db.init_app(app)
migrate = Migrate(app, db)
api = Api(app)

app.config["JWT_SECRET_KEY"] = "jose"
jwt = JWTManager(app)

@jwt.token_in_blocklist_loader
def check_if_token_in_blocklist(jwt_header, jwt_payload):
return jwt_payload["jti"] in BLOCKLIST

@jwt.revoked_token_loader
def revoked_token_callback(jwt_header, jwt_payload):
return (
jsonify(
{"description": "The token has been revoked.", "error": "token_revoked"}
),
401,
)

@jwt.needs_fresh_token_loader
def token_not_fresh_callback(jwt_header, jwt_payload):
return (
jsonify(
{
"description": "The token is not fresh.",
"error": "fresh_token_required",
}
),
401,
)

@jwt.additional_claims_loader
def add_claims_to_jwt(identity):
# Look in the database and see whether the user is an admin
if identity == 1:
return {"is_admin": True}
return {"is_admin": False}

@jwt.expired_token_loader
def expired_token_callback(jwt_header, jwt_payload):
return (
jsonify({"message": "The token has expired.", "error": "token_expired"}),
401,
)

@jwt.invalid_token_loader
def invalid_token_callback(error):
return (
jsonify(
{"message": "Signature verification failed.", "error": "invalid_token"}
),
401,
)

@jwt.unauthorized_loader
def missing_token_callback(error):
return (
jsonify(
{
"description": "Request does not contain an access token.",
"error": "authorization_required",
}
),
401,
)

@app.before_first_request
def create_tables():
db.create_all()

api.register_blueprint(ItemBlueprint)
api.register_blueprint(StoreBlueprint)
api.register_blueprint(TagBlueprint)
api.register_blueprint(UserBlueprint)

return app
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
"""
blocklist.py

This file just contains the blocklist of the JWT tokens. It will be imported by
app and the logout resource so that tokens can be added to the blocklist when the
user logs out.
"""

BLOCKLIST = set()
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
#!/bin/sh

flask db upgrade

exec gunicorn --bind 0.0.0.0:80 "app:create_app()"
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
Single-database configuration for Flask.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
# A generic, single database configuration.

[alembic]
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s

# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false


# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic,flask_migrate

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARN
handlers = console
qualname =

[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic

[logger_flask_migrate]
level = INFO
handlers =
qualname = flask_migrate

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
from __future__ import with_statement

import logging
from logging.config import fileConfig

from flask import current_app

from alembic import context

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)
logger = logging.getLogger('alembic.env')

# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
config.set_main_option(
'sqlalchemy.url',
str(current_app.extensions['migrate'].db.get_engine().url).replace(
'%', '%%'))
target_metadata = current_app.extensions['migrate'].db.metadata

# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.


def run_migrations_offline():
"""Run migrations in 'offline' mode.

This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.

Calls to context.execute() here emit the given string to the
script output.

"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
compare_type=True,
literal_binds=True
)

with context.begin_transaction():
context.run_migrations()


def run_migrations_online():
"""Run migrations in 'online' mode.

In this scenario we need to create an Engine
and associate a connection with the context.

"""

# this callback is used to prevent an auto-migration from being generated
# when there are no changes to the schema
# reference: http://alembic.zzzcomputing.com/en/latest/cookbook.html
def process_revision_directives(context, revision, directives):
if getattr(config.cmd_opts, 'autogenerate', False):
script = directives[0]
if script.upgrade_ops.is_empty():
directives[:] = []
logger.info('No changes in schema detected.')

connectable = current_app.extensions['migrate'].db.get_engine()

with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
process_revision_directives=process_revision_directives,
compare_type=True,
**current_app.extensions['migrate'].configure_args
)

with context.begin_transaction():
context.run_migrations()


if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
"""${message}

Revision ID:${up_revision}
Revises:${down_revision| comma,n}
Create Date:${create_date}

"""
from alembic import op
import sqlalchemy as sa
${importsif importselse""}

# revision identifiers, used by Alembic.
revision =${repr(up_revision)}
down_revision =${repr(down_revision)}
branch_labels =${repr(branch_labels)}
depends_on =${repr(depends_on)}


def upgrade():
${upgradesif upgradeselse"pass"}


def downgrade():
${downgradesif downgradeselse"pass"}
Loading
[8]ページ先頭
©2009-2025 Movatter.jp