Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

refactor: add project management#651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
cwegener wants to merge2 commits intoFlareSolverr:master
base:master
Choose a base branch
Loading
fromcwegener:master

Conversation

cwegener
Copy link

High-level overview of changes:

  1. rename src to flaresolverr in order to namespace the package
    appropriately

  2. Move package.json from root folder into flaresolverr so that it can
    be read by the utils.get_flaresolverr_version() method

  3. Rewrite necessary import statements to use the full package name as
    per step 1

  4. Move script entry point into separate method (cli_run)

  5. Manually added poetry dependencies as per existing requirements.txt
    file

HLFH reacted with thumbs up emoji
High-level overview of changes:1. rename src to flaresolverr in order to namespace the package   appropriately2. Move package.json from root folder into flaresolverr so that it can   be read by the utils.get_flaresolverr_version() method3. Rewrite necessary import statements to use the full package name as   per step 14. Move script entry point into separate method (cli_run)5. Manually added poetry dependencies as per existing requirements.txt   file
@cwegenercwegener mentioned this pull requestJan 5, 2023
@HLFH
Copy link

HLFH commentedJan 5, 2023

Cool thanks.

For 5, why not usingHatch instead of Poetry?

@cwegener
Copy link
Author

cwegener commentedJan 5, 2023
edited
Loading

Cool thanks.

For 5, why not usingHatch instead of Poetry?

No particular reason. It's just what I'm used to using. Happy to have it be hatch instead of poetry.

I did give it a quick try to create pyproject.toml forhatch, but I'm not very comfortable in usinghatch.@HLFH If you have a bit of experience withhatch, maybe you can paste what you think a solidpyproject.toml should look like forhatch ...

Here is what I have at the moment ...

[build-system]requires = ["hatchling"]build-backend = "hatchling.build"[project]name = "flaresolverr"description = "Proxy server to bypass Cloudflare protection"version = "3.0.0"readme = "README.md"requires-python = ">=3.7"license = "MIT"keywords = []authors = [  { name = "Diego Heras (ngosang)", email = "ngosang@hotmail.es" },]classifiers = [  "Development Status :: 4 - Beta",  "Programming Language :: Python",  "Programming Language :: Python :: 3.7",  "Programming Language :: Python :: 3.8",  "Programming Language :: Python :: 3.9",  "Programming Language :: Python :: 3.10",  "Programming Language :: Python :: 3.11",  "Programming Language :: Python :: Implementation :: CPython",  "Programming Language :: Python :: Implementation :: PyPy",]dependencies = [  "bottle==0.12.23",  "waitress==2.1.2",  "selenium==4.4.3",  "func-timeout==4.3.5",  "requests==2.28.1",  "websockets==10.3",  "xvfbwrapper==0.2.9",][project.urls]Documentation = "https://github.com/flaresolverr/flaresolverr#readme"Issues = "https://github.com/flaresolverr/flaresolverr/issues"Source = "https://github.com/flaresolverr/flaresolverr"[project.scripts]flaresolverr = "flaresolverr.flaresolverr:cli_run"[tool.hatch.build]include = [  "flaresolverr",][tool.hatch.envs.default]dependencies = [  "pytest",  "pytest-cov",][tool.hatch.envs.default.scripts]cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=flaresolverr --cov=tests {args}"no-cov = "cov --no-cov {args}"[[tool.hatch.envs.test.matrix]]python = ["37", "38", "39", "310", "311"][tool.coverage.run]branch = trueparallel = true[tool.coverage.report]exclude_lines = [  "no cov",  "if __name__ == .__main__.:",  "if TYPE_CHECKING:",]

@cwegener
Copy link
Author

@HLFH I pushed the change from poetry to hatch.

Also, I just noticed that there is aDockerfile in the project ... which would also need to be updated with the changes from the renaming ofsrc toflaresolverr

HLFH reacted with heart emoji

@cwegenercwegener changed the titlerefactor: use python poetry for project managementrefactor: add project managementJan 5, 2023
@HLFH
Copy link

HLFH commentedJan 5, 2023
edited
Loading

I pushed the change from poetry to hatch.

Thanks a lot! I have a few comments on your greatpyproject.toml but I'll complete the review it later as it is 3:00am here.

Also, I just noticed that there is a Dockerfile in the project ... which would also need to be updated with the changes

Indeed, thanks!

@HLFH
Copy link

HLFH commentedJan 5, 2023
edited
Loading

I updated thepyproject.toml after your PR here:https://github.com/HLFH/FlareSolverr/.
I have set Python >= 3.10 because of theupdated TLS settings from the 3.10 version, and because the Dockerfile is using version 3.11.
The dynamic versioning needs to be adjusted to work.
Thepackage.json file should be removed, and some code around versioning should be adjusted/removed as well.
I am looking into it.
I still need to update the *.txt files which I will do withpip-compile.

@ngosang
Copy link
Member

@cwegener@HLFH@M4RC0Sx I don't know much about Python packaging so I have some questions.

@ngosangngosang added this to the3.1.0 milestoneJan 5, 2023
@HLFH
Copy link

HLFH commentedJan 5, 2023
edited
Loading

https://github.com/HLFH/FlareSolverr/commits/master

I have turned off dynamic versioning until it is fixed.
requirements*.txt files are now autogenerated withpip-compile

@ngosang I'll reply to you ASAP.

@cwegener
Copy link
Author

@cwegener@HLFH@M4RC0Sx I don't know much about Python packaging so I have some questions.

* why hatch and no other package manager? there is a pr with pienv => [V3beta - Python improvements #567](https://github.com/FlareSolverr/FlareSolverr/pull/567)

Here are my $.02

hatch is provided by the Python Packaging Authority (PyPA), which is a core working group that provides amongst other things:

  • thepip software
  • thepipx software
  • the Python Packaging Guide

In terms of project management tools, there is no more authoritative and mature tool thanhatch.
https://packaging.python.org/en/latest/key_projects/#pypa-projects

* what do you think about the custom logger? => [V3beta - Python improvements #567](https://github.com/FlareSolverr/FlareSolverr/pull/567)

Good idea. Not in scope for this change set though.

* what are the instructions to install the dependencies? we have to update the readme and the dockerfile

Dependencies are declared in the core metadata as specified in the PyPA Packaging Guidehttps://packaging.python.org/en/latest/specifications/core-metadata/#requires-dist-multiple-use

Any Python Source Distribution that follows the PyPA Packaging Guide should have theseRequires-Dist fields declared in its metadata. Creating the correct metadata is handled by the project management tool, based on the configured input of installation time dependencies.

I have simply copied the contents ofrequirements.txt by hand into the respectivepyproject.toml

As an enhancement to that,@HLFH has added the use of thepip-compile tool in his fork to enhance the management of those dependency declarations.pip-compile takes what is defined inpyproject.toml and builds an explicit tree of all dependencies including all transitive dependencies and stores them in one or morerequirements.txt files.

The advantage of therequirements.txt file that is generated bypip-compile is that it has all transitive dependencies and their pinned versions, so therefore therequirements.txt files can then be used to performreproducible builds, which may or may not be of importance depending on people's requirements.

Therequirements.txt files are not used when performing apip install of a source distribution though.

So, long story short:

For end-users, the FlareSolverr python releases should be installed from an index (PyPI is the authoritative index), usingpip install flaresolverr

For Docker, I have no idea.

@ngosang Can you clarify what the original intention of theDockerfile is? Is it so that potential contributors who want to use docker for local development can quickly launch a container? Or is the intention to use theDockerfile as the basis for publishing aflaresolverr image to a Container registry like the Docker Hub or Quay?

ofek and HLFH reacted with thumbs up emoji

@ngosang
Copy link
Member

Thank you for the explanation. I will add all the changes from both PRs in the next release. Now I'm dealing with bug reports.

Can you clarify what the original intention of the Dockerfile is?

This porject is distributed as Docker image with Chrome and everithing configured inside. I have 12MM downloads as today =>https://hub.docker.com/r/flaresolverr/flaresolverr/
There are few users installing from source code. Mostly Windows and MacOS users until I provide compilled binaries =>#660

cwegener and Gauvino reacted with thumbs up emoji

@ngosangngosang modified the milestones:3.1.0,3.2.0Mar 20, 2023
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.4.0
Development

Successfully merging this pull request may close these issues.

3 participants
@cwegener@HLFH@ngosang

[8]ページ先頭

©2009-2025 Movatter.jp