- Notifications
You must be signed in to change notification settings - Fork5
Replace custom build system with Poetry#792
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
I did this in a Docker container based upon the `python:3.10` image.
- The FastAPI container image builds and container runs- The Dagster Daemon container image builds (container does not run)- The Dagster Dagit container image builds (container does not run)
I'm pleasantly surprised that the GHA workflows all passed! I'll resume looking at this PR next week. |
Hi@dwinston, if we stop using the current I'm not too familiar with those files. Based on my vague memory of when you,@pkalita-lbl,@shreddd, and I worked on automating Runtime-related parts of the release process (last year, when making commitcbd58f3 and nearby commits), I believe these files came into play when building |
@eecavanna they are not currently used. Destroy them. |
Thanks,@dwinston. I'll put them on my ✂️ list. |
@@ -16,15 +16,18 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ | |||
apt-get -y clean && \ | |||
rm -rf /var/lib/apt/lists/* | |||
# Install Poetry. | |||
RUN pip install poetry |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
what version of poetry? is there a "go-to" docker image that distributes poetry?
I'll resolve the current issues and then shelve this until stakeholders are back in the office. |
dwinston commentedApr 30, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
eecavanna commentedJun 27, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
During today's infrastructure meeting, we talked about "leapfrogging" Poetry and, instead, going straight to uv.@dwinston and I were both in favor of that.@shreddd and I have both gotten a little bit of experience using uv as part of the BERtron work. Also,@pkalita-lbl mentioned (here) LinkML is starting to move toward uv. I will shelve this Poetry-specific PR and create a ticket about moving to uv. |
Uh oh!
There was an error while loading.Please reload this page.
In this branch, I am working on replacing the current
pip-tools
-based build system with a Poetry-based one.As of now, all the containers start up OK in my development environment. I have deleted the
requirements
directory. I have introduced Poetry and updated the Dockerfiles and all other dependent files I've come across so far, accordingly. I have not tested the behavior of the Runtime or Dagster (beyond confirming their web UIs load) and have not run the automated tests locally (they run OK on GitHub Actions, though).Details
I'll add some details here when this branch is further along.
It seems to me that multiple programs — all of whose code is managed in this repo — share a single dependency list (meaning, there's one bucket of dependencies/packages that different applications pull from as they need something, but that bucket containsall dependencies ofall programs). That may be another opportunity for simplification/cleanup here: separate the different programs (e.g. Dagster stuff, maybe eventually the standalone Minter) out into their own Poetry projects. We can discuss this.
Note: The Mongo migration notebooks donot share a dependency list/bucket with the rest of the applications in the repo. Those notebookshappen to live in the repo, but don't have any dependency upon anything in the repo outside of their own directory (they depend only on either packages definedwithin that directory, or packages hosted on PyPI).
Related issue(s)
Fixes#563
Related subsystem(s)
docs
directory)Testing
I confirmed that whatever tests GitHub Actions runs, all pass.
Documentation
docs
directory)Maintainability
study_id: str
)# TODO
or# FIXME
black
to format all the Python files I created/modified