- Notifications
You must be signed in to change notification settings - Fork4
Efficiently keep track of changes to genomes
License
ginkgobioworks/edge
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Edge helps propagate annotations from parent genome to child genomes.Annotations added to parent genomes are automatically visible on all childgenomes, even on child genomes that were created before annotation addition.Edge does this by keeping structural changes between a genome and child genomesderived from it. A user creates a modified genome by applying a sequence-basedoperation, such as homologous recombination, to a parent genome. Users canannotate or make corrections to sequences on a genome; Edge automaticallyapplies the changes to the appropriate regions on the derived genomes.
Edge provides UIs to look at operations and changes and APIs for makingchanges. Edge can export genome sequences and annotations as GFF files. WhileEdge comes with a simple UI for browsing features and sequences, the UI isprimitive compared to other specialized applications.
- Use
docker-compose
:
The Docker environment is defined indocker-compose.yml
. Use theedge
service for yourcommands.
To start the edge server:
docker-compose up
Then check it out in your browser:http://localhost:9000/edge/#/genomes .
To import a genome, use:
docker-compose run edge python src/manage.py import_gff 'Saccharomyces cerevisiae' example/sc_s288c.gff
To run a shell inside the Edge container:
docker-compose run --rm edge bash
- Alternatively, you can use the
Makefile
:
TheMakefile
holds all the commands necessary for managing the server and database, both inusage and development. Runmake
without arguments to see a list of commmands.
Any of thesemake
targets can be run directly from a shell inside a container:
you@localhost:edge$ docker-compose run --rm edge bash# Now you're inside the Docker containerroot@docker-image:/usr/src/edge# make test
Furthermore, any target can have-ext
added to it. Commands that end in-ext
are meant to berunexternally to the image,i.e., from the host system.
For example, to start the edge server:
make start-ext
To run a shell:
make bash-ext
To import a genome as an example:
make add-s288c-ext
If the edge app is already running in a container, or you don't want to rebuild the image yet, youcan change-ext
to-ext_fast
, which will run the make target in a new container withouttrying to rebuild the image.
On your own machine, Construct your virtual environment and pip-install dependencies (userequirements.txt
).
To start a server, first updatesrc/server/settings.py
to use either sqlite or MySQL. For MySQL,create the appropriate databse. Then,
make migrate(cd example; gunzip ecoli-mg1655.gff.gz; gunzip yeast.gff.gz)python src/manage.py import_gff 'E. coli MG1655' example/ecoli-mg1655.gffpython src/manage.py import_gff 'Saccharomyces cerevisiae' example/yeast.gffmake run
Then set your browser tohttp://localhost:8000/edge/. Note the port is different than the Dockercase
If you needNCBI BLAST orPrimer3 support, you'll need to make sure the packages are installedon your system. Debian and Ubuntu distributions provide binary versions of both of these packages.
Depending on where the NCBI BLAST tools and Primer3 are installed, you will probably need to telledge where to find them, using the following environment variables:
NCBI_BIN_DIR # Path to directory holding ncbi binaries, e.g. /usr/binPRIMER3_BIN # Path to primer3 binary, e.g. /usr/bin/primer3_corePRIMER3_CONFIG_DIR # Path to primer3 config directory, e.g. etc/primer3_config/
Then, to set up the edge BLAST db, from thesrc
subdirectory,
python manage.py build_edge_blastdb
You can edit genome and fragment metadata, such as name, notes, circular attributes, from the Djangoadmin. Create a Django admin superuser, (see thesuperuser
make target), then set your browserto the/admin/
endpoint of wherever you are running your dev server.
Donot use the Dockerfile as-is for production, or themake run
task. Django'srunserver
command is not meant to run a production server. Instead, you'll need to spin up a production WSGIserver and run the Django projct with that, with your own settings. In this situation, it's betterto simply install theedge-genome
python package on your deployed system and add it to yourdeployment Django server'sinstalled_apps
setting. The package is designed so that, when built,it already contains all of the javascript assets compiled in their final state.
When developing locally, you can run tests in the controlled environment of the docker containerfrom your local machine withmake test-all-ext
. Make sure you've run the migrations at least oncebefore doing this. If your server is already running, and you want to run tests from the hostmachine in a separate container, usemake test-all-ext_fast
. Or just keep a container up and runthe tests from inside it.
Note that edge useswebassets for compilation of static assets. These assets are not automaticallycompiled (because the integration of that with Django is flaky). Instead, compile assets aftercahnging them withmake build_assets
. To constantly recompile them, seemake watch
.
Static dependencies are managed withBower. (Eventually to be replaced withnpm/webpack).Dependencies are downloaded before the python package is built so Python package consumers alreadyhave all required JavaScript.
Edge is versioned semantically. Continuous integration builds are done automatically on all branchesthrough Travis CI, and tagged commits to master are automatically released to PyPI. To release a newversion, bump the version number with the appropriate severity of the changes (major, minor, orpatch), and push the resulting tagged commits to the GitHub remote repo:
you@localhost:edge$ docker-compose run --rm edge make bump/patch-ext # Or bump/major, or bump/minoryou@localhost:edge$ git push --tags origin master
If you cannot push to master directly, do the same thing on a new branch and submit a pull request.
About
Efficiently keep track of changes to genomes