Repository files navigation

beyond-tp is a data migration service.

This project is now under high development, please do not apply to production environment

• Contributor Guide
• Development Requirement
  • protobuf
  • flutter
• Build
  • Build Binary
  • Build Frontend Pages
  • Generate Code
• Usage
  • Start a Server
  • Start a Staff
  • What's more

You are always welcome to act as a contributor to beyond-tp.

Since beyond-tp is an application underBeyondStorage,you should follow the contributor guide of the community.

Please feel free to submit issues, create a pull request, or just star this repo :)

For more details, please refer to:https://beyondstorage.io/docs/general/contributor-guide

Beyond-tp uses protobuf for RPC call and data serialization. So protobuf tools are needed when generate codefrom.proto files, related components are listed below:

protoc is theProtocol Compiler. To installprotoc, you cantakeProtocol Compiler Installationas a reference.

protoc-gen-go andprotoc-gen-go-grpc are two plugins used to generate Go code from.proto files.

Install them using Go 1.16 or higher by running:

go install google.golang.org/protobuf/cmd/protoc-gen-go@latestgo install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

This will installprotoc-gen-go andprotoc-gen-go-grpc binary files in$GOBIN.Set the$GOBIN environment variable to change the installation location.It must be in your$PATH for the protocol buffer compiler to find it.

You can also specify a version number by replacinglatest after@.

Since we develop beyond-tp with:

➜  ~ protoc --versionlibprotoc 3.15.7➜  ~ protoc-gen-go --versionprotoc-gen-go v1.25.0➜  ~ protoc-gen-go-grpc --versionprotoc-gen-go-grpc 1.1.0

So we recommend you to use these tools equal or higher than this.

Beyond-tp uses flutter to conduct frontend web pages.v2.0.1 or higher version is recommended for development.

To install flutter, you can takeflutter installation as a reference.

Notice: For some network reason, you can also get help fromhttps://flutter.cn/community/china.

make build

Runmake build to build the binary file used to start a beyond-tp server or staff.

make build-frontend

Runmake build-frontend to build frontend pages.

Notice that beyond-tp uses flutter to build web pages,so please make sure flutter is wellinstalled, and added in$PATH directories.

For more details, please runmake help as a reference.

make generate

Runmake generate to generate the latest code by GraphQL schema and grpc.proto files.

Notice that beyond-tp uses protobuf tools to generate grpc code,so please make sure protobuf is wellinstalled,and added in$PATH directories for the protocol buffer compiler to find it.

Server used as a web server as well as a task manager. It allows you to manage beyond-tp visually,and support a rpc server to communicate withStaffs. You can easily start a beyond-tp server by:

beyondtp server --db /path/to/db --host localhost --port 7436 --rpc-port 7000

This command will start a beyond-tp server, including a web server, with local db indicated bydb flag.You can visit the home page atlocalhost:7436/ui in your web browser.

Runbeyondtp server --help for more examples and flags' usage.

Staff used as a task executor. It can run a migration task, distributed byServer.You can easily start a beyond-tp staff by:

beyondtp staff --host localhost --manager localhost:7000

This command will start a beyond-tp staff connected to the managerlocalhost:7000.The manager's host and port are specified inserver command by flaghost andrpc-port.

Runbeyondtp staff --help for more examples and flags' usage.

We support to use environment variable to replace flags. All the flags follow these rules:

• All flags correspond to environment variable with prefixBEYONDTP_
• Kebab-case (log-level) to Snake_case (LOG_LEVEL) with upper case
• Flags for specific commands should add command prefix, e.g.BEYONDTP_SERVER_PORT andBEYONDTP_STAFF_PORT for--port flag underserver command andstaff command
• Global flags do not need to add command prefix, e.g.BEYONDTP_LOG_LEVEL

If the same flag set by both environment variable and command flag, command flag will behigher priority than environment variable, for example:

BEYONDTP_LOG_LEVEL=debug bin/beyondtp server --db badger --log-level=warn

The final log level would bewarn instead ofdebug.

For more command usage, please runbeyondtp --help as a reference.

• Project Roadmap
• Discussions on forum