generated fromszheng3/rust-new-project-template
- Notifications
You must be signed in to change notification settings - Fork0
Rust cli that summarizes text with pre-trained models
License
NotificationsYou must be signed in to change notification settings
szheng3/rust-individual-project-1
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
This project aims to build a Rust CLI tool that summarizes text, based on the common task of reading and summarizing books among students. The project uses therust clap andlibtorch to run a pre-trainedhugging-face model for summarization.
- Develop my first Rust project
- Use Github Codespaces and Copilot
- Integrate libtorch and 'hugging-face pretrained models' into a Rust Cli project
- Install rust viarustup
- Install the libtorch (for Mac M1), Intel chips users can skip this step
brew install pytorch@1.13.1- Run, you can pass any text as the parameter at the end of the command. See below.
make run PARAMETER='The Chinese monarchy collapsed in 1912 with the Xinhai Revolution, when the Republic of China (ROC) replaced the Qing dynasty. In its early years as a republic, the country underwent a period of instability known as the \"Warlord Era\" before mostly reunifying in 1928 under a Nationalist government. A civil war between the nationalist Kuomintang (KMT) and the Chinese Communist Party (CCP) began in 1927. Japan invaded China in 1937, starting the Second Sino-Japanese War and temporarily halting the civil war. The surrender and expulsion of Japanese forces from China in 1945 left a power vacuum in the country, which led to renewed fighting between the CCP and the Kuomintang.'- Release
make releasex86- Bench
make benchx86- change the path in the Makefile to your libtorch path
export LIBTORCH=/opt/homebrew/Cellar/pytorch/1.13.1 &&export LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH- Run, you can pass any text as the parameter at the end of the command. See below.
make runarm PARAMETER='The Chinese monarchy collapsed in 1912 with the Xinhai Revolution, when the Republic of China (ROC) replaced the Qing dynasty. In its early years as a republic, the country underwent a period of instability known as the \"Warlord Era\" before mostly reunifying in 1928 under a Nationalist government. A civil war between the nationalist Kuomintang (KMT) and the Chinese Communist Party (CCP) began in 1927. Japan invaded China in 1937, starting the Second Sino-Japanese War and temporarily halting the civil war. The surrender and expulsion of Japanese forces from China in 1945 left a power vacuum in the country, which led to renewed fighting between the CCP and the Kuomintang.'- Release
make release- Bench
make benchBelow is the screenshot for the results
Github Actions configured in .github/workflows
- This repo main branch is automatically published to Dockerhub withCI/CD, you can pull the image fromhere
docker pull szheng3/sz-rust-ml-cli:latest- Run the docker image, you can pass any text as the parameter at the end of the command. See below.
docker run szheng3/sz-rust-ml-cli:latest 'The Chinese monarchy collapsed in 1912 with the Xinhai Revolution, when the Republic of China (ROC) replaced the Qing dynasty. In its early years as a republic, the country underwent a period of instability known as the \"Warlord Era\" before mostly reunifying in 1928 under a Nationalist government. A civil war between the nationalist Kuomintang (KMT) and the Chinese Communist Party (CCP) began in 1927. Japan invaded China in 1937, starting the Second Sino-Japanese War and temporarily halting the civil war. The surrender and expulsion of Japanese forces from China in 1945 left a power vacuum in the country, which led to renewed fighting between the CCP and the Kuomintang.'The binary could be downloaded from the release pages.release
- Configure Github Codespaces.
- Initialise Rust project with pretrained model fromhugging-face
- Add clap command line parsing for arguments (Text)
- Dockerized the project.
- CI/CD with Github Actions
- Tag and Releases
- Benchmark
About
Rust cli that summarizes text with pre-trained models
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
No packages published
Uh oh!
There was an error while loading.Please reload this page.