- Notifications
You must be signed in to change notification settings - Fork13
Track and report network stat#124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
| type XorbUploadValueType = (MerkleHash, Vec<u8>, Vec<(MerkleHash, usize)>); | ||
| struct NetworkStatCheckPoint { | ||
| n_bytes: u64, | ||
| start: Instant, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I found usingstd::time::SystemTime more useful in sending these over compared to Instant in#121
| use utils::progress::ProgressUpdater; | ||
| use utils::ThreadPool; | ||
| use crate::data_processing::CASDataAggregator; | ||
| use crate::errors::DataProcessingError::*; | ||
| use crate::errors::*; | ||
| const DEFAULT_NETWORK_STAT_REPORT_INTERVAL_SEC: u32 = 2; // 2 s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
You probably have this set low for testing, would prefer larger in prod - maybe 100s
| @@ -139,24 +217,28 @@ impl XorbUpload for ParallelXorbUploader { | |||
| let mut upload_tasks = self.upload_tasks.lock().await; | |||
| while let Some(result) = upload_tasks.join_next().await { | |||
| result??; | |||
| let metrics = result??; | |||
| let mut egress_rate = self.egress_stat.lock().await; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Try the lock and optimistically report. I think it is completely OK to skip theupdate_and_report call if there is lock contention
Uh oh!
There was an error while loading.Please reload this page.
In progress adding number of retries to the metrics.
Example tracing: