Repository files navigation

Obtaining of statistics about areplica-setoplog of aMongoDB instance.

If you have ever wondered why is the duration of your oplog so short, look nofurther. The tool queries the oplog of a MongoDB instance and prints anoverview about which collections and operation types have the greatest impacton its duration. For each collection and operation type, it provides the numberof operations, their total size, and their share among all the operations inthe oplog.

In a greater detail, each MongoDB operation (e.g. insert or updated) isrepresented by aBSONdocument. Whenever you run a MongoDB operation, it is recorded in the oplog sothat all thesecondarynodes in yourreplicas set can reflect the operation in their storage. The bigger the size ofthe oplog, the longer your replicas can be down without a need for acompleteresync.Sometimes, you may have a large oplog, but its duration is not as long as youwould expect. In such cases, you can utilize the fact that the oplog is aMongoDB collection. Themongodb-oplog-stats tool queries the documents in theoplog and gives you its overview that you can use to see which collections andoperation types are shortening your oplog duration the most.

For very basic info about the oplog, you can also use thers.printReplicationInfo()function from themongo shell:

> rs.printReplicationInfo()configured oplog size:   1024MBlog length start to end: 65422secs (18.22hrs)oplog first event time:  Mon Jun 23 2019 17:47:18 GMT+0200 (CEST)oplog last event time:   Tue Jun 24 2019 11:57:40 GMT+0200 (CEST)now:                     Thu Jun 26 2019 14:24:39 GMT+0200 (CEST)

A recent-enough version ofRust. The tool iswritten in the2018 edition ofRust.

git clone https://github.com/avast/mongodb-oplog-stats.gitcd mongodb-oplog-statscargo run --release --host myhost.com --username myuser

For more information about the available parameters, run

cargo run --release -- --help

Connection and authentication parameters are identical to those used bystandard MongoDB tools.

Notes:

• The user that you use must havesufficientprivileges tobe able to query the oplog.
• If you are missing a parameter, pleasecreate anissue or submit apullrequest.

+--------------------------------------+-----------+------------+-----------+| Entry                                | Documents | Total size | Share (%) |+--------------------------------------+-----------+------------+-----------+| moviedb.movies:i                     |      2634 |  675.12 MB |     71.04 || store.books:i                        |       196 |  146.23 MB |     17.68 || moviedb.actors:u                     |        35 |   10.22 MB |      3.64 || blog.comments:i                      |     10598 |    9.07 MB |      2.34 || moviedb.movies:i                     |        18 |    7.87 MB |      2.03 || store.books:u                        |        81 |    7.59 MB |      1.96 || store.reviews:i                      |     14459 |    5.07 MB |      1.31 || moviedb.trailers:d                   |       205 |    3.17 MB |      0.82 || store.employees:d                    |         1 |      170 B |    < 0.01 || blog.articles:i                      |         1 |      158 B |    < 0.01 |+--------------------------------------+-----------+------------+-----------+

Each entry has the following format:

database.collection:op

where

• database is the name of the database,
• collection is the name of the collection,
• op is one of the following abbreviations:
  • i forinserts,
  • u forupdates,
  • d fordeletes,
  • c forcommands that affect databases at a high level,
  • db which announces the presence of adatabase,
  • n forno-ops, used for operations which do not result in a change inthe stored data. If you have a lot of these operations in your oplog,take a lookhere andhere.

As for the other columns,Documents is the number of documents correspondingto the operations,Total size is the total size of the BSON representation ofthose documents, andShare gives you the percentage of the entry with regardsto other entries in the oplog.

Here are few useful commands to use during development:

• Check for errors without code generation (faster thancargo run):

  $ cargo check

• Format sources files:

  $ rustup component add rustfmt$ cargo fmt

• Run lints to catch common mistakes and improve the code:

  $ rustup component add clippy$ cargo clippy

• Add a new dependency:

  $ cargo install cargo-edit$ cargo add PACKAGE_NAME

• List outdated dependencies (-R is for the direct ones):

  $ cargo install cargo-outdated$ cargo outdated -R

Copyright (c) 2020 Avast Software, licensed under the MIT license. See theLICENSEfile for more details.