You signed in with another tab or window.Reload to refresh your session.You signed out in another tab or window.Reload to refresh your session.You switched accounts on another tab or window.Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Aug 16, 2025. It is now read-only.
Copy file name to clipboardExpand all lines: blog/2023-12-23-how-we-built-cost-effective-generative-ai-application/index.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -49,7 +49,8 @@ Additionally, we implemented triage logic to skip trivial changes from the revie
49
49
50
50
By using this dual-model approach for summarization and filtering out trivial changes, we save almost 50% on costs.
51
51
52
-
##Rate-limiting: Enforcing Fair Usage
52
+
53
+
##2. Rate-limiting: Enforcing Fair Usage
53
54
54
55
Upon launching our free service for open source projects, we noticed individual developers using it as a coding co-pilot by making hundreds of incremental commits for continuous feedback. CodeRabbit, designed for thorough code reviews unlike tools such as GitHub Copilot, incurs high costs when used in this manner. Therefore, we implemented hourly rate-limits on the number of files and commits reviewed per user, to control excessive usage without compromising user experience. These limits vary across different product tiers. For example, we set more aggressive limits for open source users compared to trial and paid users.
55
56
@@ -70,7 +71,7 @@ Given the high cost and capacity constraints of state-of-the-art models such as
70
71
![Rate limit metrics for open source users](rate-limits-metrics-screenshot.png"image_tRate limit metrics for open source usersooltip")
71
72
Rate limit metrics for open source users
72
73
73
-
##Caching: Avoid Re-generating Similar Review Comments
74
+
##3.Caching: Avoid Re-generating Similar Review Comments
74
75
75
76
We believe that building user habits around AI involves seamlessly augmenting existing workflows. Therefore, AI code reviews must be continuous: they should trigger as soon as a PR is opened and incrementally update the summary and generate review comments as more commits are added.