Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit06eb64c

Browse files
committed
Merge branch 'version-2.8.4-streaming'
2 parentsa290129 +48d1f6b commit06eb64c

File tree

29 files changed

+1392
-316
lines changed

29 files changed

+1392
-316
lines changed

‎AIDocuments/CODE_QUALITY_ANALYSIS_COMPREHENSIVE.md‎

Lines changed: 509 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
#Output Streaming Implementation Guide
2+
3+
##Problem Statement
4+
Currently, RsyncUI buffers the**entire** rsync output in memory before processing it. For large sync operations with thousands of files, this can:
5+
- Consume excessive memory
6+
- Delay feedback to the user
7+
- Make the app unresponsive during large transfers
8+
9+
##Solution: Stream Processing
10+
11+
Process rsync output**line-by-line** as it arrives, instead of waiting for the entire output.
12+
13+
---
14+
15+
##Implementation Options
16+
17+
###Option 1: Use Existing`printLine` Closure ✅ RECOMMENDED
18+
19+
Your`RsyncProcess` package already supports streaming via the`printLine` parameter! This is visible in:
20+
-`CreateHandlers.swift` line 33:`printLine: RsyncOutputCapture.shared.makePrintLinesClosure()`
21+
22+
**Advantages:**
23+
- No need to modify RsyncProcess package
24+
- Already working for`RsyncRealtimeView`
25+
- Clean separation of concerns
26+
27+
**How to Implement:**
28+
29+
1.**Use`StreamingOutputHandler.swift`** (already created)
30+
- Manages a rolling buffer (keeps last N lines)
31+
- Provides callbacks for line-by-line and batch processing
32+
- Prevents memory overflow
33+
34+
2.**Use`CreateStreamingHandlers.swift`** (already created)
35+
- Wrapper around`CreateHandlers`
36+
- Injects streaming handler into the flow
37+
- Maintains compatibility with existing code
38+
39+
3.**Update your classes** (example:`EstimateWithStreaming.swift`)
40+
```swift
41+
// Before: Waits for all output
42+
let handlers=CreateHandlers().createHandlers(...)
43+
44+
// After: Streams output line-by-line
45+
let streamingHandler=StreamingOutputHandler(
46+
maxBufferSize:100,
47+
onLineReceived: { linein
48+
// Process each line immediately
49+
self.handleLine(line)
50+
}
51+
)
52+
53+
let handlers=CreateStreamingHandlers().createHandlers(
54+
fileHandler: fileHandler,
55+
processTermination: processTermination,
56+
streamingHandler: streamingHandler
57+
)
58+
```
59+
60+
---
61+
62+
###Option 2: Modify RsyncProcess Package (If Needed)
63+
64+
If`printLine` isn't working or you need more control, implement streaming at the`Process` level using`Pipe.readabilityHandler`.
65+
66+
See`ProcessStreamingExample.swift` for a complete implementation showing:
67+
- How to use`fileHandle.readabilityHandler`
68+
- Line-by-line processing with partial line handling
69+
- Memory-efficient streaming
70+
71+
**Key Code:**
72+
```swift
73+
outputPipe.fileHandleForReading.readabilityHandler= { fileHandlein
74+
let data= fileHandle.availableData// Non-blocking read
75+
// Process data immediately as it arrives
76+
onLineReceived(line)
77+
}
78+
```
79+
80+
---
81+
82+
##Migration Strategy
83+
84+
###Phase 1: Test with Quick Task ✅ Low Risk
85+
1. Modify`extensionQuickTaskView.swift`
86+
2. Use`CreateStreamingHandlers` instead of`CreateHandlers`
87+
3. Test with small sync operations
88+
89+
###Phase 2: Update Estimate
90+
1. Modify`Estimate.swift` using`EstimateWithStreaming.swift` as reference
91+
2. Keep rolling buffer of last 100 lines
92+
3. Only send summary (last 20 lines) to UI
93+
94+
###Phase 3: Update Execute
95+
1. Modify`Execute.swift`
96+
2. Stream output to progress indicator
97+
3. Use streaming for log file writing
98+
99+
###Phase 4: Update Verify Operations
100+
1.`VerifyTasks.swift`
101+
2.`ExecutePushPullView.swift`
102+
3.`PushPullView.swift`
103+
104+
---
105+
106+
##Benefits
107+
108+
###Memory Usage
109+
**Before:**
110+
- 10,000 file sync = 10,000 lines buffered =~500 KB per operation
111+
- Multiple simultaneous operations = potential memory issues
112+
113+
**After:**
114+
- Rolling buffer of 100 lines =~5 KB per operation
115+
- 99% memory reduction
116+
117+
###User Experience
118+
**Before:**
119+
- No feedback until process completes
120+
- UI appears frozen on large transfers
121+
122+
**After:**
123+
- Real-time line-by-line updates
124+
- Responsive UI during transfers
125+
- Better progress indication
126+
127+
###Performance
128+
**Before:**
129+
- Process finishes → Parse 10,000 lines → Create UI data → Display
130+
- User waits for all steps
131+
132+
**After:**
133+
- Line arrives → Display immediately
134+
- Continuous feedback
135+
136+
---
137+
138+
##Files Created
139+
140+
1.**`StreamingOutputHandler.swift`**
141+
- Core streaming logic
142+
- Rolling buffer management
143+
- Callbacks for line/batch processing
144+
145+
2.**`CreateStreamingHandlers.swift`**
146+
- Drop-in replacement for`CreateHandlers`
147+
- Integrates streaming handler
148+
- Maintains API compatibility
149+
150+
3.**`EstimateWithStreaming.swift`**
151+
- Example implementation for Estimate
152+
- Shows how to use streaming in practice
153+
154+
4.**`ProcessStreamingExample.swift`**
155+
- Low-level reference implementation
156+
- Use if modifying RsyncProcess package
157+
158+
---
159+
160+
##Testing
161+
162+
1.**Small sync** (< 100 files)
163+
- Verify no regression
164+
- Check output completeness
165+
166+
2.**Large sync** (> 1,000 files)
167+
- Monitor memory usage (should stay low)
168+
- Verify UI responsiveness
169+
- Check that summary data is correct
170+
171+
3.**Error cases**
172+
- Rsync errors are caught immediately
173+
- Partial output handled correctly
174+
175+
4.**Real-time view**
176+
-`RsyncRealtimeView` continues to work
177+
- No duplicate output
178+
179+
---
180+
181+
##Configuration
182+
183+
In`StreamingOutputHandler`:
184+
```swift
185+
maxBufferSize:Int=100// Adjust based on needs
186+
```
187+
188+
**Recommendations:**
189+
-**Estimate operations**: 100 lines (only need summary)
190+
-**Execute operations**: 200 lines (keep more context)
191+
-**Restore operations**: 500 lines (user wants full list)
192+
-**Quick tasks**: 100 lines (small operations)
193+
194+
---
195+
196+
##Rollback Plan
197+
198+
If streaming causes issues:
199+
1. Keep`CreateHandlers` unchanged
200+
2. Use`CreateStreamingHandlers` only where needed
201+
3. Easy to revert individual classes
202+
203+
---
204+
205+
##Next Steps
206+
207+
1.**Test`printLine` behavior**
208+
- Verify it's called line-by-line
209+
- Check timing (immediate vs batched)
210+
211+
2.**Start with Quick Task**
212+
- Lowest risk area
213+
- Quick to test and verify
214+
215+
3.**Gradually migrate**
216+
- One class at a time
217+
- Monitor memory and performance
218+
219+
4.**Remove buffering in PrepareOutputFromRsync**
220+
- Currently keeps entire output to extract last 20 lines
221+
- With streaming, can extract during process
222+
- Further memory savings
223+
224+
---
225+
226+
##Code Quality Notes
227+
228+
✅**Follows existing patterns**
229+
- Uses`@MainActor` appropriately
230+
- Proper logging with`Logger.process`
231+
- Error handling via`SharedReference.shared.errorobject`
232+
233+
✅**Maintains compatibility**
234+
- Doesn't break existing code
235+
- Optional adoption
236+
- Can run both approaches side-by-side
237+
238+
✅**Memory safe**
239+
- Rolling buffer prevents unbounded growth
240+
- Configurable limits
241+
- Clear ownership and lifecycle
242+
243+
---
244+
245+
##References
246+
247+
- Original recommendation:`AIDocuments/CODE_QUALITY_ANALYSIS_COMPREHENSIVE_DEC16.md` line 705
248+
- Existing streaming UI:`RsyncUI/Views/OutputViews/RsyncRealtimeView.swift`
249+
- Current handler:`RsyncUI/Model/Execution/CreateHandlers/CreateHandlers.swift`
250+
- Output capture: Check RsyncProcess package's`PrintLines.swift`

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
APP = RsyncUI
22
BUNDLE_ID = no.blogspot.$(APP)
3-
VERSION = 2.8.3
3+
VERSION = 2.8.4
44
BUILD_PATH =$(PWD)/build
55
APP_PATH = "$(BUILD_PATH)/$(APP).app"
66
ZIP_PATH = "$(BUILD_PATH)/$(APP).$(VERSION).zip"

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp