- Notifications
You must be signed in to change notification settings - Fork26.3k
Open
Labels
module: cppRelated to C++ APImodule: performanceIssues related to performance, either of kernel code or framework glueneeds reproductionEnsure you have actionable steps to reproduce the issue. Someone else needs to confirm the repro.oncall: jitAdd this issue/PR to JIT oncall triage queuetriage review
Description
Hi,
Scripted CNNs are predicting much slower, especially on C++.
On the python side, this test shows the decrease in performance:
model = Model(128, 128, 19**2, 0.1)input = torch.tensor(torch.ones(1,1,128,128))torch.cuda.synchronize()torch.set_num_threads(1)start_time1 = time.time()output1 = model(input)print('Time for total prediction 1 = {0}'.format(time.time()-start_time1))# Trace the model and convert the functionality to scripttraced_model = torch.jit.trace(model, input)start_time2 = time.time()output2 = traced_model(input)print('Time for total prediction 2 = {0}'.format(time.time()-start_time2))The output is:
Time for total prediction 1 = 0.008975505828857422Time for total prediction 2 = 0.015474081039428711So after tracing the module, prediction is much slower. When I load the model in C++, theforward method actually decreases the speed of the real-time prediction from 120Hz to 30Hz. Is there a way this performance could be improved?
cc@ezyang@gchanan@zou3519@suo@yf225@VitalyFedyunin@ngimel@mruberry
Metadata
Metadata
Assignees
Labels
module: cppRelated to C++ APImodule: performanceIssues related to performance, either of kernel code or framework glueneeds reproductionEnsure you have actionable steps to reproduce the issue. Someone else needs to confirm the repro.oncall: jitAdd this issue/PR to JIT oncall triage queuetriage review