Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork56.4k
Ported GStreamerSource to OpenCV#20709
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
891445e to5a16a93Compare
sivanov-work left a comment• edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
please pay attention about GStreamerPtr assigning pointer ourselves problem at least. Moreover, from my perspective, whole class implementation is danger
| voidmapBufferToFrame(GstBuffer* buffer, GstVideoInfo* info, GstVideoFrame* frame, | ||
| GstMapFlags mapFlags) { | ||
| GAPI_Assert(info &&"GstVideoInfo is absent during GstBuffer mapping"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
You can get rid of validity checking by passing references as argument (for buffer and frame also)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed, thanks!
Uh oh!
There was an error while loading.Please reload this page.
| GST_MAP_WRITE); | ||
| m_mappedForWrite.store(true, std::memory_order_release); | ||
| }else { | ||
| gstreamer_utils::mapBufferToFrame(m_buffer, m_videoInfo, &m_videoFrame, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
thread 1, R: goes here and make mapBufferToFrame
thread 2, W goes line 67-68 and make gst_video_frame_unmap AND putfalse inm_isMapped <context switch)
thread 1, R awakes and setm_isMapped totrue at line 95.
Does this situation lead to broken invariants?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This situation should never happen according to framework design
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
| GStreamerPtr&operator=(GStreamerPtr&&) =delete; | ||
| private: | ||
| T* ptr; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
How's about to do not use home-grown implementation but default and highly-testedstd::unique_ptr with custom dtor :)
tempate<...>using GStreamerPtr = std::unique_ptr<Type, decltype(GStreamerPtrRelease<Type>)>make_ptr(Type* ptr) {return GStreamerPtr (ptr, &GStreamerPtrRelease<Type>);}}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It is very good suggestion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed! Thanks!
| m_outputType(outputType) | ||
| { | ||
| auto appsinks = m_pipeline->getElementsByFactoryName("appsink"); | ||
| GAPI_Assert(1ul == appsinks.size()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What do you think to add warning log message additionally?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think that this is very good idea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed, thanks!
| } | ||
| break; | ||
| } | ||
| default: { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
from my point of view we should handle this situation at construction time
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I don't understand, what do you mean here? Could you please add some more details?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
sure, I mean do not allow to create Source with invalidm_output here
https://github.com/opencv/opencv/pull/20709/files#diff-1387240a1379983779d72695d2b19e9c70195f80ab2784ef8dd3f4885cf78e3fR64
and to remove this switch-case like check from other part of a Source code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks! Got the idea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed, thanks!
| gst_app_sink_set_emit_signals(GST_APP_SINK(m_appsink.get()),FALSE); | ||
| GStreamerPtr<GstCaps>nv12Caps(gst_caps_from_string( | ||
| "video/x-raw,format=NV12;video/x-raw(memory:DMABuf),format=NV12"));/* transfer full*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
suggest to make global constant
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed, thanks!
Uh oh!
There was an error while loading.Please reload this page.
0557257 to98ccf8fCompare| // | ||
| // Copyright 2021 Intel Corporation. | ||
| // | ||
| // This software and the related documents are Intel copyrighted materials, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Use the right license header.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for the catch!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed, thanks!
modules/gapi/CMakeLists.txt Outdated
| src/streaming/gstreamer/gstreamersource.cpp | ||
| src/streaming/gstreamer/gstreamer_buffer_utils.cpp | ||
| src/streaming/gstreamer/gstreamer_media_adapter.cpp | ||
| src/streaming/gstreamer/gstreamerinit.cpp |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Don't forget to add: "${CMAKE_CURRENT_LIST_DIR}/include/opencv2/${name}/streaming/gstreamer/*.hpp" togapi_ext_hdrs.
Could I also ask you to add:
"${CMAKE_CURRENT_LIST_DIR}/include/opencv2/${name}/streaming/onevpl/*.hpp" to there ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks! Sure!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Done, please check!
| GStreamerSource(const std::string& pipeline, | ||
| const GStreamerSource::OutputType outputType = | ||
| GStreamerSource::OutputType::MAT); | ||
| GStreamerSource(std::shared_ptr<GStreamerPipelineFacade> pipeline, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Is thector really needed inpublic section ?
Since there is only forward declaration forGStreamerPipelineFacade it's supposed to be inprivate section
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
No, no, this one is used inGStreamerPipeline class, and these classes are not friends.
| cv::GComputationc(cv::GIn(in),cv::GOut(out)); | ||
| // Graph compilation for streaming mode: | ||
| auto ccomp = c.compileStreaming(std::move(cv::compile_args(cv::gapi::core::cpu::kernels()))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
U don't need to pass this package explicitly, do you ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes, I don't, thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed, thanks!
TolyaTalamanov commentedOct 7, 2021
I've opened this one:#20832 |
TolyaTalamanov commentedOct 7, 2021 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Whitespaces: |
98ccf8f to37339d2Compare
sivanov-work left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
| "GStreamer pipeline has not been created!"); | ||
| PipelineState state; | ||
| GstClockTime timeout = 5 * GST_SECOND; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Then what happened if query state would reach timeout?
Uh oh!
There was an error while loading.Please reload this page.
| } | ||
| } | ||
| template<typename T>staticinlinevoidGStreamerPtrRelease(T** pPtr); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
No, i mean - use determined specializations for several types of T, but when user or-someone-else just call this function with something not related to correct GStreamer resource, forFoo class for example. Then correct specialization would not be deducted and compiler generated declaration forFoo, which has no body definition and user would face with LINK error, at least
The good approach is do not wait for LINK error but break compilation with some message usingstatic_assert
Something like that:
template<T> GStreamerRelease( T***) {static_assert(std::is_same<T, ...>::value, "Unsupported type in GStreamer Release")}The part withstd::is_same can be more intellectual with variadic templates
Feel free to ignore this comment
| } | ||
| break; | ||
| } | ||
| default: { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
sure, I mean do not allow to create Source with invalidm_output here
https://github.com/opencv/opencv/pull/20709/files#diff-1387240a1379983779d72695d2b19e9c70195f80ab2784ef8dd3f4885cf78e3fR64
and to remove this switch-case like check from other part of a Source code
modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamer_pipeline.hpp OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
| * To create GStreamerSource instance you need to pass 'pipeline' and, optionally, 'outputType' | ||
| * arguments into constructor. | ||
| * 'pipeline' should represent GStreamer pipeline in form of textual description. | ||
| * Almost any custom pipeline is supported which can be successfully ran via gst-launch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
| * To create GStreamerSource instance you need to pass'pipeline'and, optionally,'outputType' | |
| * arguments into constructor. | |
| *'pipeline' should represent GStreamer pipeline in form of textual description. | |
| * Almost any custom pipeline is supported which can be successfullyran via gst-launch. | |
| * To createaGStreamerSource instance you need to pass'pipeline'and, optionally,'outputType' | |
| * arguments into constructor. | |
| *'pipeline' should representaGStreamer pipeline in form of textual description. | |
| * Almost any custom pipeline is supported which can be successfullyrun via gst-launch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It is irregular verb, this is why "ran" is used here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes, but here is 3rd form needed, which is "run"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
please consider to correct
modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamersource.hpp OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
ff807b5 to7c22705CompareAsyaPronina commentedOct 15, 2021
AsyaPronina commentedOct 15, 2021
| #ifndef OPENCV_GAPI_STREAMING_GSTREAMER_GStreamerEnv_HPP | ||
| #defineOPENCV_GAPI_STREAMING_GSTREAMER_GStreamerEnv_HPP | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
#ifdef HAVE_GSTREAMER?
OrestChura left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Great!
| * To create GStreamerSource instance you need to pass 'pipeline' and, optionally, 'outputType' | ||
| * arguments into constructor. | ||
| * 'pipeline' should represent GStreamer pipeline in form of textual description. | ||
| * Almost any custom pipeline is supported which can be successfully ran via gst-launch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
please consider to correct
modules/gapi/src/streaming/gstreamer/gstreamer_pipeline_facade.cpp OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
6cf3d58 to0dd69f1Compare…v/integrate_gstreamer_source
0dd69f1 to7a38fb0Compare| pipelineFacade.completePreroll(); | ||
| cv::gapi::wip::gst::GStreamerPtr<GstSample>prerollSample( | ||
| // gst_app_sink_try_pull_preroll(GST_APP_SINK(appsink), 5 * GST_SECOND)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Debug code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for a catch!! This code should be version-dependent!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed!
OrestChura commentedDec 1, 2021
👍🏻 |
| std::tuple<cv::GComputation, cv::gapi::wip::GStreamerSource::OutputType> params =GetParam(); | ||
| cv::GComputation extractImage =std::move(std::get<0>(params)); | ||
| cv::gapi::wip::GStreamerSource::OutputType outputType = std::get<1>(params); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
canstd::tie() be used here instead?
| std::tuple<cv::GComputation, cv::gapi::wip::GStreamerSource::OutputType> params = GetParam(); | |
| cv::GComputation extractImage = std::move(std::get<0>(params)); | |
| cv::gapi::wip::GStreamerSource::OutputType outputType= std::get<1>(params); | |
| cv::GComputation extractImage; | |
| cv::gapi::wip::GStreamerSource::OutputType outputType =/*default value*/0; | |
| std::tie(extractImage, outputType) = GetParam(); |
Is it because GComputation can't be declared like that?
dmatveev left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Merge this please
alalek commentedDec 6, 2021
PR should not be merged in current form due to broken compilation in Custom builder with GStreamer: |
5309176 to97fadfcCompare…treamer_sourcePorted GStreamerSource to OpenCV* Ported GStreamerSource to OpenCV* Fixed CI failures* Whitespaces* Whitespaces + removed exception from destructors C4722* Removed assert for Priv's getSS and descr_of* Removed assert for pull* Fixed last review commentCo-authored-by: Pashchenkov Maxim <maxim.pashchenkov@intel.com>
Uh oh!
There was an error while loading.Please reload this page.
Pull Request Readiness Checklist
See details athttps://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.
Build configuration
This PR is about porting of GStreamer streaming source for G-API pipeline functionality to public OpenCV.