Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Ported GStreamerSource to OpenCV#20709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@AsyaPronina
Copy link
Contributor

@AsyaProninaAsyaPronina commentedSep 16, 2021
edited
Loading

Pull Request Readiness Checklist

See details athttps://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

Build configuration

force_builders=Custom,Custom Win,Custom Macbuild_gapi_standalone:Linux x64=ade-0.1.1fbuild_gapi_standalone:Win64=ade-0.1.1fbuild_gapi_standalone:Mac=ade-0.1.1fbuild_gapi_standalone:Linux x64 Debug=ade-0.1.1fXbuild_image:Custom=centos:7Xbuildworker:Custom=linux-1build_gapi_standalone:Custom=ade-0.1.1fbuild_image:Custom=ubuntu-openvino-2021.4.1:20.04build_image:Custom Win=openvino-2021.2.0build_image:Custom Mac=openvino-2021.2.0test_modules:Custom=gapi,python2,python3,javatest_modules:Custom Win=gapi,python2,python3,javatest_modules:Custom Mac=gapi,python2,python3,javabuildworker:Custom=linux-1# disabled due high memory usage: test_opencl:Custom=ONtest_opencl:Custom=OFFtest_bigdata:Custom=1test_filter:Custom=*

This PR is about porting of GStreamer streaming source for G-API pipeline functionality to public OpenCV.

Copy link
Contributor

@sivanov-worksivanov-work left a comment
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

please pay attention about GStreamerPtr assigning pointer ourselves problem at least. Moreover, from my perspective, whole class implementation is danger


voidmapBufferToFrame(GstBuffer* buffer, GstVideoInfo* info, GstVideoFrame* frame,
GstMapFlags mapFlags) {
GAPI_Assert(info &&"GstVideoInfo is absent during GstBuffer mapping");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

You can get rid of validity checking by passing references as argument (for buffer and frame also)

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed, thanks!

GST_MAP_WRITE);
m_mappedForWrite.store(true, std::memory_order_release);
}else {
gstreamer_utils::mapBufferToFrame(m_buffer, m_videoInfo, &m_videoFrame,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

thread 1, R: goes here and make mapBufferToFrame
thread 2, W goes line 67-68 and make gst_video_frame_unmap AND putfalse inm_isMapped <context switch)
thread 1, R awakes and setm_isMapped totrue at line 95.

Does this situation lead to broken invariants?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This situation should never happen according to framework design

GStreamerPtr&operator=(GStreamerPtr&&) =delete;

private:
T* ptr;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

How's about to do not use home-grown implementation but default and highly-testedstd::unique_ptr with custom dtor :)

tempate<...>using GStreamerPtr = std::unique_ptr<Type, decltype(GStreamerPtrRelease<Type>)>make_ptr(Type* ptr) {return GStreamerPtr (ptr, &GStreamerPtrRelease<Type>);}

}

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It is very good suggestion

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed! Thanks!

m_outputType(outputType)
{
auto appsinks = m_pipeline->getElementsByFactoryName("appsink");
GAPI_Assert(1ul == appsinks.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

What do you think to add warning log message additionally?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think that this is very good idea

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed, thanks!

}
break;
}
default: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

from my point of view we should handle this situation at construction time

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't understand, what do you mean here? Could you please add some more details?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

sure, I mean do not allow to create Source with invalidm_output here
https://github.com/opencv/opencv/pull/20709/files#diff-1387240a1379983779d72695d2b19e9c70195f80ab2784ef8dd3f4885cf78e3fR64

and to remove this switch-case like check from other part of a Source code

AsyaPronina reacted with thumbs up emoji
Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks! Got the idea

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed, thanks!

gst_app_sink_set_emit_signals(GST_APP_SINK(m_appsink.get()),FALSE);

GStreamerPtr<GstCaps>nv12Caps(gst_caps_from_string(
"video/x-raw,format=NV12;video/x-raw(memory:DMABuf),format=NV12"));/* transfer full*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

suggest to make global constant

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed, thanks!

@AsyaProninaAsyaProninaforce-pushed theasyadev/integrate_gstreamer_source branch 4 times, most recently from0557257 to98ccf8fCompareOctober 1, 2021 22:29
//
// Copyright 2021 Intel Corporation.
//
// This software and the related documents are Intel copyrighted materials,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Use the right license header.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for the catch!

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed, thanks!

src/streaming/gstreamer/gstreamersource.cpp
src/streaming/gstreamer/gstreamer_buffer_utils.cpp
src/streaming/gstreamer/gstreamer_media_adapter.cpp
src/streaming/gstreamer/gstreamerinit.cpp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Don't forget to add:
"${CMAKE_CURRENT_LIST_DIR}/include/opencv2/${name}/streaming/gstreamer/*.hpp" togapi_ext_hdrs.
Could I also ask you to add:
"${CMAKE_CURRENT_LIST_DIR}/include/opencv2/${name}/streaming/onevpl/*.hpp" to there ?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks! Sure!

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Done, please check!

GStreamerSource(const std::string& pipeline,
const GStreamerSource::OutputType outputType =
GStreamerSource::OutputType::MAT);
GStreamerSource(std::shared_ptr<GStreamerPipelineFacade> pipeline,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is thector really needed inpublic section ?
Since there is only forward declaration forGStreamerPipelineFacade it's supposed to be inprivate section

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

No, no, this one is used inGStreamerPipeline class, and these classes are not friends.

cv::GComputationc(cv::GIn(in),cv::GOut(out));

// Graph compilation for streaming mode:
auto ccomp = c.compileStreaming(std::move(cv::compile_args(cv::gapi::core::cpu::kernels())));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

U don't need to pass this package explicitly, do you ?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yes, I don't, thanks!

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed, thanks!

@TolyaTalamanov
Copy link
Contributor

I've opened this one:#20832
based on your branch

@TolyaTalamanov
Copy link
Contributor

TolyaTalamanov commentedOct 7, 2021
edited
Loading

Whitespaces:

modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamersource.hpp:33: trailing whitespace.+ * modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamersource.hpp:42: trailing whitespace.+ *        modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamersource.hpp:44: trailing whitespace.+ * modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamersource.hpp:51: trailing whitespace.+ * modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamersource.hpp:54: trailing whitespace.+ * modules/gapi/include/opencv2/gapi/streaming/gstreamer/gstreamersource.hpp:70: trailing whitespace.+ modules/gapi/src/streaming/gstreamer/gstreamer_pipeline_facade.cpp:62: trailing whitespace.+// The destructors are noexcept by default (since C++11). modules/gapi/src/streaming/gstreamer/gstreamer_pipeline_facade.hpp:69: trailing whitespace.+    std::string m_pipelineDesc; modules/gapi/src/streaming/gstreamer/gstreamerinit.hpp:26: trailing whitespace.+ * modules/gapi/src/streaming/gstreamer/gstreamerinit.hpp:27: trailing whitespace.+ * modules/gapi/src/streaming/gstreamer/gstreamerptr.hpp:69: trailing whitespace.+{ modules/gapi/src/streaming/gstreamer/gstreamersource_priv.hpp:60: trailing whitespace.+    GstBuffer*                             m_buffer = nullptr; // Actual frame memory holder modules/gapi/test/streaming/gapi_gstreamer_pipeline_facade_int_tests.cpp:125: trailing whitespace.+    GstStateChangeReturn status = modules/gapi/test/streaming/gapi_gstreamersource_tests.cpp:66: trailing whitespace.+    // Streaming - pulling of frames until the end:  modules/gapi/test/streaming/gapi_gstreamersource_tests.cpp:333: trailing whitespace.+ modules/gapi/test/streaming/gapi_gstreamersource_tests.cpp:339: trailing whitespace.+    cv::gapi::wip::Data leftImData, rightImData; modules/gapi/test/streaming/gapi_gstreamersource_tests.cpp:347: trailing whitespace.+    std::tuple<cv::GComputation, cv::gapi::wip::GStreamerSource::OutputType> params = GetParam();

@AsyaProninaAsyaProninaforce-pushed theasyadev/integrate_gstreamer_source branch from98ccf8f to37339d2CompareOctober 7, 2021 23:30
Copy link
Contributor

@sivanov-worksivanov-work left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM

"GStreamer pipeline has not been created!");

PipelineState state;
GstClockTime timeout = 5 * GST_SECOND;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Then what happened if query state would reach timeout?

}
}

template<typename T>staticinlinevoidGStreamerPtrRelease(T** pPtr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

No, i mean - use determined specializations for several types of T, but when user or-someone-else just call this function with something not related to correct GStreamer resource, forFoo class for example. Then correct specialization would not be deducted and compiler generated declaration forFoo, which has no body definition and user would face with LINK error, at least

The good approach is do not wait for LINK error but break compilation with some message usingstatic_assert

Something like that:

template<T> GStreamerRelease( T***) {static_assert(std::is_same<T, ...>::value,  "Unsupported type in GStreamer Release")}

The part withstd::is_same can be more intellectual with variadic templates

Feel free to ignore this comment

AsyaPronina reacted with thumbs up emoji
}
break;
}
default: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

sure, I mean do not allow to create Source with invalidm_output here
https://github.com/opencv/opencv/pull/20709/files#diff-1387240a1379983779d72695d2b19e9c70195f80ab2784ef8dd3f4885cf78e3fR64

and to remove this switch-case like check from other part of a Source code

AsyaPronina reacted with thumbs up emoji
Comment on lines +34 to +29
* To create GStreamerSource instance you need to pass 'pipeline' and, optionally, 'outputType'
* arguments into constructor.
* 'pipeline' should represent GStreamer pipeline in form of textual description.
* Almost any custom pipeline is supported which can be successfully ran via gst-launch.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change
* To create GStreamerSource instance you need to pass'pipeline'and, optionally,'outputType'
* arguments into constructor.
*'pipeline' should represent GStreamer pipeline in form of textual description.
* Almost any custom pipeline is supported which can be successfullyran via gst-launch.
* To createaGStreamerSource instance you need to pass'pipeline'and, optionally,'outputType'
* arguments into constructor.
*'pipeline' should representaGStreamer pipeline in form of textual description.
* Almost any custom pipeline is supported which can be successfullyrun via gst-launch.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It is irregular verb, this is why "ran" is used here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yes, but here is 3rd form needed, which is "run"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

please consider to correct

@AsyaProninaAsyaProninaforce-pushed theasyadev/integrate_gstreamer_source branch 5 times, most recently fromff807b5 to7c22705CompareOctober 11, 2021 09:20
@AsyaPronina
Copy link
ContributorAuthor

@AsyaPronina
Copy link
ContributorAuthor


#ifndef OPENCV_GAPI_STREAMING_GSTREAMER_GStreamerEnv_HPP
#defineOPENCV_GAPI_STREAMING_GSTREAMER_GStreamerEnv_HPP

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

#ifdef HAVE_GSTREAMER?

OrestChura reacted with thumbs up emoji
Copy link
Contributor

@OrestChuraOrestChura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Great!

Comment on lines +34 to +29
* To create GStreamerSource instance you need to pass 'pipeline' and, optionally, 'outputType'
* arguments into constructor.
* 'pipeline' should represent GStreamer pipeline in form of textual description.
* Almost any custom pipeline is supported which can be successfully ran via gst-launch.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

please consider to correct

@AsyaProninaAsyaProninaforce-pushed theasyadev/integrate_gstreamer_source branch 5 times, most recently from6cf3d58 to0dd69f1CompareNovember 14, 2021 21:58
@AsyaProninaAsyaProninaforce-pushed theasyadev/integrate_gstreamer_source branch from0dd69f1 to7a38fb0CompareNovember 22, 2021 13:23
@dmatveevdmatveev added this to the4.5.5 milestoneNov 26, 2021
pipelineFacade.completePreroll();

cv::gapi::wip::gst::GStreamerPtr<GstSample>prerollSample(
// gst_app_sink_try_pull_preroll(GST_APP_SINK(appsink), 5 * GST_SECOND));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Debug code?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for a catch!! This code should be version-dependent!

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed!

@OrestChura
Copy link
Contributor

👍🏻

Comment on lines +336 to +338
std::tuple<cv::GComputation, cv::gapi::wip::GStreamerSource::OutputType> params =GetParam();
cv::GComputation extractImage =std::move(std::get<0>(params));
cv::gapi::wip::GStreamerSource::OutputType outputType = std::get<1>(params);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

canstd::tie() be used here instead?

Suggested change
std::tuple<cv::GComputation, cv::gapi::wip::GStreamerSource::OutputType> params = GetParam();
cv::GComputation extractImage = std::move(std::get<0>(params));
cv::gapi::wip::GStreamerSource::OutputType outputType= std::get<1>(params);
cv::GComputation extractImage;
cv::gapi::wip::GStreamerSource::OutputType outputType =/*default value*/0;
std::tie(extractImage, outputType) = GetParam();

Is it because GComputation can't be declared like that?

@dmatveevdmatveev self-assigned thisDec 6, 2021
Copy link
Contributor

@dmatveevdmatveev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Merge this please

@alalek
Copy link
Member

PR should not be merged in current form due to broken compilation in Custom builder with GStreamer:

/build/precommit_custom_linux/4.x/opencv/modules/gapi/test/streaming/gapi_gstreamer_pipeline_facade_int_tests.cpp: In member function 'virtual void opencv_test::GStreamerPipelineFacadeTest_CompletePrerollUnitTest_Test::Body()':/build/precommit_custom_linux/4.x/opencv/modules/gapi/test/streaming/gapi_gstreamer_pipeline_facade_int_tests.cpp:95:13: error: 'gst_app_sink_try_pull_prerollGST_APP_SINK' was not declared in this scope; did you mean 'gst_app_sink_try_pull_preroll'?   95 |             gst_app_sink_try_pull_prerollGST_APP_SINK(appsink), 5 * GST_SECOND));      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~      |             gst_app_sink_try_pull_preroll/build/precommit_custom_linux/4.x/opencv/modules/gapi/test/streaming/gapi_gstreamer_pipeline_facade_int_tests.cpp:95:80: error: expected ',' or ';' before ')' token   95 |             gst_app_sink_try_pull_prerollGST_APP_SINK(appsink), 5 * GST_SECOND));      |                                                                                ^make[2]: *** [modules/gapi/CMakeFiles/opencv_test_gapi.dir/build.make:1025: modules/gapi/CMakeFiles/opencv_test_gapi.dir/test/streaming/gapi_gstreamer_pipeline_facade_int_tests.cpp.o] Error 1

@AsyaProninaAsyaProninaforce-pushed theasyadev/integrate_gstreamer_source branch from5309176 to97fadfcCompareDecember 6, 2021 14:42
@alalekalalek merged commit8dd6882 intoopencv:4.xDec 6, 2021
@alalekalalek mentioned this pull requestDec 30, 2021
@alalekalalek mentioned this pull requestFeb 22, 2022
a-sajjad72 pushed a commit to a-sajjad72/opencv that referenced this pull requestMar 30, 2023
…treamer_sourcePorted GStreamerSource to OpenCV* Ported GStreamerSource to OpenCV* Fixed CI failures* Whitespaces* Whitespaces + removed exception from destructors C4722* Removed assert for Priv's getSS and descr_of* Removed assert for pull* Fixed last review commentCo-authored-by: Pashchenkov Maxim <maxim.pashchenkov@intel.com>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@alalekalalekalalek left review comments

@TolyaTalamanovTolyaTalamanovTolyaTalamanov left review comments

@OrestChuraOrestChuraOrestChura approved these changes

@mpashchenkovmpashchenkovmpashchenkov left review comments

@sivanov-worksivanov-worksivanov-work approved these changes

@dmatveevdmatveevdmatveev approved these changes

@rgarnovrgarnovAwaiting requested review from rgarnov

@smirnov-alexeysmirnov-alexeyAwaiting requested review from smirnov-alexey

Assignees

@dmatveevdmatveev

Projects

None yet

Milestone

4.5.5

Development

Successfully merging this pull request may close these issues.

8 participants

@AsyaPronina@TolyaTalamanov@OrestChura@alalek@dmatveev@mpashchenkov@sivanov-work@asmorkalov

[8]ページ先頭

©2009-2025 Movatter.jp