Background
When a user uses cloud mobile phone service, the user does not need to need a camera of a local mobile phone of the user to perform real-time video streaming, and the current cloud mobile phone cannot provide service, so that the interactive functions of scanning payment, face recognition login, real-time video monitoring, photographing and the like of a remote cloud mobile phone cannot be provided, and the real experience of the cloud mobile phone of the user is reduced to a certain extent.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide a cloud mobile phone video acquisition method, a system, a device and a storage medium which can improve user experience.
The technical scheme adopted by the invention is as follows:
a cloud mobile phone video acquisition method comprises the following steps:
carrying out video acquisition through a camera of a local mobile phone to obtain video stream data;
coding the video stream data to obtain coded video stream data;
and transmitting the coded video stream data to the cloud mobile phone through a video stream transmission protocol.
As a further improvement of the cloud mobile phone video acquisition method, the encoding processing of the video stream data specifically includes:
the video stream data is subjected to H264 encoding processing.
As a further improvement of the cloud mobile phone video acquisition method, the encoding processing is performed on the video stream data to obtain encoded video stream data, and the method specifically includes:
creating an encoding session layer;
and starting to encode the video stream data according to the preset real-time encoding output, the bit rate and the frame rate to obtain the encoded video stream data.
As a further improvement of the cloud mobile phone video acquisition method, the encoding processing is performed on the video stream data to obtain encoded video stream data, and the step further includes:
when the image of the video stream data is coded, each frame of image is sent to a coder through a queue interface to be coded and recalled in real time.
As a further improvement of the cloud mobile phone video acquisition method, the video streaming protocol adopts a so transmission protocol.
As a further improvement of the cloud mobile phone video acquisition method, the method further comprises the following steps:
when the cloud mobile phone receives the coded video stream data, the camera is triggered to capture the callback, so that the local mobile phone receives the cloud mobile phone trigger time node, and the camera of the local mobile phone is synchronized with the cloud mobile phone.
The other technical scheme adopted by the invention is as follows:
a cloud cell-phone video capture system includes:
the local acquisition unit is used for carrying out video acquisition through a camera of a local mobile phone to obtain video stream data;
the encoding unit is used for encoding the video stream data to obtain encoded video stream data;
and the transmission unit is used for transmitting the coded video stream data to the cloud mobile phone through a video stream transmission protocol.
As a further improvement of the cloud mobile phone video capture system, the method further comprises:
the synchronization unit is used for triggering the camera to capture the callback when the cloud mobile phone receives the coded video stream data, so that the local mobile phone receives the cloud mobile phone trigger time node, and the camera of the local mobile phone is synchronized with the cloud mobile phone.
The invention adopts another technical scheme that:
a cloud cell-phone video capture device includes:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the cloud cell phone video capture method.
The invention adopts another technical scheme that:
a computer-readable storage medium comprising a computer program which, when run on a computer, causes the cloud-phone video capture method to be performed.
The invention has the beneficial effects that:
the cloud mobile phone video acquisition method, the cloud mobile phone video acquisition system, the cloud mobile phone video acquisition device and the storage medium can control the camera of the local mobile phone to acquire videos through the cloud mobile phone, so that the problems of cloud mobile phone face recognition, two-dimension code scanning payment, login, cloud mobile phone video monitoring, cloud picture storage and the like can be solved, the cloud mobile phone can play the largest functions under the hardware of the cloud mobile phone, and the user experience can be greatly improved.
Detailed Description
The following further describes embodiments of the present invention with reference to the accompanying drawings:
referring to fig. 1, an embodiment of the present invention provides a cloud mobile phone video acquisition method, including the following steps:
s1, carrying out video acquisition through a camera of the local mobile phone to obtain video stream data;
in this embodiment, data acquisition of the camera is completed by an AVCaptureSession class of an avfoundation. frame, and it is responsible for allocating data streams between audio and video input and output, and a session layer adds configuration input and output objects, which are connected to a CaptureSession through an avcapturedeviceinfput, and the output is an AVCaptureVideoDataOutput, and can take acquired video data through the AVCaptureSession class to start the AVCaptureSession. The captured video data is taken in a proxy following avcapturevideodataoutputsamplebufferdelete. The CMSampleBuffer stores a container data structure of the video image before and after encoding and decoding, the stored data is uncoded camera data, and the data before encoding the CVImageBuffer can be taken through a CMSampleBufferGetImageBuffer () interface.
S2, coding the video stream data to obtain coded video stream data;
and S3, transmitting the coded video stream data to the cloud mobile phone through a video stream transmission protocol.
In this embodiment, when the user operates the cloud mobile phone app software at the client, the user interacts with the cloud app with functions of code scanning, photographing, AR, and the like to listen to the callback, calling a VIDEO control function interface for processing through a hardware bottom INPUT/output port and a remote-play program, establishing a service link and passing through a control service penetration, triggering an EVENT _ INPUT _ VIDEO _ DATA by a so dynamic library instruction, receiving an instruction locally, initializing a camera parameter, starting a camera, encoding a VIDEO stream, writing a real-time VIDEO stream into a so dynamic library function sendInputvideo function frame, establishing the service link and passing through the control service penetration, and receiving the processing of a VIDEO control function interface at a remote-play program of a cloud mobile phone hardware driving layer, performing VIDEO decoding, driving virtual cloud mobile phone equipment on a remote side of the remote-play, acquiring VIDEO stream frames for transmission, and completing a drawing rasterization rendering stage at running opportunity runloop by a client under a callback instruction of an EVENT _ VIDEO _ UPDATE _ DATA UPDATE image of a dynamic library. And the cloud software receives a frame of video stream, and completes functions of recognizing and scanning codes and the like, so that the whole process of matching different sound code rates according to different image qualities is completed.
Further as a preferred embodiment, the encoding process performed on the video stream data specifically includes:
the video stream data is subjected to H264 encoding processing.
Further as a preferred embodiment, the encoding processing on the video stream data to obtain encoded video stream data specifically includes:
creating an encoding session layer;
and starting to encode the video stream data according to the preset real-time encoding output, the bit rate and the frame rate to obtain the encoded video stream data.
In this embodiment, a Session layer Session is created, a real-time encoding output is set, encoding delay is reduced, an appropriate bit rate and frame rate are set, encoding is started, source data (yuv type data, obtained directly from a camera) is circularly input, encoded H264 data is obtained to end encoding, in encoding a frame of image, queue is used to prevent blocking of a system camera acquisition thread, encoding is sent to an encoder encoding vtcompressionsessionencodeckle frame, real-time encoding callback is performed, whether a current frame is a key frame and sps & pps data is judged in the callback, note that an information frame only needs to be obtained once, a memory address of a start position is obtained from a CMBlockBufferRef, an image of a frame may need to be written into a plurality of NALU units, NALU data is circularly obtained, an address of a length of a H264 Headrength is copied from the start position, naluguentLength is calculated, data encoded by H264 is a big-end mode (byte order), PoNSOffset is created from a tadata, and a byte order is reset, move to write a block, convert to NALU unit, and operate in a loop.
Further as a preferred embodiment, the encoding process of the video stream data to obtain encoded video stream data further includes:
when the image of the video stream data is coded, each frame of image is sent to a coder through a queue interface to be coded and recalled in real time.
Further, as a preferred embodiment, the video streaming protocol adopts so transport protocol.
Further as a preferred embodiment, the method further comprises the following steps:
when the cloud mobile phone receives the coded video stream data, the camera is triggered to capture the callback, so that the local mobile phone receives the cloud mobile phone trigger time node, and the camera of the local mobile phone is synchronized with the cloud mobile phone.
When a frame of VIDEO stream is acquired and is subjected to frame separation and transmission through a so transport protocol control layer sendlnputvideo, the nFrameType frame type (SPS, PPS, P, I), buffer (VIDEO DATA) and len (length), as a cloud mobile phone is based on a virtual system of an arm chip, a camera capture callback is triggered in hardware abnormal interruption, and an EVENT _ INPUT _ VIDEO _ DATA instruction is newly added in an information configuration instruction and used for a local mobile phone app to receive a cloud mobile phone trigger time node, so that synchronization between the local camera and the cloud mobile phone is started.
Referring to fig. 2, the present embodiment provides a cloud mobile phone video capture system, including:
the local acquisition unit is used for carrying out video acquisition through a camera of a local mobile phone to obtain video stream data;
the encoding unit is used for encoding the video stream data to obtain encoded video stream data;
and the transmission unit is used for transmitting the coded video stream data to the cloud mobile phone through a video stream transmission protocol.
The contents in the above method embodiments are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above method embodiment, and the beneficial effects achieved by the present system embodiment are also the same as those achieved by the above method embodiment.
Further, as a preferred embodiment, the method further comprises:
the synchronization unit is used for triggering the camera to capture the callback when the cloud mobile phone receives the coded video stream data, so that the local mobile phone receives the cloud mobile phone trigger time node, and the camera of the local mobile phone is synchronized with the cloud mobile phone.
This embodiment also provides a cloud cell-phone video acquisition device, includes:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the cloud cell phone video capture method.
The contents in the above method embodiments are all applicable to the present apparatus embodiment, the functions specifically implemented by the present apparatus embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present apparatus embodiment are also the same as those achieved by the above method embodiments.
The embodiment also provides a computer-readable storage medium, which is characterized by comprising a computer program, and when the computer program runs on a computer, the cloud mobile phone video capture method is executed.
The contents in the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present storage medium embodiment are also the same as those achieved by the above method embodiments.
In summary, the cloud mobile phone video acquisition method, system, device and storage medium provided by the invention can control the camera of the local mobile phone to acquire video through the cloud mobile phone, so that the problems of cloud mobile phone face recognition, two-dimensional code scanning payment, login, cloud mobile phone video monitoring, cloud picture storage and the like can be solved, the cloud mobile phone can exert the maximum function under the hardware of the cloud mobile phone, and the user experience can be greatly improved.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.