技术领域technical field
本发明涉及以智能终端同时作为主机、鼠标和触摸板的系统及方法。The invention relates to a system and a method using an intelligent terminal as a host, a mouse and a touch panel at the same time.
背景技术Background technique
随着智能手机／平板电脑等便携式智能终端的迅速普及和智能终端运算能力的不断提高，以及成千上万、丰富多彩的软件应用的涌现，智能终端正在越来越多的场合取代甚至超越传统的PC或笔记本电脑。事实上，现在的智能终端已经可以连接电脑显示器、键盘和鼠标，使用户可以像使用PC一样使用智能终端，为用户在办公室或家里需要处理文字、邮件或运行较复杂的应用时提供了方便，并且可以节省购买传统的个人电脑的费用。With the rapid popularization of portable intelligent terminals such as smartphones/tablets and the continuous improvement of the computing power of intelligent terminals, as well as the emergence of thousands of rich and colorful software applications, intelligent terminals are replacing or even surpassing traditional PC or laptop. In fact, today's smart terminals can be connected to computer monitors, keyboards and mice, so that users can use smart terminals like PCs, which provides convenience for users when they need to process text, emails or run more complex applications in the office or at home. And can save the cost of buying a traditional personal computer.
目前智能终端是通过各种接口连接电脑显示器、键盘和鼠标，如通过MHL端口/AirPlay等有线或者无线方式连接显示器，通过蓝牙或USB端口连接键盘和鼠标，从而使用户可以像使用PC一样使用智能终端。然而，目前的智能终端连接显示器和键盘作为主机使用时，还需要单独购置鼠标/触摸板，外出使用时也需额外携带鼠标/触摸板，由于鼠标／触摸板体积也不小，这必然导致携带有所不便。At present, smart terminals are connected to computer monitors, keyboards and mice through various interfaces, such as MHL ports/AirPlay and other wired or wireless methods to connect monitors, and Bluetooth or USB ports to connect keyboards and mice, so that users can use smart phones like PCs. terminal. However, when the current smart terminal is connected to a display and a keyboard as a host computer, it is necessary to purchase a mouse/touchpad separately, and it is also necessary to carry an additional mouse/touchpad when going out for use. Since the mouse/touchpad is not small in size, this will inevitably lead to Inconvenience.
发明内容Contents of the invention
本发明的目的在于克服现有技术的不足，提供一种操作简单、携带方便的以智能终端同时作为主机、鼠标和触摸板的系统及方法，在智能终端作为主机使用时，智能终端还能实现虚拟鼠标和触摸板的功能，无需另外连接外围鼠标和触摸板，不仅节省成本，且携带使用方便。The purpose of the present invention is to overcome the deficiencies of the prior art, and provide a system and method that is simple to operate and easy to carry, using a smart terminal as a host, mouse and touchpad at the same time. When the smart terminal is used as a host, the smart terminal can also realize The function of virtual mouse and touchpad does not need to be connected with peripheral mouse and touchpad, which not only saves cost, but also is easy to carry and use.
本发明的目的是通过以下技术方案来实现的：以智能终端同时作为主机、鼠标和触摸板的系统，它包括智能终端和外部显示器，智能终端和外部显示器通过有线或无线连接，所述智能终端包括CPU、图形处理器、只读内存、随机存储器、外部存储器、触摸板、显示屏、运动传感器、无线网络模块、移动通信模块和蓝牙模块。所述智能终端还包括显示控制模块、模式切换控制模块和多功能触摸板控制模块，所述显示控制模块用于侦测系统于外部显示器的连接状态，驱动外部显示器并在智能终端内置显示屏上展示虚拟鼠标按键及其用户交互过程；模式切换控制模块用于实现主机模式与终端模式的切换；多功能触摸板控制模块负责将智能终端的触摸屏虚拟成电脑的鼠标与触摸板，同时控制智能终端内置显示屏上虚拟鼠标按键的展示，感应用户在触摸屏上的点按、移动、缩放、旋转等动作并进行处理。The purpose of the present invention is achieved through the following technical solutions: a system with an intelligent terminal as a host, a mouse and a touch panel simultaneously, which includes an intelligent terminal and an external display, and the intelligent terminal and the external display are connected by wire or wirelessly, and the intelligent terminal Including CPU, graphics processing unit, read-only memory, random access memory, external memory, touch panel, display screen, motion sensor, wireless network module, mobile communication module and bluetooth module. The smart terminal also includes a display control module, a mode switching control module and a multi-function touch panel control module, the display control module is used to detect the connection status of the system to the external display, drive the external display and display the display on the built-in display screen of the smart terminal Display the virtual mouse button and its user interaction process; the mode switching control module is used to realize the switching between the host mode and the terminal mode; the multi-function touch panel control module is responsible for virtualizing the touch screen of the smart terminal into a computer mouse and touch pad, and simultaneously controlling the smart terminal Display of the virtual mouse button on the built-in display screen, sense and process the user's click, move, zoom, rotate and other actions on the touch screen.
所述多功能触摸板控制模块包括虚拟鼠标控制模块和多点触控控制模块；虚拟鼠标控制模块用于捕捉智能终端的运动传感器（如加速度传感器、陀螺仪等）的信息，控制并模拟鼠标的移动，同时在智能终端内置显示屏上展示虚拟鼠标按键，并将智能终端触摸屏上对应的位置虚拟成鼠标按键，对用户的鼠标按键动作进行处理；所述多点触控控制模块用于响应用户在触摸板上的上的点按、移动、旋转与缩放操作。The multi-functional touch panel control module includes a virtual mouse control module and a multi-touch control module; the virtual mouse control module is used to capture information of motion sensors (such as acceleration sensors, gyroscopes, etc.) move, and at the same time display virtual mouse buttons on the built-in display screen of the smart terminal, virtualize the corresponding position on the touch screen of the smart terminal into a mouse button, and process the user's mouse button action; the multi-touch control module is used to respond to user Click, move, rotate and zoom on the touchpad.
进一步的，所述多功能触摸板控制模块还包括虚拟键盘控制模块，虚拟键盘控制模块用于在智能终端内置显示屏上展示虚拟键盘按键，并通过智能终端触摸屏对用户在虚拟键盘上的操作进行处理。Further, the multifunctional touch panel control module also includes a virtual keyboard control module, the virtual keyboard control module is used to display the virtual keyboard keys on the built-in display screen of the intelligent terminal, and perform operations on the virtual keyboard by the user through the touch screen of the intelligent terminal. deal with.
所述的虚拟鼠标控制模块包括虚拟按键模块和虚拟运动模块，虚拟运动模块用于捕捉智能终端的运动传感器的信息，控制并模拟鼠标的移动，虚拟按键模块用于在智能终端内置显示屏上展示虚拟鼠标按键并响应按键操作。The virtual mouse control module includes a virtual button module and a virtual motion module, the virtual motion module is used to capture the information of the motion sensor of the smart terminal, controls and simulates the movement of the mouse, and the virtual button module is used to display on the built-in display screen of the smart terminal Virtual mouse buttons and respond to key presses.
所述的模式切换控制模块包括自动模式切换控制模块和手动模式切换控制模块。The mode switching control module includes an automatic mode switching control module and a manual mode switching control module.
所述的手动模式切换控制模块包括位于智能终端屏幕上的切换功能快捷键、位于智能终端机身上的切换按钮和位于智能终端功能菜单下的切换按键。The manual mode switch control module includes a switch function shortcut key on the smart terminal screen, a switch button on the smart terminal body and a switch button under the function menu of the smart terminal.
以智能终端同时作为主机、鼠标和触摸板的实现方法，它包括如下步骤：The realization method of using the intelligent terminal as the host computer, the mouse and the touch panel at the same time includes the following steps:
S0：将智能终端与外部连接器连接，连接方式包括有线方式和无线方式；S0: Connect the smart terminal with the external connector, the connection methods include wired and wireless;
S1：将智能终端按设定的切换模式从终端模式切换到主机模式；S1: Switch the smart terminal from terminal mode to host mode according to the set switching mode;
S2：显示控制模块驱动外部显示器，自动适应外部显示器的分辨率，使外部显示器正常显示原智能终端屏幕的内容和鼠标指针，并按设定的显示模式控制智能终端内置显示屏的开启或关闭，所述显示模式具体包括：S2: The display control module drives the external display, automatically adapts to the resolution of the external display, makes the external display normally display the content of the original smart terminal screen and the mouse pointer, and controls the opening or closing of the built-in display screen of the smart terminal according to the set display mode, The display modes specifically include:
（1）双显示模式：外部显示器与智能终端内置显示屏同时开启，智能终端内置显示屏显示虚拟鼠标按键和多点触控区域，智能终端兼具鼠标与触摸板功能，智能终端的显示控制模块检测外部显示器的分辨率并控制外部显示器显示屏幕内容及鼠标指针；(1) Dual display mode: the external display and the built-in display of the smart terminal are turned on at the same time, the built-in display of the smart terminal displays virtual mouse buttons and multi-touch areas, the smart terminal has both mouse and touchpad functions, and the display control module of the smart terminal Detect the resolution of the external display and control the display screen content and mouse pointer of the external display;
（2）单显示模式：智能终端内置显示屏关闭，外部显示器开启，智能终端兼具鼠标与触摸板功能，智能终端的显示控制模块检测外部显示器的分辨率并控制外部显示器显示屏幕内容及鼠标指针；(2) Single display mode: the built-in display screen of the smart terminal is turned off, the external display is turned on, the smart terminal has both mouse and touchpad functions, and the display control module of the smart terminal detects the resolution of the external display and controls the external display to display the screen content and the mouse pointer ;
S3：虚拟运动模块捕捉运动传感器的信息，进而根据智能终端的移动，模拟鼠标的移动个，虚拟按键模块利用智能终端触摸屏响应鼠标按键动作，多点触控控制模块捕捉用户的触摸板手势动作，经多点触控控制模块处理后，执行相应的动作；S3: The virtual motion module captures the information of the motion sensor, and then simulates the movement of the mouse according to the movement of the smart terminal. The virtual button module uses the touch screen of the smart terminal to respond to the mouse button action, and the multi-touch control module captures the user's touchpad gestures. Execute corresponding actions after being processed by the multi-touch control module;
S4：根据用户操作指令退出主机模式，返回终端模式。S4: Exit the host mode according to the user operation instruction, and return to the terminal mode.
所述切换模式包括如下：The switching modes include the following:
（1）自动切换：智能终端的显示控制模块监测到智能终端与外部显示器相连接，智能终端读取切换模式设置，如是自动切换模式，智能终端的模式切换控制模块自动切换到主机模式，若否，则进入终端模式；(1) Automatic switching: The display control module of the smart terminal detects that the smart terminal is connected to an external display, and the smart terminal reads the switching mode setting. If it is an automatic switching mode, the mode switching control module of the smart terminal automatically switches to the host mode, if not , enter terminal mode;
（2）手动切换：当用户将智能终端与外部显示器连接后，用户手动调出系统功能菜单，手动执行切换功能；或者用户点击智能终端屏幕上的切换功能快捷键进行手动切换；或者用户通过按下智能终端机身上的切换按钮进行手动切换。(2) Manual switching: After the user connects the smart terminal to the external display, the user manually calls up the system function menu and manually executes the switching function; or the user clicks the switching function shortcut key on the smart terminal screen to perform manual switching; or the user presses the Press the switch button on the body of the smart terminal to switch manually.
所述退出主机模式的方式包括如下：The ways of exiting the host mode include the following:
（1）自动退出：智能终端的显示控制模块监测到智能终端与外部显示器断开，模式切换控制模块自动退出主机模式，进入到终端模式；(1) Automatic exit: The display control module of the intelligent terminal detects that the intelligent terminal is disconnected from the external display, and the mode switching control module automatically exits the host mode and enters the terminal mode;
（2）手动退出：用户调出系统菜单，手动退出主机模式；或者用户点击智能终端屏幕上的切换功能快捷键进行手动退出主机模式；或者用户通过智能终端机身上的切换按钮手动退出主机模式。(2) Manual exit: the user calls up the system menu and manually exits the host mode; or the user clicks the switch function shortcut key on the screen of the smart terminal to manually exit the host mode; or the user manually exits the host mode through the switch button on the smart terminal body .
所述的虚拟按键模块响应用户按键动作的步骤如下：The steps of the virtual button module responding to the user's button action are as follows:
S311:检测触控位置是否在虚拟按键范围内，若是，则进入S312，若否，则响应结束；S311: Detect whether the touch position is within the virtual key range, if so, then enter S312, if not, then the response ends;
S312：检测是否单指按下，若是，则进一步进行虚拟鼠标按键范围的判断，若否，则响应结束。S312: Detect whether a single finger is pressed, if yes, further judge the range of the virtual mouse button, if not, end the response.
所述多点触控控制模块响应用户触摸动作的步骤如下：The steps of the multi-touch control module responding to the user's touch action are as follows:
S321：检测触控位置是否在多点触控范围内，若是，则进入S322，若否，则响应结束；S321: Detect whether the touch position is within the multi-touch range, if so, enter S322, if not, end the response;
S322：检测是否单指滑动，若是，根据移动的方向、距离和速度，移动当前鼠标指针位置并通过外部显示器显示，若否，进入S323；S322: Detect whether one-finger slides, if yes, move the current mouse pointer position according to the moving direction, distance and speed and display it on an external display, if not, go to S323;
S323：检测是否系统预定义手势，若是，则执行系统预定义动作，若否，则结束响应。S323: Detect whether the gesture is predefined by the system, if yes, execute the action predefined by the system, if not, end the response.
所述虚拟鼠标按键范围包括第一虚拟鼠标按键范围、第二虚拟鼠标按键范围和第三虚拟鼠标按键范围，当检测单指按下的范围在第一虚拟鼠标按键范围内时，响应鼠标左键点击的动作，当检测单指按下的范围在第二虚拟鼠标按键范围内时，响应鼠标右键点击的动作，当检测单指按下的范围在第三虚拟鼠标按键范围内时，响应鼠标中键点击的动作。根据按键的不同设定不同的声音效果，以便给用户更好的操作反馈。The range of virtual mouse buttons includes a first range of virtual mouse buttons, a second range of virtual mouse buttons and a third range of virtual mouse buttons. Click action, when detecting that the range of single-finger pressing is within the range of the second virtual mouse button, respond to the action of clicking the right button of the mouse; when detecting that the range of single-finger pressing is within the range of the third virtual mouse button, responding to the middle of the mouse The action of a key click. Set different sound effects according to different buttons, so as to give users better operation feedback.
本发明的有益效果是：The beneficial effects of the present invention are:
（1）设有模式切换控制模块，能根据需求自动或手动的切换智能终端的工作模式，操作简单，控制智能；(1) Equipped with a mode switching control module, which can automatically or manually switch the working mode of the smart terminal according to the demand, with simple operation and intelligent control;
（2）设有多点触控控制模块和虚拟鼠标控制模块，在作为主机使用时，能同时实现虚拟鼠标及触摸板的功能，无需另外连接外接触摸板和鼠标，不仅节省成本，且携带使用方便；(2) It is equipped with a multi-touch control module and a virtual mouse control module. When used as a host, it can realize the functions of a virtual mouse and a touchpad at the same time. There is no need to connect an external touchpad and mouse, which not only saves costs, but is also portable. convenient;
（3）设有显示控制模块，能智能的驱动外部显示器并控制智能终端内置显示屏的显示状况，能将智能终端的信息定向的显示在外部显示器上，连接的外部显示器可以是电脑显示器，也可以是电视机，使用范围广；(3) It is equipped with a display control module, which can intelligently drive the external display and control the display status of the built-in display screen of the smart terminal, and can display the information of the smart terminal on the external display in a directional manner. The connected external display can be a computer monitor or It can be a TV set with a wide range of applications;
（4）能自动检测外部显示器的分辨率，进而自动合理的布局显示的内容，提高显示的清晰度。(4) It can automatically detect the resolution of the external display, and then automatically and reasonably layout the displayed content to improve the clarity of the display.
附图说明Description of drawings
图1为本发明的结构框图；Fig. 1 is a block diagram of the present invention;
图2为本发明的主控流程图；Fig. 2 is the main control flowchart of the present invention;
图3为本发明的切换流程图；Fig. 3 is the switching flowchart of the present invention;
图4为本发明显示控制的流程图；Fig. 4 is the flowchart of display control of the present invention;
图5为本发明检测并响应触摸板鼠标操作的流程图；Fig. 5 is the flow chart that the present invention detects and responds to touchpad mouse operation;
图6为智能终端不具有虚拟键盘功能时多功能触摸板的结构示意图；FIG. 6 is a schematic structural diagram of a multifunctional touch panel when the smart terminal does not have a virtual keyboard function;
图7为智能终端具有虚拟键盘功能时多功能触摸板的结构示意图。FIG. 7 is a schematic structural diagram of a multifunctional touch panel when the smart terminal has a virtual keyboard function.
具体实施方式detailed description
下面结合附图进一步详细描述本发明的技术方案，但本发明的保护范围不局限于以下所述。The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the following description.
如图1所示，以智能终端同时作为主机、鼠标和触摸板的系统，它包括智能终端（包括但不限于智能手机、平板电脑）和外部显示器，智能终端和外部显示器通过数据线或无线连接，所述智能终端包括CPU、图形处理器（GPU）、非易失存储器（包括但不限于ROM、NVRAM、Flash）、随机存储器（RAM）、外部存储器、触摸板、显示屏、运动传感器、无线网络模块、移动通信模块和蓝牙模块。所述智能终端它还包括显示控制模块、模式切换控制模块、多功能触摸板控制模块，所述显示控制模块用于侦测系统与外部显示器的连接状态，驱动外部显示器与智能终端内置显示屏的显示，并在智能终端内置显示屏上展示虚拟鼠标按键及其用户交互过程；模式切换控制模块用于实现主机模式与终端模式的切换；多功能触摸板控制模块的功能是将智能终端的触摸屏虚拟成电脑的鼠标与触摸板，同时在智能终端内置显示屏上展示虚拟鼠标按键，并对用户在触摸屏上的点按、移动、缩放、旋转等动作进行处理。As shown in Figure 1, the system uses the smart terminal as the host, mouse and touchpad at the same time, which includes smart terminals (including but not limited to smart phones, tablet computers) and external displays, and the smart terminals and external displays are connected through data lines or wirelessly , the smart terminal includes CPU, graphics processing unit (GPU), non-volatile memory (including but not limited to ROM, NVRAM, Flash), random access memory (RAM), external memory, touch panel, display screen, motion sensor, wireless Network module, mobile communication module and bluetooth module. The smart terminal also includes a display control module, a mode switch control module, and a multi-function touch panel control module. The display control module is used to detect the connection status between the system and the external display, and drive the connection between the external display and the built-in display screen of the smart terminal. display, and display the virtual mouse button and its user interaction process on the built-in display screen of the smart terminal; the mode switching control module is used to switch between the host mode and the terminal mode; It can be used as the mouse and touchpad of the computer, and at the same time, the virtual mouse buttons are displayed on the built-in display screen of the smart terminal, and the user's actions such as clicking, moving, zooming, and rotating on the touch screen are processed.
所述多功能触摸板控制模块包括虚拟鼠标控制模块和多点触控控制模块。虚拟鼠标控制模块可以通过捕捉智能终端的运动传感的信息（比如加速度传感器、陀螺仪），来模拟鼠标的移动，同时在智能终端内置显示屏上展示虚拟鼠标按键，并将智能终端触摸屏上对应的位置虚拟成鼠标按键，对用户的鼠标按键动作进行处理；所述多点触控控制模块用于响应用户在触摸屏上的点按、移动、缩放、旋转操作。The multifunctional touch panel control module includes a virtual mouse control module and a multi-touch control module. The virtual mouse control module can simulate the movement of the mouse by capturing the motion sensor information (such as acceleration sensor and gyroscope) of the smart terminal, and at the same time display the virtual mouse buttons on the built-in display screen of the smart terminal, and display the corresponding buttons on the touch screen of the smart terminal. The position of the mouse button is virtualized into a mouse button, and the user's mouse button action is processed; the multi-touch control module is used to respond to the user's click, move, zoom, and rotate operations on the touch screen.
所述多功能触摸板控制模块包括虚拟鼠标控制模块和多点触控控制模块。虚拟鼠标控制模块包括虚拟按键模块和虚拟运动模块，虚拟运动模块用于捕捉智能终端的运动传感器的信息，控制并模拟鼠标的移动，虚拟按键模块用于在智能终端内置显示屏上展示虚拟鼠标按键并响应按键操作，并将智能终端触摸屏上对应的位置虚拟成鼠标按键。所述多点触控控制模块用于响应用户在触摸板上的滑动和缩放操作。The multifunctional touch panel control module includes a virtual mouse control module and a multi-touch control module. The virtual mouse control module includes a virtual button module and a virtual motion module. The virtual motion module is used to capture the information of the motion sensor of the smart terminal to control and simulate the movement of the mouse. The virtual button module is used to display virtual mouse buttons on the built-in display screen of the smart terminal. And respond to button operations, and virtualize the corresponding position on the touch screen of the smart terminal into a mouse button. The multi-touch control module is used to respond to the user's sliding and zooming operations on the touch panel.
进一步的，所述多功能触摸板控制模块还包括虚拟键盘控制模块，虚拟键盘控制模块用于在智能终端内置显示屏上展示虚拟键盘按键，并通过智能终端触摸屏对用户在虚拟键盘上的操作进行处理。所述智能终端可通过无线或有线的方式连接外部键盘，进而实现键盘输入的功能；也可在没有外接键盘时通过智能终端上的虚拟键盘按键实现键盘输入的功能。Further, the multifunctional touch panel control module also includes a virtual keyboard control module, the virtual keyboard control module is used to display the virtual keyboard keys on the built-in display screen of the intelligent terminal, and perform operations on the virtual keyboard by the user through the touch screen of the intelligent terminal. deal with. The intelligent terminal can be connected to an external keyboard in a wireless or wired manner to realize the function of keyboard input; it can also realize the function of keyboard input through the virtual keyboard keys on the intelligent terminal when there is no external keyboard.
所述的模式切换控制模块包括自动模式切换控制模块和手动模式切换控制模块。The mode switching control module includes an automatic mode switching control module and a manual mode switching control module.
所述的手动模式切换控制模块包括位于智能终端屏幕上的切换功能快捷键、位于智能终端机身上的切换按钮和位于智能终端功能菜单下的切换按键。The manual mode switch control module includes a switch function shortcut key on the smart terminal screen, a switch button on the smart terminal body and a switch button under the function menu of the smart terminal.
所述的外部显示器包括电视机和电脑显示器。The external display includes a television and a computer monitor.
如图6所示为智能终端内置显示屏上显示的多功能触摸板，它包括多点触控区域和虚拟鼠标区域。虚拟鼠标区域包括第一虚拟鼠标按键范围、第二虚拟鼠标按键范围和第三虚拟鼠标按键范围，当检测单指按下的范围在第一虚拟鼠标按键范围内时，响应鼠标左键点击的动作，当检测单指按下的范围在第二虚拟鼠标按键范围内时，响应鼠标右键点击的动作，当检测单指按下的范围在第三虚拟鼠标按键范围内时，响应鼠标中键点击的动作。As shown in FIG. 6 , the multifunctional touchpad displayed on the built-in display screen of the smart terminal includes a multi-touch area and a virtual mouse area. The virtual mouse area includes the first virtual mouse button range, the second virtual mouse button range and the third virtual mouse button range, and when the detected single-finger press range is within the first virtual mouse button range, it responds to the left mouse button click action , when the detected single-finger pressed range is within the range of the second virtual mouse button, respond to the action of the right mouse button, and when the detected single-finger pressed range is within the range of the third virtual mouse button, respond to the middle mouse button click action.
如图7所示，通过自定义的手势动作，还可将多功能触摸板上的多点触控区域切换为虚拟键盘区域，实现键盘输入的功能。As shown in FIG. 7 , through custom gestures, the multi-touch area on the multi-function touch panel can also be switched to a virtual keyboard area to realize the function of keyboard input.
如图2所示，以智能终端同时作为主机、鼠标和触摸板的实现方法，它包括如下步骤：As shown in Figure 2, the implementation method of using the intelligent terminal as the host computer, mouse and touch panel at the same time includes the following steps:
S0：将智能终端与外部显示器连接，连接方式包括有线方式和无线方式；S0: Connect the smart terminal with an external display, the connection methods include wired and wireless methods;
S1：将智能终端按设定的切换模式从终端模式切换到主机模式；S1: Switch the smart terminal from terminal mode to host mode according to the set switching mode;
S2：显示控制模块驱动外部显示器，自动适应外部显示器的分辨率，使外部显示器正常显示原智能终端屏幕的内容和鼠标指针，并按设定的显示模式控制智能终端内置显示屏的开启或关闭，所述显示模式具体包括：S2: The display control module drives the external display, automatically adapts to the resolution of the external display, makes the external display normally display the content of the original smart terminal screen and the mouse pointer, and controls the opening or closing of the built-in display screen of the smart terminal according to the set display mode, The display modes specifically include:
（1）双显示模式：外部显示器与智能终端内置显示屏同时开启，智能终端内置显示屏显示虚拟的鼠标按键与多点触控区域，智能终端作为主机加鼠标与触摸板使用，智能终端的显示控制模块检测外部显示器的分辨率并控制外部显示器显示屏幕内容及鼠标指针；(1) Dual display mode: the external display and the built-in display of the smart terminal are turned on at the same time, the built-in display of the smart terminal displays virtual mouse buttons and multi-touch areas, the smart terminal is used as a host plus a mouse and a touchpad, and the display of the smart terminal The control module detects the resolution of the external display and controls the external display to display screen content and mouse pointer;
（2）单显示模式：智能终端内置显示屏关闭，外部显示器开启，智能终端作为主机加鼠标与触摸板使用，智能终端的显示控制模块检测外部显示器的分辨率并控制外部显示器显示屏幕内容及鼠标指针。(2) Single display mode: the built-in display screen of the smart terminal is turned off, the external display is turned on, the smart terminal is used as a host with a mouse and a touchpad, and the display control module of the smart terminal detects the resolution of the external display and controls the content of the external display and the mouse pointer.
如图4所示，显示控制模块控制显示的具体流程如下：As shown in Figure 4, the specific process of the display control module controlling the display is as follows:
S21：检测是否连接外部显示器；若是，进入S22，若否，则结束；S21: Detect whether an external display is connected; if yes, go to S22, if not, end;
S22：检测外部显示器的分辨率；S22: Detect the resolution of the external display;
S23：根据分辨率布局外部显示器的显示内容；S23: Layout the display content of the external display according to the resolution;
S24：外部显示器显示屏幕内容及鼠标指针，同时显示控制模块判断智能终端内置显示屏是否显示，若是，则进入S25，否则进入S26；S24: The external monitor displays the screen content and the mouse pointer, and at the same time, the display control module judges whether the built-in display screen of the smart terminal is displayed, if so, enters S25, otherwise enters S26;
S25：智能终端内置显示屏显示虚拟鼠标按键与多点触控区域并结束；S25: The built-in display screen of the intelligent terminal displays the virtual mouse button and the multi-touch area and ends;
S26：关闭智能终端内置显示屏屏幕背光并结束；S26: Turn off the backlight of the smart terminal's built-in display screen and end;
S3：运动传感器或陀螺仪分别检测智能终端的移动情况，多功能控制模块检测触摸信号，通过智能终端的触摸板控制模块处理后，响应相应的动作。具体为虚拟运动模块捕捉运动传感器的信息，进而捕捉智能终端的移动，根据移动的方向、速度、距离控制鼠标光标的移动，虚拟按键模块利用智能终端触摸屏响应鼠标按键动作，多点触控控制模块捕捉用户的触摸板手势动作，经多点触控控制模块处理后，执行相应的动作。S3: The motion sensor or gyroscope respectively detects the movement of the smart terminal, the multi-function control module detects the touch signal, and responds to corresponding actions after being processed by the touch panel control module of the smart terminal. Specifically, the virtual motion module captures the information of the motion sensor, and then captures the movement of the smart terminal, and controls the movement of the mouse cursor according to the direction, speed, and distance of the movement. The virtual button module uses the touch screen of the smart terminal to respond to the mouse button action. Capture the user's touchpad gestures, and execute corresponding actions after being processed by the multi-touch control module.
S4：退出主机模式，返回终端模式。S4: Exit host mode and return to terminal mode.
当智能终端兼具虚拟键盘功能时，智能终端内置显示屏显示虚拟的鼠标按键与多点触控区域或者显示虚拟的鼠标按键与虚拟键盘区域。When the smart terminal has a virtual keyboard function, the built-in display screen of the smart terminal displays virtual mouse buttons and multi-touch areas or displays virtual mouse buttons and virtual keyboard areas.
如图3所示，所述切换到主机模式的切换模式包括如下：As shown in Figure 3, the switching mode of switching to the host mode includes as follows:
（1）自动切换：智能终端的显示控制模块监测到智能终端与外部显示器相连接，智能终端判断是否切换为主机模式，若是，智能终端的模式切换控制模块自动切换到主机模式，若否，则进入终端模式；(1) Automatic switching: The display control module of the smart terminal detects that the smart terminal is connected to an external display, and the smart terminal judges whether to switch to the host mode. If so, the mode switching control module of the smart terminal automatically switches to the host mode. If not, then enter terminal mode;
（2）手动切换：用户调出系统功能菜单，手动切换；或者用户点击智能终端屏幕上的切换功能快捷键进行手动切换；或者用户通过智能终端机身上的切换按钮手动切换。(2) Manual switching: the user calls up the system function menu and manually switches; or the user clicks the switching function shortcut key on the smart terminal screen to manually switch; or the user manually switches through the switching button on the smart terminal body.
所述退出主机模式的方式包括如下：The ways of exiting the host mode include the following:
（1）自动退出：智能终端的显示控制模块监测到智能终端与外部显示器断开，模式切换控制模块自动退出主机模式，进入到终端模式；(1) Automatic exit: The display control module of the intelligent terminal detects that the intelligent terminal is disconnected from the external display, and the mode switching control module automatically exits the host mode and enters the terminal mode;
（2）手动退出：用户调出系统菜单，手动退出主机模式；用户点击智能终端屏幕上的切换功能快捷键进行手动退出主机模式；用户通过智能终端机身上的切换按钮手动退出主机模式。(2) Manual exit: The user calls up the system menu and manually exits the host mode; the user clicks the switch function shortcut key on the screen of the smart terminal to manually exit the host mode; the user manually exits the host mode through the switch button on the smart terminal body.
如图5所示，多功能触摸板控制模块检测触摸信号并响应相应动作的流程如下：As shown in Figure 5, the process of the multi-function touch panel control module detecting touch signals and responding to corresponding actions is as follows:
S31：检测触控位置是否在虚拟按键范围内，若是，则进入S32，若否，则进入S33；S31: Detect whether the touch position is within the range of the virtual key, if yes, go to S32, if not, go to S33;
S32：检测是否单指按下，若是，则进一步进行虚拟鼠标按键范围的判断，若否，则进入S33；S32: Detect whether a single finger is pressed, if yes, then further judge the range of the virtual mouse button, if not, then enter S33;
S33：检测触控位置是否在多点触控范围内，若是，则进入S34，若否，则响应结束；S33: Detect whether the touch position is within the range of multi-touch, if so, go to S34, if not, end the response;
S34：检测是否单指滑动，若是，根据移动的方向、距离和速度，移动当前鼠标指针位置并通过外部显示器显示，若否，进入S35；S34: Detect whether a single finger slides, if yes, move the current mouse pointer position and display it on an external display according to the moving direction, distance and speed, if not, go to S35;
S35：检测是否系统预定义手势，若是，则执行系统预定义动作，若否，则结束响应。S35: Detect whether the system has predefined gestures, if yes, execute the system predefined actions, if not, end the response.
上述步骤S35具体为：The above step S35 is specifically:
S351：检测是否双指滑动，若是，则执行当前应用的屏幕卷滚动作，若否，则进入S352；S351: Detect whether two-finger slide, if yes, execute the screen scrolling action of the current application, if not, enter S352;
S352：检测是否双指缩放，若是，则执行当前应用的内容缩放动作，若否，则进入S353；S352: Detect whether pinch-to-zoom, if yes, execute the content zoom action of the current application, if not, enter S353;
S353：检测是否三指滑动，若是且智能终端具有虚拟键盘功能，则执行多点触控区域与虚拟键盘区域的切换，若否，则结束响应。S353: Detect whether the three-finger slide, if yes and the smart terminal has a virtual keyboard function, perform switching between the multi-touch area and the virtual keyboard area, if not, end the response.
该系统还可根据需求设定四指滑动手势或五指滑动手势，同时执行对应的动作。The system can also set four-finger sliding gestures or five-finger sliding gestures according to requirements, and perform corresponding actions at the same time.
所述虚拟鼠标按键范围包括第一虚拟鼠标按键范围、第二虚拟鼠标按键范围和第三虚拟鼠标按键范围，当检测单指按下的范围在第一虚拟鼠标按键范围内时，响应鼠标左键点击的动作，当检测单指按下的范围在第二虚拟鼠标按键范围内时，响应鼠标右键点击的动作，当检测单指按下的范围在第三虚拟鼠标按键范围内时，响应鼠标中键点击的动作。根据按键的不同设定不同的声音效果，以便给用户更好的操作反馈。The range of virtual mouse buttons includes a first range of virtual mouse buttons, a second range of virtual mouse buttons and a third range of virtual mouse buttons. Click action, when detecting that the range of single-finger pressing is within the range of the second virtual mouse button, respond to the action of clicking the right button of the mouse; when detecting that the range of single-finger pressing is within the range of the third virtual mouse button, responding to the middle of the mouse The action of a key click. Set different sound effects according to different buttons, so as to give users better operation feedback.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310276853.3ACN103345312B (en) | 2013-07-03 | 2013-07-03 | Using intelligent terminal simultaneously as the system and method for main frame, mouse and touch pad |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310276853.3ACN103345312B (en) | 2013-07-03 | 2013-07-03 | Using intelligent terminal simultaneously as the system and method for main frame, mouse and touch pad |
| Publication Number | Publication Date |
|---|---|
| CN103345312A CN103345312A (en) | 2013-10-09 |
| CN103345312Btrue CN103345312B (en) | 2016-01-13 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310276853.3AExpired - Fee RelatedCN103345312B (en) | 2013-07-03 | 2013-07-03 | Using intelligent terminal simultaneously as the system and method for main frame, mouse and touch pad |
| Country | Link |
|---|---|
| CN (1) | CN103345312B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9753569B2 (en) | 2014-03-25 | 2017-09-05 | Intel Corporation | Switchable input modes for external display operation |
| CN104216670B (en)* | 2014-08-07 | 2018-02-23 | 惠州Tcl移动通信有限公司 | The system of external display and its external display methods |
| CN106292988A (en)* | 2015-05-13 | 2017-01-04 | 中兴通讯股份有限公司 | A kind of method realizing application controls and terminal |
| CN106293413A (en)* | 2015-05-14 | 2017-01-04 | 中兴通讯股份有限公司 | The control method of display device and device |
| CN104915029A (en)* | 2015-05-18 | 2015-09-16 | 苏州市职业大学 | Multifunctional mouse internally provided with operating system |
| CN105446429A (en)* | 2015-12-28 | 2016-03-30 | 天津云锦瑞物联网技术研发有限责任公司 | Computer system with multiple display terminals capable of sharing host |
| CN105787338B (en)* | 2016-02-14 | 2019-03-29 | 联想(北京)有限公司 | A kind of information processing method and the first electronic equipment |
| CN108227938A (en)* | 2016-12-12 | 2018-06-29 | 苏州乐聚堂电子科技有限公司 | Intelligent expansion device and with its intelligent expansion terminal |
| CN107357438B (en)* | 2017-07-22 | 2020-02-04 | 任文 | Method for realizing virtual mouse function of keyboard touch screen |
| US10782872B2 (en) | 2018-07-27 | 2020-09-22 | Asustek Computer Inc. | Electronic device with touch processing unit |
| CN110874180A (en)* | 2018-09-03 | 2020-03-10 | 中兴通讯股份有限公司 | Method and device for realizing functions of mouse and keyboard by using intelligent terminal |
| CN111007977A (en)* | 2018-10-04 | 2020-04-14 | 邱波 | Intelligent virtual interaction method and device |
| CN111124156B (en)* | 2018-11-01 | 2023-07-18 | 中兴通讯股份有限公司 | Interaction control method of mobile terminal and mobile terminal |
| CN110633044B (en)* | 2019-08-27 | 2021-03-19 | 联想(北京)有限公司 | Control method, control device, electronic equipment and storage medium |
| CN111580669A (en)* | 2020-05-12 | 2020-08-25 | 南京睿悦信息技术有限公司 | Interaction method and device for virtual reality and augmented reality mobile end plane application |
| CN113434077B (en)* | 2021-06-30 | 2023-09-01 | 海联智芯(深圳)科技有限公司 | Touch device and computer equipment |
| CN115494970A (en)* | 2022-10-14 | 2022-12-20 | 西安雷风电子科技有限公司 | System and method for cloud terminal to realize input through touch panel |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201491081U (en)* | 2009-07-10 | 2010-05-26 | 马文师 | Net surfing cell phone with host machine function |
| CN102945088A (en)* | 2012-10-19 | 2013-02-27 | 北京小米科技有限责任公司 | Method, device and mobile equipment for realizing terminal-simulated mouse to operate equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7534988B2 (en)* | 2005-11-08 | 2009-05-19 | Microsoft Corporation | Method and system for optical tracking of a pointing object |
| US20100216508A1 (en)* | 2009-02-23 | 2010-08-26 | Augusta Technology, Inc. | Systems and Methods for Driving an External Display Device Using a Mobile Phone Device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201491081U (en)* | 2009-07-10 | 2010-05-26 | 马文师 | Net surfing cell phone with host machine function |
| CN102945088A (en)* | 2012-10-19 | 2013-02-27 | 北京小米科技有限责任公司 | Method, device and mobile equipment for realizing terminal-simulated mouse to operate equipment |
| Publication number | Publication date |
|---|---|
| CN103345312A (en) | 2013-10-09 |
| Publication | Publication Date | Title |
|---|---|---|
| CN103345312B (en) | Using intelligent terminal simultaneously as the system and method for main frame, mouse and touch pad | |
| CN106843739B (en) | A kind of display control method and mobile terminal of mobile terminal | |
| TWI511027B (en) | Method, device and apparatus of splitting screen | |
| CN102722334B (en) | The control method of touch screen and device | |
| JP5759660B2 (en) | Portable information terminal having touch screen and input method | |
| TWI507965B (en) | Method, apparatus and computer program product for window management of multiple screens | |
| CN105630393B (en) | A kind of control method and control device of touch screen operating mode | |
| CN102739887B (en) | Wireless control method based on touch-screen mobile phone | |
| US9122337B2 (en) | Information processing terminal, and method for controlling same | |
| WO2013155735A1 (en) | Off-screen touch control interaction system having projection point coordinate indication of detected proximity | |
| CN203301578U (en) | Cellphone with auxiliary touch controller | |
| TWI643091B (en) | Mechanism for providing visual feedback on computing system command gestures | |
| CN106527904A (en) | Method for adjusting display interface and mobile terminal | |
| CN102622170B (en) | Electronic device and control method thereof | |
| CN101430620A (en) | Notebook computer with multi-point touch screen | |
| CN102023735A (en) | Touch input equipment, electronic equipment and mobile phone | |
| CN110851061B (en) | Method for controlling terminal by ring type mouse | |
| CN101482799A (en) | Method for controlling electronic equipment through touching type screen and electronic equipment thereof | |
| CN104333633A (en) | Mobile terminal supporting back control and back control method of mobile terminal | |
| CN102685314B (en) | Mobile device and mobile device control method | |
| CN102467283A (en) | Touch device with multi-touch function and touch operation method | |
| CN111078032A (en) | Ring type mouse capable of prejudging merging trend of two fingers and prejudging method | |
| CN115480690A (en) | Split-screen display method, split-screen display device and electronic equipment | |
| CN103279297B (en) | A kind of screen unlock method of mobile terminal | |
| KR20130015511A (en) | Mouse pad type input apparatus and method |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20160113 |