TWI872180B - System and method related to data fusing - Google Patents

Abstract

A system and a method related to data fusing are provided. The first sensing data and second sensing data are obtained based on different sensing manners, to further obtain their data combination. The third sensing data is obtained based on images. Whether the human body portion of the user exists in the third sensing data is determined during the first duration. The motion information of the user is determined by a first determining factor in response to the human body portion existing in the third sensing data, and the first determining factor includes the data combination and the third sensing data of the first duration. The motion information is determined by a second determining factor in response to the human body portion not existing in the third sensing data, and the second determining factor includes the data combination and the third sensing data of the second duration. Accordingly, the motion of the user can be tracked with improved accuracy.

Description

Translated fromChinese

相關於資料融合的系統和方法Systems and methods related to data fusion

本發明是有關於一種追蹤使用者的動作的方法，且特別是有關於一種相關於資料融合的系統和方法。The present invention relates to a method for tracking user actions, and more particularly to a system and method for data fusion.

為了在電子設備（諸如遊戲機、電腦、智慧手機、智慧電器等）上提供直觀的操作，可偵測使用者的動作，以根據使用者的動作直接操作電子設備。In order to provide intuitive operation on electronic devices (such as game consoles, computers, smart phones, smart appliances, etc.), the user's actions can be detected to directly operate the electronic device according to the user's actions.

在傳統技術中，一些電子設備可允許使用者的人體部位（諸如手、腿、頭等）來控制這些電子設備的操作，且可追蹤這些人體部位的動作。然而，這些電子設備僅提供一種同時偵測多個人體部位的動作的方式。舉例來說，虛擬實境（virtual reality；VR）產品可提供手持型控制器，且每個手持型控制器包含慣性測量單元（inertial measurement unit；IMU）以追蹤使用者的手的動作。有時，僅單一動作追蹤方式可受其硬體或追蹤機制的限制，且導致追蹤結果異常或不準確。In traditional technology, some electronic devices allow the user's body parts (such as hands, legs, head, etc.) to control the operation of these electronic devices, and can track the movements of these body parts. However, these electronic devices only provide a way to detect the movements of multiple body parts at the same time. For example, virtual reality (VR) products may provide handheld controllers, and each handheld controller includes an inertial measurement unit (IMU) to track the movements of the user's hands. Sometimes, a single motion tracking method may be limited by its hardware or tracking mechanism, and cause abnormal or inaccurate tracking results.

有時候，僅使用單一感測器的追蹤結果可能不準確。有鑑於此，本發明提供一種相關於資料融合的系統和方法。Sometimes, the tracking results using only a single sensor may be inaccurate. In view of this, the present invention provides a system and method related to data fusion.

本發明實施例的相關於資料融合的方法適用於系統，所述系統包含可穿戴在使用者的人體部位上的第一動作感測設備、第二動作感測設備以及第三動作感測設備。相關於資料融合的方法包含（但不限於）以下步驟。基於設置在第一動作感測設備、第二動作感測設備以及第三動作感測設備上的動作感測器取得第一感測資料。基於在第一動作感測設備、第二動作感測設備以及第三動作感測設備之間發射的無線信號取得第二感測資料。根據包含第一感測資料和第二感測資料的判斷因素判斷使用者的動作資訊。The method for data fusion of the embodiment of the present invention is applicable to a system, wherein the system includes a first motion sensing device, a second motion sensing device, and a third motion sensing device that can be worn on a body part of a user. The method for data fusion includes (but is not limited to) the following steps. First sensing data is obtained based on motion sensors provided on the first motion sensing device, the second motion sensing device, and the third motion sensing device. Second sensing data is obtained based on wireless signals transmitted between the first motion sensing device, the second motion sensing device, and the third motion sensing device. The motion information of the user is judged based on a judgment factor including the first sensing data and the second sensing data.

本發明實施例的相關於資料融合的系統包含（但不限於）三個動作感測設備和處理器。動作感測設備可穿戴在使用者的人體部位上。各動作感測設備包含無線收發器和動作感測器。無線收發器用於發射或接收無線信號。動作感測器用於感測使用者的一個人體部位的動作。處理器經配置用以基於動作感測設備的動作感測器取得第一感測資料，且基於在所述三個動作感測設備之間發射的所述無線信號據取得第二感測資料，且由包含所述第一感測資料和所述第二感測資料的判斷因素判斷所述使用者的動作資訊。The system related to data fusion of the embodiment of the present invention includes (but is not limited to) three motion sensing devices and a processor. The motion sensing device can be worn on a body part of a user. Each motion sensing device includes a wireless transceiver and a motion sensor. The wireless transceiver is used to transmit or receive wireless signals. The motion sensor is used to sense the movement of a body part of the user. The processor is configured to obtain first sensing data based on the motion sensor of the motion sensing device, and to obtain second sensing data based on the wireless signal data transmitted between the three motion sensing devices, and to judge the motion information of the user based on the judgment factors including the first sensing data and the second sensing data.

基於上述，根據本發明實施例的相關於資料融合的系統及方法，可基於數種感測方法的感測資料來追蹤使用者的單一操作部位的動作。藉此，可提供準確及可靠的追蹤機制。Based on the above, the system and method related to data fusion according to the embodiment of the present invention can track the movement of a single operating part of the user based on the sensing data of multiple sensing methods, thereby providing an accurate and reliable tracking mechanism.

為讓本發明的上述特徵和優點能更明顯易懂，下文特舉實施例，並配合所附圖式作詳細說明如下。In order to make the above features and advantages of the present invention more clearly understood, embodiments are specifically cited below and described in detail with reference to the accompanying drawings.

現將詳細參考本發明的優選實施例，其範例在附圖中說明。只要可能，相同附圖符號在附圖和描述中用以代表相同或相似部件。Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same drawing symbols are used in the drawings and description to represent the same or similar parts.

圖1是示出根據本發明的示例性實施例中的一個的動作追蹤系統10的方塊圖。參看圖1，動作追蹤系統10包含（但不限於）三個或更多個動作感測設備100以及計算設備200。動作追蹤系統10可適用於VR、AR、MR、XR或其它現實相關的技術。Fig. 1 is a block diagram showing amotion tracking system 10 according to one of the exemplary embodiments of the present invention. Referring to Fig. 1, themotion tracking system 10 includes (but is not limited to) three or moremotion sensing devices 100 and acomputing device 200. Themotion tracking system 10 can be applied to VR, AR, MR, XR or other reality-related technologies.

每個動作感測設備100包含（但不限於）無線收發器110和動作感測器130。動作感測設備100可以是手持型控制器或可穿戴設備，諸如可穿戴控制器、智慧手錶、腳踝感測器、腰帶、頭戴式顯示器（head-mounted display；HMD）或類似物。在一個實施例中，每個動作感測設備100可穿戴在使用者的一個人體部位上。人體部位可以是手、頭、腳踝、腿、腰或其它部位。Eachmotion sensing device 100 includes (but is not limited to) awireless transceiver 110 and amotion sensor 130. Themotion sensing device 100 can be a handheld controller or a wearable device, such as a wearable controller, a smart watch, an ankle sensor, a belt, a head-mounted display (HMD), or the like. In one embodiment, eachmotion sensing device 100 can be worn on a body part of the user. The body part can be a hand, head, ankle, leg, waist, or other body part.

無線收發器110可以是與藍牙、Wi-Fi、IR、RFID或其它無線通訊技術相容的通訊收發器。在一個實施例中，無線收發器110用於與其它動作感測設備100的無線收發器110一起發射和/或接收無線信號，且將基於在動作感測設備100之間發射的無線信號產生序列的第一感測資料。將在隨後介紹用於產生序列的第一感測資料的詳細流程。Thewireless transceiver 110 may be a communication transceiver compatible with Bluetooth, Wi-Fi, IR, RFID or other wireless communication technologies. In one embodiment, thewireless transceiver 110 is used to transmit and/or receive wireless signals together with thewireless transceivers 110 of othermotion sensing devices 100, and to generate the first sensing data of the sequence based on the wireless signals transmitted between themotion sensing devices 100. The detailed process for generating the first sensing data of the sequence will be introduced later.

動作感測器130可以是加速計、陀螺儀、磁強計、慣性測量單元（IMU）或前述感測器的任何組合。在實施例中，動作感測器130用於在一段時間內感測穿戴動作感測設備100的使用者的對應人體部位的動作，以在時間段內的多個時間點從動作感測器130的感測結果（諸如加速度、旋轉、磁力等）產生序列的第一感測資料。例如，第一感測資料包含3-自由度（3-degree of freedom；3-DoF）資料，且3-DoF資料與三維（three-dimensional；3D）空間中的人體部位的朝向資訊相關，諸如偏航、滾轉以及俯仰的加速度。Themotion sensor 130 may be an accelerometer, a gyroscope, a magnetometer, an inertial measurement unit (IMU), or any combination of the aforementioned sensors. In an embodiment, themotion sensor 130 is used to sense the motion of the corresponding human body part of the user wearing themotion sensing device 100 over a period of time, so as to generate a sequence of first sensing data from the sensing results (such as acceleration, rotation, magnetism, etc.) of themotion sensor 130 at multiple time points in the time period. For example, the first sensing data includes 3-degree of freedom (3-DoF) data, and the 3-DoF data is related to the orientation information of the human body part in three-dimensional (3D) space, such as acceleration of yaw, roll, and pitch.

計算設備200包含（但不限於）記憶體240和處理器250。計算設備200可以是電腦、伺服器、智慧手機、平板電腦或動作感測設備100中的一個。Thecomputing device 200 includes (but is not limited to) amemory 240 and aprocessor 250. Thecomputing device 200 may be a computer, a server, a smart phone, a tablet computer, or one of themotion sensing devices 100.

記憶體240可以是任何類型的固定或可移動隨機存取記憶體（Random-Access Memory；RAM）、唯讀記憶體（read-only memory；ROM）、快閃記憶體或類似裝置或以上裝置的組合。記憶體240可用於儲存程式碼、裝置配置、緩衝資料或永久性資料（諸如感測資料、動作資訊、距離關係等），且將在隨後介紹這些資料。Thememory 240 may be any type of fixed or removable random-access memory (RAM), read-only memory (ROM), flash memory, or similar devices or a combination of the above devices. Thememory 240 may be used to store program code, device configuration, buffer data, or permanent data (such as sensor data, motion information, distance relationship, etc.), and these data will be introduced later.

處理器250連接到記憶體240，且處理器250經配置用以載入儲存在記憶體240中的程式碼，從而執行本發明的示例性實施例的程式。在一個實施例中，處理器150的功能透過使用諸如中央處理單元（central processing unit；CPU）、微處理器、微控制器、數位信號處理（digital signal processing；DSP）晶片、現場可程式設計閘陣列（field programmable gate array；FPGA）等可程式設計單元來實施。在一些實施例中，處理器250的功能也可透過獨立電子裝置或積體電路（integrated circuit；IC）來實施，且處理器150的操作也可由軟體來實施。Theprocessor 250 is connected to thememory 240, and theprocessor 250 is configured to load the program code stored in thememory 240, thereby executing the program of the exemplary embodiment of the present invention. In one embodiment, the functions of the processor 150 are implemented by using a programmable unit such as a central processing unit (CPU), a microprocessor, a microcontroller, a digital signal processing (DSP) chip, a field programmable gate array (FPGA), etc. In some embodiments, the functions of theprocessor 250 can also be implemented by an independent electronic device or an integrated circuit (IC), and the operation of the processor 150 can also be implemented by software.

應注意，處理器250可或可不與一個、部位或全部動作感測設備100設置在同一設備上。然而，分別裝配有動作感測器130和處理器250的設備可更包含具有諸如藍牙、Wi-Fi、IR或物理傳輸線路的相容通訊技術的通訊收發器，以彼此發射/接收資料。It should be noted that theprocessor 250 may or may not be disposed on the same device as one, part, or all of themotion sensing devices 100. However, the devices equipped with themotion sensor 130 and theprocessor 250 may further include communication transceivers with compatible communication technologies such as Bluetooth, Wi-Fi, IR, or physical transmission lines to transmit/receive data to each other.

在一個實施例中，動作追蹤系統10可更包含頭戴式顯示器（HMD）300。HMD 300可穿戴在使用者的頭部上。HMD 300包含（但不限於）無線收發器310和影像感測器360。In one embodiment, themotion tracking system 10 may further include a head mounted display (HMD) 300. The HMD 300 may be worn on the user's head. The HMD 300 includes (but is not limited to) awireless transceiver 310 and animage sensor 360.

無線收發器310的描述可指無線收發器110的描述，且將省略。這意味著HMD 300可透過無線收發器310與動作感測設備100通訊。The description of thewireless transceiver 310 may refer to the description of thewireless transceiver 110 and will be omitted. This means that the HMD 300 can communicate with themotion sensing device 100 via thewireless transceiver 310.

影像感測器360可以是相機，如單色相機或彩色相機、深度相機、錄影機或能夠擷取影像的其它感測器。Theimage sensor 360 may be a camera, such as a monochrome camera or a color camera, a depth camera, a video camera, or other sensor capable of capturing images.

圖2是示出根據本發明的示例性實施例中的一個的動作追蹤系統20的示意圖。參看圖2，動作追蹤系統20包含HMD 300和四個動作感測設備100（所述四個設備是穿戴在人體部位B1和人體部位B2（即，兩個腳踝）上的兩個腳踝感測器和穿戴在人體部位B3和人體部位B4（即，兩隻手）上的兩個手持型控制器）。在一些實施例中，HMD 300可更包含另一動作感測器130（未示出），以取得人體部位B5（即，頭部）的朝向資訊。處理器250嵌入在HMD 300中。FIG2 is a schematic diagram showing amotion tracking system 20 according to one of the exemplary embodiments of the present invention. Referring to FIG2 , themotion tracking system 20 includes an HMD 300 and four motion sensing devices 100 (the four devices are two ankle sensors worn on body part B1 and body part B2 (i.e., two ankles) and two handheld controllers worn on body part B3 and body part B4 (i.e., two hands)). In some embodiments, the HMD 300 may further include another motion sensor 130 (not shown) to obtain orientation information of body part B5 (i.e., head). Theprocessor 250 is embedded in the HMD 300.

應注意，動作追蹤系統20僅是舉例以示出動作感測設備100、HMD 300以及處理器250的設置方式。然而，仍存在動作追蹤系統20的許多其它實施方式，且本發明不限於此。It should be noted that themotion tracking system 20 is only an example to illustrate the configuration of themotion sensing device 100, the HMD 300, and theprocessor 250. However, there are many other implementations of themotion tracking system 20, and the present invention is not limited thereto.

為了更好地理解在本發明的一個或多個實施例中提供的操作流程，下文將例示若干實施例以詳細解釋動作追蹤系統10或動作追蹤系統20的操作流程。動作追蹤系統10或動作追蹤系統20中的裝置和模組在以下實施例應用以解釋本文所提供的控制方法。可根據實際實施情況調整控制方法的每個步驟，且不應限於本文描述的內容。In order to better understand the operation process provided in one or more embodiments of the present invention, several embodiments are illustrated below to explain in detail the operation process of themotion tracking system 10 or themotion tracking system 20. The devices and modules in themotion tracking system 10 or themotion tracking system 20 are applied in the following embodiments to explain the control method provided herein. Each step of the control method can be adjusted according to the actual implementation situation and should not be limited to the content described herein.

圖3是示出根據本發明的示例性實施例中的一個的動作追蹤方法的流程圖。參看圖3，處理器250可基於設置在三個動作感測設備100上的動作感測器130取得第一感測資料（步驟S310）。確切地說，關於不同類型的動作感測器130，可取得2D/3D空間中對應人體部位的動作的加速度、旋轉、磁力、朝向和/或3-DoF/6-DoF，且動作感測器130的一個或多個感測結果將成為人體部位的序列第一感測資料。FIG3 is a flow chart showing a motion tracking method according to one of the exemplary embodiments of the present invention. Referring to FIG3 , theprocessor 250 may obtain first sensing data based on themotion sensors 130 provided on the three motion sensing devices 100 (step S310). Specifically, with respect to different types ofmotion sensors 130, acceleration, rotation, magnetism, orientation and/or 3-DoF/6-DoF of the motion corresponding to the human body part in 2D/3D space may be obtained, and one or more sensing results of themotion sensor 130 will become the sequence first sensing data of the human body part.

另一方面，處理器250可基於在三個動作感測設備100之間發射的無線信號取得第二感測資料（步驟S330）。在一個實施例中，處理器250可在多個時間點從三個或更多個動作感測設備100取得無線信號的信號強度，且每個信號強度將與其對應的發射器和接收器一起記錄在記憶體240中。信號強度可以是接收的信號強度指示（received signal strength indication；RSSI）、接收通道功率參數（received channel power indicator；RCPI）、參考信號接收功率（reference signal received power；RSRP）或類似者。在一個實施例中，動作感測設備100可監測所有可偵測的無線信號的信號強度，且每個無線信號包含發射器和/或接收器的特定識別碼。動作感測設備100可進一步將具有對應識別碼的信號強度回饋給計算設備200。在另一實施例中，計算設備200可監測所有可偵測的無線信號的信號強度，且處理器250將信號強度與發射器的對應識別碼記錄在記憶體240中。將記錄信號強度一段時間以產生序列的第二感測資料。這意味著第二感測資料包含按時間排列的序列信號強度。On the other hand, theprocessor 250 may obtain second sensing data based on the wireless signals transmitted between the three motion sensing devices 100 (step S330). In one embodiment, theprocessor 250 may obtain the signal strength of the wireless signals from the three or moremotion sensing devices 100 at multiple time points, and each signal strength will be recorded in thememory 240 together with its corresponding transmitter and receiver. The signal strength may be a received signal strength indication (RSSI), a received channel power indicator (RCPI), a reference signal received power (RSRP), or the like. In one embodiment, themotion sensing device 100 may monitor the signal strength of all detectable wireless signals, and each wireless signal includes a specific identification code of a transmitter and/or a receiver. Themotion sensing device 100 may further feed back the signal strength with the corresponding identification code to thecomputing device 200. In another embodiment, thecomputing device 200 may monitor the signal strength of all detectable wireless signals, and theprocessor 250 records the signal strength and the corresponding identification code of the transmitter in thememory 240. The signal strength is recorded for a period of time to generate a sequence of second sensing data. This means that the second sensing data includes a sequence of signal strengths arranged in time.

在一些實施例中，處理器250可進一步基於從影像感測器360所擷取的影像取得第三感測資料。第三感測資料可以是序列的影像和/或影像中的像素的感測結果（諸如亮度、色彩、深度等）。In some embodiments, theprocessor 250 may further obtain third sensing data based on the image captured by theimage sensor 360. The third sensing data may be a sensing result (such as brightness, color, depth, etc.) of a sequence of images and/or pixels in the image.

接著，處理器250可由包含第一感測資料和第二感測資料的判斷因素判斷使用者的動作資訊（步驟S350）。在一個實施例中，動作資訊可包含位置資訊和朝向資訊。首先關於位置資訊，在一個實施例中，處理器250可根據第一感測資料判斷使用者的位置資訊。在此實施例中，判斷因素包含第一感測資料。對應人體部位的位移可透過對人體部位在三條軸線上的偵測到的加速度（即，第二感測資料）二重積分來估測，以基於位移進一步判斷位置資訊。舉例來說，位置資訊可以是兩條或三條軸線上的座標，相對於參考的位置等。Next, theprocessor 250 may determine the user's motion information based on the determination factors including the first sensing data and the second sensing data (step S350). In one embodiment, the motion information may include position information and orientation information. First, regarding the position information, in one embodiment, theprocessor 250 may determine the user's position information based on the first sensing data. In this embodiment, the determination factors include the first sensing data. The displacement of the corresponding human body part may be estimated by doubly integrating the acceleration (i.e., the second sensing data) detected on the three axes of the human body part to further determine the position information based on the displacement. For example, the position information may be coordinates on two or three axes, a position relative to a reference, and the like.

在另一實施例中，處理器250可根據基於三個動作感測設備100之間的無線信號的第二感測資料取得位置資訊。在此實施例中，判斷因素包含第二感測資料，應注意，無線信號的信號強度與兩個動作感測設備100之間的相對距離相關。另外，基於三邊測量，三個點之間的三個距離可用於判斷三個點的相對位置資訊。假定三個動作感測設備100作為前述三個點，那麼處理器250可將每兩個動作感測設備100之間的相對距離判斷為動作感測設備100之間的距離關係。接著，處理器250可基於距離關係和三邊測量產生追蹤設備的位置資訊。In another embodiment, theprocessor 250 may obtain the location information based on the second sensing data based on the wireless signal between the threemotion sensing devices 100. In this embodiment, the determination factor includes the second sensing data, and it should be noted that the signal strength of the wireless signal is related to the relative distance between the twomotion sensing devices 100. In addition, based on trilateration, the three distances between the three points can be used to determine the relative location information of the three points. Assuming that the threemotion sensing devices 100 are the aforementioned three points, theprocessor 250 can determine the relative distance between every twomotion sensing devices 100 as the distance relationship between themotion sensing devices 100. Theprocessor 250 may then generate location information of the tracking device based on the distance relationship and trilateration.

以動作追蹤系統20為例，處理器250可取得從用於人體部位B3的動作感測設備100到用於人體部位B5的HMD 300（在此實施例中是動作感測設備100中的一個）的無線信號、從用於人體部位B4的動作感測設備100到用於人體部位B5的HMD 300的無線信號以及從用於人體部位B3的動作感測設備100到用於人體部位B4的動作感測設備100的無線信號的信號強度。處理器250可根據信號強度判斷其距離關係，且接著基於距離關係產生人體部位B3的位置資訊。位置資訊可以是座標或相對位置。Taking themotion tracking system 20 as an example, theprocessor 250 may obtain the signal strength of the wireless signal from themotion sensing device 100 for body part B3 to theHMD 300 for body part B5 (one of themotion sensing devices 100 in this embodiment), the wireless signal from themotion sensing device 100 for body part B4 to theHMD 300 for body part B5, and the wireless signal from themotion sensing device 100 for body part B3 to themotion sensing device 100 for body part B4. Theprocessor 250 may determine the distance relationship according to the signal strength, and then generate the position information of the body part B3 based on the distance relationship. The position information may be a coordinate or a relative position.

應注意，實施例不限制選定三個動作感測設備100。舉例來說，從用於人體部位B2的動作感測設備100到用於人體部位B3的動作感測設備100的無線信號、從用於人體部位B3的動作感測設備100到用於人體部位B1的動作感測設備100的無線信號以及從用於人體部位B2的動作感測設備100到用於人體部位B1的動作感測設備100的無線信號的信號強度可用於估測人體部位B1的位置資訊。動作感測設備100的組合可根據需求而變化。It should be noted that the embodiment is not limited to selecting threemotion sensing devices 100. For example, the signal strengths of the wireless signal from themotion sensing device 100 for body part B2 to themotion sensing device 100 for body part B3, the wireless signal from themotion sensing device 100 for body part B3 to themotion sensing device 100 for body part B1, and the wireless signal from themotion sensing device 100 for body part B2 to themotion sensing device 100 for body part B1 can be used to estimate the position information of body part B1. The combination ofmotion sensing devices 100 can be changed as needed.

在另一實施例中，處理器250可根據第三感測資料判斷使用者的位置資訊。在此實施例中，判斷因素包含第三感測資料。影像中的人體部位的位置和位移可用於判斷真實環境中的位置資訊。以圖2為例，對應於影像中的人體部位B4的感測強度和像素位置可用於估測人體部位B4的深度資訊（即，相對於HMD 300的距離）和估測人體部位B4在平行於影像感測器360的平面上的2D位置。In another embodiment, theprocessor 250 may determine the user's position information based on the third sensing data. In this embodiment, the determination factor includes the third sensing data. The position and displacement of the human body part in the image can be used to determine the position information in the real environment. Taking FIG. 2 as an example, the sensing intensity and pixel position corresponding to the human body part B4 in the image can be used to estimate the depth information of the human body part B4 (i.e., the distance relative to the HMD 300) and estimate the 2D position of the human body part B4 on a plane parallel to theimage sensor 360.

應注意，基於僅一種感測方式（例如，是基於無線收發器110、動作感測器130以及影像感測器360中的一個）的位置資訊的準確度可以是不同的。因此，兩種或更多種感測方式可用於判斷對應人體部位的位置資訊。It should be noted that the accuracy of the position information based on only one sensing method (e.g., based on one of thewireless transceiver 110, themotion sensor 130, and the image sensor 360) may be different. Therefore, two or more sensing methods may be used to determine the position information corresponding to the human body part.

在一個實施例中，處理器250可根據第一感測資料取得第一位置信息，根據第二感測資料取得第二位置資訊，以及根據第一位置信息和第二位置資訊取得調整後的位置資訊。在此實施例中，判斷因素包含第一感測資料和第二感測資料。處理器250可根據第一位置信息和第二位置資訊的組合判斷位置資訊。在一些實施例中，組合是加權組合。根據加權的第一位置信息和加權的第二位置資訊判斷調整後的位置資訊。In one embodiment, theprocessor 250 may obtain the first location information based on the first sensing data, obtain the second location information based on the second sensing data, and obtain the adjusted location information based on the first location information and the second location information. In this embodiment, the determination factor includes the first sensing data and the second sensing data. Theprocessor 250 may determine the location information based on a combination of the first location information and the second location information. In some embodiments, the combination is a weighted combination. The adjusted location information is determined based on the weighted first location information and the weighted second location information.

在一個實施例中，第一位置信息和第二位置資訊的加權組合的權重可以是固定的。在另一實施例中，第一位置信息和第二位置資訊的加權組合的權重可以是變化的。第一位置信息的權重可以是從0到100%的值，且第二位置資訊的權重可以是從0到100%的值。然而，第一位置信息和第二位置資訊的權重都不能同時是0。In one embodiment, the weight of the weighted combination of the first location information and the second location information may be fixed. In another embodiment, the weight of the weighted combination of the first location information and the second location information may be variable. The weight of the first location information may be a value from 0 to 100%, and the weight of the second location information may be a value from 0 to 100%. However, the weights of the first location information and the second location information cannot both be 0 at the same time.

應注意，在一些實施例中，基於由影像感測器360的影像產生的第三感測資料而判斷的位置資訊可比基於無線收發器110和/或動作感測器130而判斷的位置資訊更準確。因此，在一個實施例中，判斷因素可包含第二感測資料和第三感測資料。處理器250可根據基於第一感測資料、第二感測資料以及第三感測資料取得的位置資訊的組合判斷位置資訊。It should be noted that in some embodiments, the position information determined based on the third sensing data generated by the image of theimage sensor 360 may be more accurate than the position information determined based on thewireless transceiver 110 and/or themotion sensor 130. Therefore, in one embodiment, the determination factor may include the second sensing data and the third sensing data. Theprocessor 250 may determine the position information based on the combination of the position information obtained based on the first sensing data, the second sensing data, and the third sensing data.

在一個實施例中，處理器250可在第一期間內根據第二感測資料取得位置資訊的第一部分，在第二期間內根據第三感測資料取得位置資訊的第二部分，以及將位置資訊的第一部分和第二部分組合為組合位置資訊。偵測人體部位的第三感測資料可用於在第一期間和第二期間內基於第二感測資料來校正位置資訊。處理器250可在不同期間內基於位置資訊的第一部分和第二部分來判斷組合位置資訊。舉例來說，在第一期間基於第二感測資料判斷位置（1，1），在第二期間基於第三感測資料判斷另一位置（2，1），且組合位置資訊可以是從位置（1，1）到位置（2，1）的位移。In one embodiment, theprocessor 250 may obtain a first portion of the position information based on the second sensing data during a first period, obtain a second portion of the position information based on the third sensing data during a second period, and combine the first portion and the second portion of the position information into combined position information. The third sensing data for detecting a body part may be used to correct the position information based on the second sensing data during the first period and the second period. Theprocessor 250 may determine the combined position information based on the first portion and the second portion of the position information during different periods. For example, a position (1, 1) is determined based on the second sensing data during the first period, and another position (2, 1) is determined based on the third sensing data during the second period, and the combined position information may be a displacement from the position (1, 1) to the position (2, 1).

在一些實施例中，處理器250可根據第二位置資訊和第三位置資訊的加權組合判斷位置資訊。基於實際情況，第二位置資訊和第三位置資訊的權重可以是變化的或固定的。舉例來說，第三位置資訊的權重可大於第二位置資訊的權重。在另一實施例中，如果人體部位存在於第三感測資料中，那麼位置資訊是加權組合，且如果人體部位不存在於第三感測資料中，那麼位置是第二位置資訊。In some embodiments, theprocessor 250 may determine the location information based on a weighted combination of the second location information and the third location information. Based on actual circumstances, the weights of the second location information and the third location information may be variable or fixed. For example, the weight of the third location information may be greater than the weight of the second location information. In another embodiment, if the human body part exists in the third sensing data, then the location information is a weighted combination, and if the human body part does not exist in the third sensing data, then the location is the second location information.

在一個實施例中，影像感測器360可設計有特定視場（specific field of view；FOV）。如果一個人體部位定位於影像感測器360的視場的外部，那麼處理器250可能不能夠僅使用第三感測資料來判斷此人體部位的動作資訊，且應考慮第一感測資料或第二感測資料。In one embodiment, theimage sensor 360 may be designed with a specific field of view (FOV). If a human body part is located outside the FOV of theimage sensor 360, theprocessor 250 may not be able to determine the motion information of the human body part using only the third sensing data, and should consider the first sensing data or the second sensing data.

在一個實施例中，處理器250可判斷使用者的一個人體部位是否存在於序列的第三感測資料中，且根據人體部位的存在的判斷的結果來判斷是否使用三個動作感測設備100之間的距離關係，以基於三邊測量判斷位置資訊。處理器250可使用機器學習技術（諸如深度學習、人工神經網路（artificial neural network；ANN）或支持向量機（support vector machine；SVM）等），以識別第三感測資料中的目標人體部位。In one embodiment, theprocessor 250 may determine whether a human body part of the user exists in the third sensing data of the sequence, and determine whether to use the distance relationship between the threemotion sensing devices 100 to determine the position information based on trilateration according to the result of the determination of the existence of the human body part. Theprocessor 250 may use machine learning technology (such as deep learning, artificial neural network (ANN) or support vector machine (SVM) etc.) to identify the target human body part in the third sensing data.

圖4是示出根據本發明的示例性實施例中的一個的動作追蹤方法的示意圖。參看圖4，假定人體部位B4的動作感測設備100是目標設備。在此圖中，人體部位B4存在於HMD 300的視場FOV中（即，人體部位B4存在於第三感測資料中）。Fig. 4 is a schematic diagram showing a motion tracking method according to one of the exemplary embodiments of the present invention. Referring to Fig. 4, it is assumed that themotion sensing device 100 of the human body part B4 is the target device. In this figure, the human body part B4 exists in the field of view FOV of the HMD 300 (that is, the human body part B4 exists in the third sensing data).

圖5是示出根據本發明的示例性實施例中的一個的動作追蹤方法的示意圖。參看圖5，假定人體部位B3的動作感測設備100是目標設備。在此圖中，人體部位B5不存在於HMD 300的視場FOV中（即，人體部位B3不存在於第三感測資料中）。Fig. 5 is a schematic diagram showing a motion tracking method according to one of the exemplary embodiments of the present invention. Referring to Fig. 5, it is assumed that themotion sensing device 100 of the human body part B3 is the target device. In this figure, the human body part B5 does not exist in the field of view FOV of the HMD 300 (that is, the human body part B3 does not exist in the third sensing data).

應注意，圖4和圖5中示出的視場的大小和形狀僅是範例，且可基於實際需要修改。It should be noted that the size and shape of the field of view shown in Figures 4 and 5 are examples only and may be modified based on actual needs.

因此，影像感測器360的視場用於判斷人體部位是否存在於第三感測資料中。在一個實施例中，假定人體部位在第一期間位於視場外（即，不存在於第三感測資料中），且人體部位在第二期間定位於影像感測器360的視場的內部（即，存在於第三感測資料中）。在一些實施例中，假定人體部位在第一期間和第二期間位於影像感測器360的視場內。Therefore, the field of view of theimage sensor 360 is used to determine whether the human body part exists in the third sensing data. In one embodiment, it is assumed that the human body part is outside the field of view during the first period (i.e., not present in the third sensing data), and the human body part is located inside the field of view of theimage sensor 360 during the second period (i.e., present in the third sensing data). In some embodiments, it is assumed that the human body part is located within the field of view of theimage sensor 360 during the first period and the second period.

在另一實施例中，處理器250可根據第一感測資料取得第一位置信息，根據第二感測資料取得第二位置資訊，根據第三感測資料取得第三位置資訊，且根據第一位置信息、第二位置資訊以及第三位置資訊取得調整後的位置資訊。在此實施例中，判斷因素包含第一感測資料、第二感測資料以及第三感測資料。處理器250可根據第一動作資訊、第二動作資訊以及第三位置資訊的組合判斷調整後的位置資訊。In another embodiment, theprocessor 250 may obtain the first location information according to the first sensing data, obtain the second location information according to the second sensing data, obtain the third location information according to the third sensing data, and obtain the adjusted location information according to the first location information, the second location information, and the third location information. In this embodiment, the determination factor includes the first sensing data, the second sensing data, and the third sensing data. Theprocessor 250 may determine the adjusted location information according to the combination of the first action information, the second action information, and the third location information.

在一個實施例中，組合是加權組合。處理器250可根據第三位置資訊判斷第一位置信息的第一權重和第二位置資訊的第二權重。在一個實施例中，第一權重和第二權重反復變化。在人體部位存在於第三感測資料中的期間內，第三位置資訊將被視為校正位置資訊，且將根據第三位置資訊調整具有第一權重和第二權重的第一位置信息和第二位置資訊的加權組合。應注意，處理器250可透過將第一權重與第一位置信息相乘取得第一參數，透過將第二權重與第二位置資訊相乘取得第二參數，且透過將第一參數與第二參數相加取得調整後的位置資訊，從而以取得加權組合。In one embodiment, the combination is a weighted combination. Theprocessor 250 may determine a first weight of the first position information and a second weight of the second position information based on the third position information. In one embodiment, the first weight and the second weight change repeatedly. During the period when the human body part is present in the third sensing data, the third position information will be regarded as the corrected position information, and the weighted combination of the first position information and the second position information having the first weight and the second weight will be adjusted according to the third position information. It should be noted that theprocessor 250 may obtain the weighted combination by multiplying the first weight by the first position information to obtain the first parameter, by multiplying the second weight by the second position information to obtain the second parameter, and by adding the first parameter to the second parameter to obtain the adjusted position information.

在一個實施例中，可基於第三位置資訊等於先前時間點的第一位置信息和第二位置資訊的加權組合的方程式而判斷後續時間點的第一權重和第二權重。舉例來說，在第三時間點，在3維坐標系統中，第一權重是0.5且第二權重是0.5，第一位置信息是（6，6，6）且第二位置資訊是（10，10，10），且調整後的位置資訊將是（8，8，8）。如果第三位置資訊是（7，7，7），那麼在第四時間點的第一權重和第二權重將分別判斷為0.75和0.25。接著，在第四時間點，如果在3維坐標系統中第一位置信息是（7，6，6）且第二位置資訊是（12，10，10），那麼調整後的位置資訊將是（8.25，7，7）。In one embodiment, the first weight and the second weight at a subsequent time point may be determined based on an equation that the third position information is equal to a weighted combination of the first position information and the second position information at the previous time point. For example, at the third time point, in the 3D coordinate system, the first weight is 0.5 and the second weight is 0.5, the first position information is (6, 6, 6) and the second position information is (10, 10, 10), and the adjusted position information will be (8, 8, 8). If the third position information is (7, 7, 7), then the first weight and the second weight at the fourth time point will be determined to be 0.75 and 0.25, respectively. Next, at the fourth time point, if the first position information is (7, 6, 6) and the second position information is (12, 10, 10) in the 3D coordinate system, then the adjusted position information will be (8.25, 7, 7).

在另一實施例中，可基於第三位置資訊等於當前時間點的第一位置信息和第二位置資訊的加權組合的方程式而判斷當前時間點的第一權重和第二權重。舉例來說，在第二時間點，在3維坐標系統中，第一位置信息是（6，6，6）且第二位置資訊是（10，10，10）。如果第三位置資訊是（7，7，7），那麼在第二時間點的第一權重和第二權重將分別判斷為0.75和0.25。接著，在第二時間點的調整後的位置資訊將判斷為（7，7，7）。In another embodiment, the first weight and the second weight at the current time point may be determined based on an equation that the third position information is equal to a weighted combination of the first position information and the second position information at the current time point. For example, at the second time point, in the 3D coordinate system, the first position information is (6, 6, 6) and the second position information is (10, 10, 10). If the third position information is (7, 7, 7), then the first weight and the second weight at the second time point will be determined to be 0.75 and 0.25, respectively. Then, the adjusted position information at the second time point will be determined to be (7, 7, 7).

在一些實施例中，如果使用者的人體部位不存在於第三感測資料中，那麼第一權重和第二權重是固定的。如果人體部位位於影像感測器360的視場外，那麼當使用者的人體部位仍存在於第三感測資料中時，第三權重和第二權重將與先前時間點的先前第一權重和第二權重相同。舉例來說，在第一時間點，人體部位位於影像感測器360的視場內，且第一權重是0.5且第二權重是0.5。接著，在第二時間點，人體部位位於影像感測器360的視場外。與在第一時間點的第一權重和第二權重相同，在第二時間點，第一權重將是0.5且第二權重將是0.5。直到使用者的人體部位存在於第三感測資料中，第一權重和第二權重將根據第三感測資料而變化。In some embodiments, if the user's body part does not exist in the third sensing data, the first weight and the second weight are fixed. If the body part is outside the field of view of theimage sensor 360, the third weight and the second weight will be the same as the previous first weight and the second weight at the previous time point while the user's body part is still present in the third sensing data. For example, at a first time point, the body part is within the field of view of theimage sensor 360, and the first weight is 0.5 and the second weight is 0.5. Then, at a second time point, the body part is outside the field of view of theimage sensor 360. Same as the first weight and the second weight at the first time point, at the second time point, the first weight will be 0.5 and the second weight will be 0.5. Until the user's body part exists in the third sensing data, the first weight and the second weight will change according to the third sensing data.

在另一實施例中，處理器250可根據第一位置信息、第二位置資訊以及第三位置資訊的加權組合判斷調整後的位置資訊。根據加權的第一位置信息、加權的第二位置資訊以及加權的第三位置資訊的總和判斷調整後的位置資訊。基於實際情況，三條位置資訊的權重可以是變化的或固定的。In another embodiment, theprocessor 250 may determine the adjusted location information based on a weighted combination of the first location information, the second location information, and the third location information. The adjusted location information may be determined based on the sum of the weighted first location information, the weighted second location information, and the weighted third location information. Based on actual circumstances, the weights of the three location information may be variable or fixed.

另一方面，關於朝向資訊，在一個實施例中，處理器250可將序列的第一感測資料直接作為朝向資訊。舉例來說，朝向資訊可以是加速度、三軸中的角速度、朝向、3-DoF資訊和/或6-DoF資訊。On the other hand, regarding the orientation information, in one embodiment, theprocessor 250 may directly use the first sensing data of the sequence as the orientation information. For example, the orientation information may be acceleration, angular velocity in three axes, orientation, 3-DoF information, and/or 6-DoF information.

在另一實施例中，處理器250可根據第三感測資料判斷朝向資訊。以圖4為例，影像中的人體部位B4在不同時間點的兩個姿勢可用於估測朝向資訊。In another embodiment, theprocessor 250 can determine the orientation information according to the third sensing data. Taking FIG. 4 as an example, two postures of the human body part B4 in the image at different time points can be used to estimate the orientation information.

在一些實施例中，處理器250可根據第一感測資料和第三感測資料判斷朝向資訊。朝向資訊可以是第一感測資料和第三感測資料的加權組合。舉例來說，根據基於動作感測器130的加權的第一朝向資訊和基於影像感測器360的加權的第二朝向資訊的總和判斷位置資訊。In some embodiments, theprocessor 250 may determine the orientation information based on the first sensing data and the third sensing data. The orientation information may be a weighted combination of the first sensing data and the third sensing data. For example, the position information is determined based on the sum of the weighted first orientation information based on themotion sensor 130 and the weighted second orientation information based on theimage sensor 360.

在一個實施例中，影像感測器360的視場將是關於是否根據第三感測資料使用朝向資訊的條件。如果人體部位存在於第三感測資料中，那麼可根據第一感測資料和第三感測資料來判斷朝向資訊。如果人體部位不存在於第三感測資料中，那麼可僅根據第一感測資料來判斷朝向資訊。In one embodiment, the field of view of theimage sensor 360 will be a condition on whether to use the orientation information based on the third sensing data. If the human body part exists in the third sensing data, then the orientation information can be determined based on the first sensing data and the third sensing data. If the human body part does not exist in the third sensing data, then the orientation information can be determined based on the first sensing data alone.

在一個實施例中，處理器250可根據朝向資訊和位置資訊判斷使用者的動作資訊。可基於第一感測資料、第三感測資料或如上所述的第一感測資料和第三感測資料的組合產生朝向資訊。可基於如上所述的第一感測資料、第二感測資料以及第三感測資料產生位置資訊。以圖2中的人體部位B1或人體部位B2為例，動作資訊可與抬起、點、踢、踩或跳躍動作相關。In one embodiment, theprocessor 250 may determine the user's motion information based on the orientation information and the position information. The orientation information may be generated based on the first sensing data, the third sensing data, or a combination of the first sensing data and the third sensing data as described above. The position information may be generated based on the first sensing data, the second sensing data, and the third sensing data as described above. Taking the human body part B1 or the human body part B2 in FIG. 2 as an example, the motion information may be related to lifting, pointing, kicking, stepping, or jumping.

在另一實施例中，處理器250可根據基於第一感測資料的朝向資訊和基於第一位置信息和第二位置資訊的調整後的位置資訊判斷使用者的動作資訊。無論人體部位是否存在於第三感測資料中，處理器250可預測使用者的動作。In another embodiment, theprocessor 250 may determine the user's motion information based on the orientation information based on the first sensing data and the adjusted position information based on the first position information and the second position information. Regardless of whether the human body part exists in the third sensing data, theprocessor 250 may predict the user's motion.

在另一實施例中，處理器250可根據基於第一感測資料的朝向資訊和基於第二感測資料和第三感測資料的組合位置資訊判斷使用者的動作資訊。這意味著，當人體部位存在及不存在於第三感測資料中時，可在兩個期間內基於朝向資訊和組合位置資訊判斷動作資訊。In another embodiment, theprocessor 250 may determine the user's motion information based on the orientation information based on the first sensing data and the combined position information based on the second sensing data and the third sensing data. This means that when a human body part exists and does not exist in the third sensing data, the motion information may be determined based on the orientation information and the combined position information in both periods.

以圖4和圖5為例，在圖4中判斷人體部位B4的舉手動作，且在圖5中判斷放下手動作。接著，判斷人體部位B4的從上到下的擺動動作。Taking Fig. 4 and Fig. 5 as examples, the hand-raising motion of the human body part B4 is determined in Fig. 4, and the hand-lowering motion is determined in Fig. 5. Next, the swinging motion of the human body part B4 from top to bottom is determined.

在一個實施例中，處理器250可僅根據基於第二感測資料的位置資訊判斷使用者的動作資訊。在另一實施例中，處理器250可僅根據基於第二感測資料和第三感測資料的組合位置資訊判斷使用者的動作資訊。在一些實施例中，如果人體部位不存在於第三感測資料中，那麼處理器250可僅根據基於第二感測資料的位置資訊判斷使用者的動作資訊，且如果人體部位存在於第三感測資料中，那麼處理器250可僅根據基於第三感測資料的位置資訊或組合位置資訊判斷使用者的動作資訊。In one embodiment, theprocessor 250 may determine the user's motion information only based on the position information based on the second sensing data. In another embodiment, theprocessor 250 may determine the user's motion information only based on the combined position information based on the second sensing data and the third sensing data. In some embodiments, if the human body part does not exist in the third sensing data, theprocessor 250 may determine the user's motion information only based on the position information based on the second sensing data, and if the human body part exists in the third sensing data, theprocessor 250 may determine the user's motion information only based on the position information or the combined position information based on the third sensing data.

可追蹤人體部位的位移或軌跡，且可基於位移或軌跡判斷動作資訊。以圖4和圖5為例，人體部位B3從上到下移動，且判斷人體部位B4從上到下擺動動作。The displacement or trajectory of a human body part can be tracked, and the motion information can be determined based on the displacement or trajectory. Taking Figures 4 and 5 as examples, the human body part B3 moves from top to bottom, and the human body part B4 is determined to swing from top to bottom.

綜上所述，在本發明實施例針對使用者數個操作部位的相關於資料融合的系統及方法中，可基於訊號強度、動作感測器的感測結果及/或相機影像追蹤人體部位的動作。若依據相機影像的偵測結果得出追蹤的人體部位未存在於FOV內，則動作感測設備之間的訊號強度可用於補償使用動作感測器所得的感測結果在位置資訊決定上的準確性。此外，若追蹤的人體部位存在於FOV內，則相機影像可用於修正基於訊號強度的位置估測結果。藉此，數個追蹤方法可用於不同條件，並據以改善追蹤結果的準確性。In summary, in the system and method for data fusion related to multiple operating parts of the user in the embodiments of the present invention, the movement of the human body part can be tracked based on the signal strength, the sensing results of the motion sensor and/or the camera image. If it is concluded that the tracked human body part does not exist in the FOV based on the detection results of the camera image, the signal strength between the motion sensing devices can be used to compensate for the accuracy of the sensing results obtained using the motion sensor in determining the position information. In addition, if the tracked human body part exists in the FOV, the camera image can be used to correct the position estimation result based on the signal strength. In this way, multiple tracking methods can be used for different conditions to improve the accuracy of the tracking results.

雖然本發明已以實施例揭露如上，然其並非用以限定本發明，任何所屬技術領域中具有通常知識者，在不脫離本發明的精神和範圍內，當可作些許的更動與潤飾，故本發明的保護範圍當視後附的申請專利範圍所界定者為準。Although the present invention has been disclosed as above by the embodiments, they are not intended to limit the present invention. Any person with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be defined by the scope of the attached patent application.

10:動作追蹤系統 20:動作追蹤系統 100:動作感測設備 110:無線收發器 130:動作感測器 200:計算設備 240:記憶體 250:處理器 300:頭戴式顯示器 310:無線收發器 360:影像感測器 B1、B2、B3、B4、B5:人體部位 S310、S330、S350:步驟 FOV:視場10: Motion tracking system20: Motion tracking system100: Motion sensing device110: Wireless transceiver130: Motion sensor200: Computing device240: Memory250: Processor300: Head mounted display310: Wireless transceiver360: Image sensorB1, B2, B3, B4, B5: Body partsS310, S330, S350: StepsFOV: Field of view

圖1是示出根據本發明的示例性實施例中的一個的動作追蹤系統的方塊圖。 圖2是示出根據本發明的示例性實施例中的一個的動作追蹤系統的示意圖。 圖3是示出根據本發明的示例性實施例中的一個的動作追蹤方法的流程圖。 圖4是示出根據本發明的示例性實施例中的一個的動作追蹤方法的示意圖。 圖5是示出根據本發明的示例性實施例中的一個的動作追蹤方法的示意圖。FIG. 1 is a block diagram showing an action tracking system according to one of the exemplary embodiments of the present invention.FIG. 2 is a schematic diagram showing an action tracking system according to one of the exemplary embodiments of the present invention.FIG. 3 is a flow chart showing an action tracking method according to one of the exemplary embodiments of the present invention.FIG. 4 is a schematic diagram showing an action tracking method according to one of the exemplary embodiments of the present invention.FIG. 5 is a schematic diagram showing an action tracking method according to one of the exemplary embodiments of the present invention.

S310~S350:步驟S310~S350: Steps

Claims (14)

Translated fromChinese

一種相關於資料融合的方法，適用於一系統，其中所述系統包括可穿戴在一使用者的人體部位上的多個動作感測設備，且所述相關於資料融合的方法包括：基於不同感測方式取得第一感測資料與第二感測資料；基於所述第一感測資料與所述第二感測資料判斷一資料組合；基於從一影像感測器所擷取的影像取得第三感測資料，其中所述第一感測資料與所述第二感測資料不是從所述影像取得；判斷所述使用者的多個人體部位中的一者在一第一期間內是否存在於所述第三感測資料中；反應於所述使用者的所述人體部位中的一者在所述第一期間內存在於所述第三感測資料中，由一第一判斷因素判斷所述使用者的動作資訊，其中所述第一判斷因素包括所述資料組合和所述第一期間的所述第三感測資料；以及反應於所述使用者的所述人體部位中的一者在所述第一期間內未存在於所述第三感測資料中，由一第二判斷因素判斷所述使用者的動作資訊，其中所述第二判斷因素包括所述資料組合和一第二期間的所述第三感測資料，且所述使用者的所述人體部位中的一者在所述第二期間內存在於所述第三感測資料中，其中判斷所述使用者的所述動作資訊的步驟包括：根據所述第一感測資料取得朝向資訊和第一位置信息；根據所述第二感測資料取得第二位置資訊；根據所述第三感測資料取得第三位置資訊；根據所述第一位置信息、所述第二位置資訊以及所述第三位置資訊取得調整後的位置資訊；以及根據所述朝向資訊和所述調整後的位置資訊判斷所述使用者的所述動作資訊，其中所述資料組合是所述第一感測資料和所述第二感測資料的加權組合，且根據所述第一位置信息、所述第二位置資訊以及所述第三位置資訊取得所述調整後的位置資訊的步驟包括：根據所述第三位置資訊判斷所述加權組合的一第一權重和一第二權重；透過將所述第一權重與所述第一位置信息相乘取得一第一參數；透過將所述第二權重與所述第二位置資訊相乘取得一第二參數；以及透過將所述第一參數與所述第二參數相加取得所述調整後的位置資訊，其中判斷所述加權組合的所述第一權重和所述第二權重的步驟包括：基於一方程式判斷所述第一權重和所述第二權重，其中所述方程式為所述第三位置資訊等於所述第一位置資訊和所述第二位置資訊的加權組合。A method for data fusion is applicable to a system, wherein the system includes a plurality of motion sensing devices wearable on a body part of a user, and the method for data fusion includes: obtaining first sensing data and second sensing data based on different sensing methods; determining a data combination based on the first sensing data and the second sensing data; obtaining third sensing data based on an image captured from an image sensor, wherein the first sensing data and the second sensing data are not obtained from the image; determining whether one of the plurality of body parts of the user exists in the third sensing data within a first period; and determining whether the third sensing data is in response to the body part of the user. In response to the fact that one of the human body parts of the user is present in the third sensing data during the first period, the motion information of the user is judged by a first judging factor, wherein the first judging factor includes the data combination and the third sensing data of the first period; and in response to the fact that one of the human body parts of the user is not present in the third sensing data during the first period, the motion information of the user is judged by a second judging factor, wherein the second judging factor includes the data combination and the third sensing data of a second period, and one of the human body parts of the user is present in the third sensing data during the second period, wherein the judging factor includes the data combination and the third sensing data of a second period. The step of obtaining the action information of the user includes: obtaining direction information and first position information according to the first sensing data; obtaining second position information according to the second sensing data; obtaining third position information according to the third sensing data; obtaining adjusted position information according to the first position information, the second position information and the third position information; and determining the action information of the user according to the direction information and the adjusted position information, wherein the data combination is a weighted combination of the first sensing data and the second sensing data, and the adjusted position information is obtained according to the first position information, the second position information and the third position information. The step of determining the adjusted location information comprises: determining a first weight and a second weight of the weighted combination according to the third location information; obtaining a first parameter by multiplying the first weight with the first location information; obtaining a second parameter by multiplying the second weight with the second location information; and obtaining the adjusted location information by adding the first parameter to the second parameter, wherein the step of determining the first weight and the second weight of the weighted combination comprises: determining the first weight and the second weight based on an equation, wherein the equation is that the third location information is equal to the weighted combination of the first location information and the second location information.如請求項1所述的相關於資料融合的方法，更包括：基於設置在所述動作感測設備上的動作感測器取得所述第一感測資料；以及基於在所述動作感測設備中的三者之間發射的無線信號取得第二感測資料。The method for data fusion as described in claim 1 further includes:Acquiring the first sensing data based on the motion sensor disposed on the motion sensing device; and acquiring the second sensing data based on the wireless signal transmitted between the three in the motion sensing device.如請求項1所述的相關於資料融合的方法，其中反應於所述使用者的所述人體部位中的一者存在於所述第三感測資料中，所述第一權重和所述第二權重根據所述第三感測資料變化。A method for data fusion as described in claim 1, wherein one of the human body parts of the user is present in the third sensing data, and the first weight and the second weight change according to the third sensing data.如請求項1所述的相關於資料融合的方法，其中反應於所述使用者的所述人體部位不存在於所述第三感測資料中，所述第一權重和所述第二權重是固定的。A method for data fusion as described in claim 1, wherein the human body part reflecting the user does not exist in the third sensing data, and the first weight and the second weight are fixed.如請求項1所述的相關於資料融合的方法，其中判斷所述使用者的所述動作資訊的步驟包括：根據所述第一感測資料取得朝向資訊；根據所述第二感測資料取得位置資訊；以及根據所述朝向資訊和所述位置資訊判斷所述使用者的所述動作資訊。The method for data fusion as described in claim 1, wherein the step of determining the action information of the user includes: obtaining direction information according to the first sensing data; obtaining location information according to the second sensing data; and determining the action information of the user according to the direction information and the location information.如請求項1所述的相關於資料融合的方法，其中判斷所述使用者的所述動作資訊的步驟包括：根據所述第一感測資料取得第一位置信息和朝向資訊；根據所述第二感測資料取得第二位置資訊；根據所述第一位置信息和所述第二位置資訊取得調整後的位置資訊；以及根據所述朝向資訊和所述調整後的位置資訊判斷所述使用者的所述動作資訊。The method for data fusion as described in claim 1, wherein the step of determining the action information of the user comprises: obtaining first position information and orientation information according to the first sensing data; obtaining second position information according to the second sensing data; obtaining adjusted position information according to the first position information and the second position information; and determining the action information of the user according to the orientation information and the adjusted position information.如請求項1所述的相關於資料融合的方法，其中判斷所述使用者的所述動作資訊的步驟包括：根據所述第一感測資料取得朝向資訊；在一第三期間內根據所述第二感測資料取得位置資訊的一第一部分；在一第四期間內根據所述第三感測資料取得位置資訊的一第二部分；將位置資訊的所述第一部分和所述第二部分組合為組合位置資訊；以及根據所述朝向資訊和所述組合位置資訊判斷所述使用者的所述動作資訊。The method for data fusion as described in claim 1, wherein the step of determining the action information of the user includes: obtaining direction information according to the first sensing data; obtaining a first part of location information according to the second sensing data in a third period; obtaining a second part of location information according to the third sensing data in a fourth period; combining the first part and the second part of location information into combined location information; and determining the action information of the user according to the direction information and the combined location information.一種相關於資料融合的系統，包括：多個動作感測設備，可穿戴在一使用者的多個人體部位上；一影像感測器；以及一處理器，經配置用以執行：基於不同感測方式取得第一感測資料與第二感測資料；基於所述第一感測資料與所述第二感測資料判斷一資料組合；基於從所述影像感測器所擷取的影像取得第三感測資料，其中所述第一感測資料與所述第二感測資料不是從所述影像取得；判斷所述使用者的所述人體部位中的一者在一第一期間內是否存在於所述第三感測資料中；反應於所述使用者的所述人體部位中的一者在所述第一期間內存在於所述第三感測資料中，由一第一判斷因素判斷所述使用者的動作資訊，其中所述第一判斷因素包括所述資料組合和所述第一期間的所述第三感測資料；反應於所述使用者的所述人體部位中的一者在所述第一期間內未存在於所述第三感測資料中，由一第二判斷因素判斷所述使用者的動作資訊，其中所述第二判斷因素包括所述資料組合和一第二期間的所述第三感測資料，且所述使用者的所述人體部位中的一者在所述第二期間內存在於所述第三感測資料中；根據所述第一感測資料取得朝向資訊和第一位置信息；根據所述第二感測資料取得第二位置資訊；根據所述第三感測資料取得第三位置資訊；根據所述第一位置信息、所述第二位置資訊以及所述第三位置資訊取得調整後的位置資訊；以及根據所述朝向資訊和所述調整後的位置資訊判斷所述使用者的所述動作資訊，其中所述資料組合是所述第一感測資料和所述第二感測資料的加權組合，其中所述處理器更經配置用以執行：根據所述第三位置資訊判斷所述加權組合的一第一權重和一第二權重；透過將所述第一權重與所述第一位置信息相乘取得一第一參數；透過將所述第二權重與所述第二位置資訊相乘取得一第二參數；透過將所述第一參數與所述第二參數相加取得所述調整後的位置資訊；以及基於一方程式判斷所述第一權重和所述第二權重，其中所述方程式為所述第三位置資訊等於所述第一位置資訊和所述第二位置資訊的加權組合。A system related to data fusion, comprising: a plurality of motion sensing devices, wearable on a plurality of body parts of a user; an image sensor; and a processor, configured to execute: obtaining first sensing data and second sensing data based on different sensing methods; determining a data combination based on the first sensing data and the second sensing data; obtaining third sensing data based on an image captured from the image sensor, wherein the first sensing data and the second sensing data are not obtained from the image; Determining whether one of the body parts of the user is present in the first period of time; In response to one of the human body parts of the user being present in the third sensing data during the first period, the action information of the user is judged by a first judging factor, wherein the first judging factor includes the data combination and the third sensing data of the first period; In response to one of the human body parts of the user not being present in the third sensing data during the first period, the action information of the user is judged by a second judging factor, wherein the second judging factor includes the data combination and the third sensing data of a second period, and the One of the body parts of the user is present in the third sensing data during the second period; obtaining orientation information and first position information according to the first sensing data; obtaining second position information according to the second sensing data; obtaining third position information according to the third sensing data; obtaining adjusted position information according to the first position information, the second position information and the third position information; and determining the action information of the user according to the orientation information and the adjusted position information, wherein the data combination is a weighted combination of the first sensing data and the second sensing data. combination, wherein the processor is further configured to perform: determining a first weight and a second weight of the weighted combination according to the third location information; obtaining a first parameter by multiplying the first weight with the first location information; obtaining a second parameter by multiplying the second weight with the second location information; obtaining the adjusted location information by adding the first parameter to the second parameter; and determining the first weight and the second weight based on an equation, wherein the equation is that the third location information is equal to the weighted combination of the first location information and the second location information.如請求項8所述的相關於資料融合的系統，其中所述動作感測設備中的每一者包括：一無線收發器，發射或接收無線信號；以及一動作感測器，感測所述使用者的所述人體部位中的一個的動作，且所述處理器經配置用以執行：基於所述動作感測設備的所述動作感測器取得所述第一感測資料：以及基於在所述動作感測設備中的三者之間發射的所述無線信號取得所述第二感測資料。A system related to data fusion as described in claim 8, wherein each of the motion sensing devices includes: a wireless transceiver that transmits or receives wireless signals; and a motion sensor that senses the motion of one of the body parts of the user, and the processor is configured to execute: obtaining the first sensing data based on the motion sensor of the motion sensing device; and obtaining the second sensing data based on the wireless signal transmitted between the three of the motion sensing devices.如請求項8所述的相關於資料融合的系統，其中反應於所述使用者的所述人體部位中的一者存在於所述第三感測資料中，所述第一權重和所述第二權重根據所述第三感測資料變化。A system related to data fusion as described in claim 8, wherein one of the human body parts of the user is present in the third sensing data, and the first weight and the second weight change according to the third sensing data.如請求項8所述的相關於資料融合的系統，其中反應於所述使用者的所述人體部位不存在於所述第三感測資料中，所述第一權重和所述第二權重是固定的。A system for data fusion as described in claim 8, wherein the first weight and the second weight are fixed in response to the fact that the human body part of the user does not exist in the third sensing data.如請求項8所述的相關於資料融合的系統，其中所述處理器經配置用以執行：根據所述第一感測資料取得朝向資訊；根據所述第二感測資料取得位置資訊；以及根據所述朝向資訊和所述位置資訊判斷所述使用者的所述動作資訊。A system related to data fusion as described in claim 8, wherein the processor is configured to execute: obtaining orientation information based on the first sensing data; obtaining location information based on the second sensing data; and determining the action information of the user based on the orientation information and the location information.如請求項8所述的相關於資料融合的系統，其中所述處理器經配置用以執行：根據所述第一感測資料取得第一位置信息和朝向資訊；根據所述第二感測資料取得第二位置資訊；根據所述第一位置信息和所述第二位置資訊取得調整後的位置資訊；以及根據所述朝向資訊和所述調整後的位置資訊判斷所述使用者的所述動作資訊。A system related to data fusion as described in claim 8, wherein the processor is configured to execute: obtaining first position information and orientation information based on the first sensing data; obtaining second position information based on the second sensing data; obtaining adjusted position information based on the first position information and the second position information; and determining the action information of the user based on the orientation information and the adjusted position information.如請求項8所述的相關於資料融合的系統，其中所述處理器經配置用以執行：根據所述第一感測資料取得朝向資訊；在一第三期間內根據所述第二感測資料取得位置資訊的第一部分；在一第四期間內根據所述第三感測資料取得位置資訊的第二部分；將位置資訊的所述第一部分和所述第二部分組合為組合位置資訊；以及根據所述朝向資訊和所述組合位置資訊判斷所述使用者的所述動作資訊。A system for data fusion as described in claim 8, wherein the processor is configured to perform: obtaining orientation information based on the first sensing data; obtaining a first portion of location information based on the second sensing data during a third period; obtaining a second portion of location information based on the third sensing data during a fourth period; combining the first portion and the second portion of location information into combined location information; and determining the action information of the user based on the orientation information and the combined location information.

Images