US11996002B2 - System and method for physical activity performance analysis - Google Patents

Abstract

A method for providing feedback to a skier includes: receiving, by a feedback computer, sensor data from a plurality of sensors located in a ski boot of the skier, the sensor data representing a movement of the skier during a ski run; receiving, by the feedback computer, video data representing a video recording of the skier during the ski run; after the ski run, synchronizing, by the feedback computer, the sensor data and the video data; and simultaneously displaying, on a display screen in electrical communication with the feedback computer, the video recording and the sensor data.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No. 17/658,197, filed Apr. 6, 2022, which is a continuation of U.S. application Ser. No. 15/154,347, filed May 13, 2016, which claims priority to U.S. Provisional Application No. 62/162,342, filed May 15, 2015, each of which are incorporated herein by reference. This application is a continuation of U.S. application Ser. No. 15/154,347, filed May 13, 2016.

TECHNICAL FIELD

The present disclosure generally relates to physical activity performance analysis, and more specifically to a system and method for using motion, pressure and position sensors embedded in equipment or footwear of a user to assess physical activity performance.

BACKGROUND

An important component to learning new skills is feedback. Often, this feedback is provided by coaches or instructors. For example, a basketball player can employ a coach to give feedback for improving their shot or dribbling, or a weightlifter can record a video of their lifting process to improve their ability to move the bar. While such feedback systems can help athletes improve, they have drawbacks. For example, it is difficult for a skier relying on a coach to receive real-time feedback. Moreover, such methods of critiquing a performance often have subjective biases rather than relying on quantitative data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG.1 is a block diagram illustrating components of a performance analysis system in accordance with an exemplary embodiment;

FIG.2 is a block diagram of a Sensor Unit in accordance with an exemplary embodiment;

FIG.3 is a block diagram of a Data Collection Unit in accordance with an exemplary embodiment;

FIG.4 is a flowchart for providing sensor information from a Data Collection Unit to the Server in accordance with an exemplary embodiment;

FIG.5A contains an insole component for the Sensor Unit in accordance with a first exemplary embodiment;

FIG.5B contains an insole component for the Sensor Unit in accordance with a second exemplary embodiment;

FIG.5C contains an insole component for the Sensor Unit in accordance with a third exemplary embodiment;

FIG.6 is an overview of a separate insole main unit sensor system design of the Sensor Unit in accordance with an exemplary embodiment;

FIG.7A contains an exposed view of a pressure sensing band and main unit sensor system design in accordance with a first exemplary embodiment;

FIG.7B contains an exposed view of a pressure sensing band and main unit sensor system design in accordance with a second exemplary embodiment;

FIG.7C illustrates and exemplary exposed view of the pressure sensing band;

FIGS.8A and8B is a flow chart of the user experience in accordance with an exemplary embodiment;

FIG.9 is a flowchart of a method for processing data in accordance with an exemplary embodiment;

FIGS.10A and10B illustrate an overview of a method to identify a type of surface a user is in contact with in accordance with an exemplary embodiment;

FIG.11 is a block diagram of a user interface in accordance with an exemplary embodiment;

FIGS.12A and12B illustrate an overview of a method to identify turns in the motion of a user in accordance with the exemplary embodiment;

FIG.13 is a flowchart of a method for the user experience and interaction between a user and their coach or trainer;

FIG.14 illustrates an exemplary computer system;

FIG.15 illustrates an example method embodiment;

FIG.16 illustrates an exemplary sensor embodiment;

FIG.17 illustrates an exemplary sensor embodiment connected to a wire; and

FIG.18 illustrates an exemplary array of sensors for deployment in footwear.

DETAILED DESCRIPTION

A system, method and computer-readable media are disclosed which aide in physical activity performance analysis, and more specifically to using motion, pressure and position sensors embedded in equipment or footwear of a user to assess physical activity performance. Various embodiments of the disclosure are described in detail below. While specific implementations are described, it should be understood that this is done for illustration purposes only. The description is not to be considered as limiting the scope of the embodiments described herein. Other components and configurations may be used without parting from the spirit and scope of the disclosure.

The term “coupled” is defined as connected, whether directly or indirectly through intervening components, and is not necessarily limited to physical connections. The term “substantially” is defined to be essentially conforming to the particular dimension, shape or other word that substantially modifies, such that the component need not be exact. For example, substantially rectangular means that the object in question resembles a rectangle, but can have one or more deviations from a true rectangle.

The phrase “physical activity” refers to sports (such as running, cycling, mountain biking, skating, skiing, snowboarding, tennis, squash, rowing, horse riding, soccer, rugby, surfing, kitesurfing, wakeboarding, water skiing, golf, or lacrosse) and non-sports activities (such as dance, walking, fashion runway walking). The phrase “activity performance” refers to the proficiency in the skill or technique used for the activity. For instance, activity performance for skiing could refer to the proficiency of the skier including aspects of technique such as the smoothness of turning, appropriate edge angle to the slope, balance of the skier, or ability to navigate types of terrain. To give another example, the activity performance in ballet would refer to movement and the accuracy of foot stretch, point, wing, lift arch, ankle, demi pointe or pointe. For fashion walking, activity performance could refer to the gait, stride length, speed of turn or synchronization of the model with others in a group. A “performance” can refer to both a “snapshot” of the user performing the activity (i.e., how the user is performing at a single point in time) or an entirety of an athletic performance (i.e., an entire ski run). The phrase “real-time” is according to the common vernacular of those persons skilled in this art—that is, a description for a system that responds to an external event within a short and predictable time frame (i.e., with immediacy).

The invention addresses the many difficulties associated with traditional instructor based physical activity and video analysis tuition; including, the expense of finding and paying an instructor, the limited resources of instructor for individuals when teaching a class, the outcome that people coach themselves without feedback and difficulties teaching a class of students with a dispersion of abilities. More specifically, the present disclosure relates to a system including motion, pressure and position sensors to collect data, process, and display data and analytics to augment and replace the current visual and verbal interactions of activity instructors with their pupils. The invention replicates the experience of an instructor by processing sensor data in novel methods to give real-time advice to the user, either through headphones or near eye displays; near-real time advice on a portable display, such as a mobile phone; or long term in-depth analysis either on a mobile phone, tablet or a computing device. By incorporating feedback from the system into their training, the user can improve their performance. The use of machine generated feedback has the advantage that it is objective and can be tailored to the specific activity, discipline, or style within a given activity.

Referring toFIG.1, a block diagram overview in accordance with an exemplary embodiment is illustrated. Thesystem100 can include aSensor Unit102, aData Collection Unit104, aServer106 and aFeedback Delivery Unit108. TheSensor Unit102 can measure data related to a user's technique. TheData Collection Unit104 can store and process performance information. TheServer106 can process the data and store the data. TheServer106 functions can include, but is not limited to, storage of user data, performance analysis, performance ranking of users or hosting of web based performance analysis. TheFeedback Delivery Unit108 can provide the user with feedback to improve the user's technique. Thesystem100 can include a plurality ofSensor Units102. The Sensor Unit(s)102, theData Collection Unit104 and/or theFeedback Delivery Unit108 can be contained in one or more units. For example, asystem100 can include aSensor Unit102 for each foot of the user and anotherSensor Unit102 located on the upper body of the user. The Sensor Unit on the upper body may consist of the GPS308,Accelerometer204,Gyroscope208, andMagnetometer206 in a mobile phone, or any combination thereof. In addition, theSensor Unit102 can have other sensor components as required for the particular activity being measured. It is also possible for one or a plurality of the Sensor Unit(s) to share the same housing as the Data Collection Unit. TheSensor Unit102 gathers sensor data and then transmits the data to theData Collection Unit104. TheSensor Unit102 can be communicatively coupled with theData Collection Unit104 wirelessly or via one and/ormore wires110. Alternatively, theSensor Unit102 can store data for later transmission via110 allowing theSensor Unit102 to function independently of theData Collection Unit104 for an extended period of time. Wireless transmission can be done via Bluetooth, Wi-Fi, or any radio frequency. TheData Collection Unit104 can be communicatively coupled with theServer106. When coupled, theData Collection Unit104 can transmit gathered data and can receive data (including data relating to the performance of other users, local maps, weather forecasts). TheServer106 functions can include, but are not limited to, storage of user data, performance analysis, performance ranking of users or hosting of web based performance analysis. TheData Collection Unit104 can be communicatively coupled with theServer106 wirelessly and/or via one ormore wires112. The wireless transmission can be done via the internet, Wi-Fi, and/or Bluetooth and/or via a combination of wired (via one or more wires112) and wireless data transmission. TheFeedback Delivery Unit108 is communicatively coupled with theData Collection Unit104 and theServer106. The coupling can be done via wireless transmission and/or via one or more wires,114,116.Connections114 and116 can be Bluetooth, Internet based, Wi-Fi, wired, or any radio frequency connection. In one or more embodiments, theData Collection Unit104 andFeedback Delivery Unit108 can be contained in the same housing. TheFeedback Delivery Unit108 can convey performance feedback to the user, either visually, using a display, haptic feedback, through audio via a speaker and/or headphones, and/or through a third party device such as a near eye display. The feedback delivery units can take the form of, but are not limited to, a near eye display, a mobile phone, a tablet, a Personal Computer, a haptic feedback unit on the upper body, a haptic feedback unit on the insoles, or a haptic feedback unit residing within the footwear of the user or residing in and/or on the Sensor Unit(s).

Referring toFIG.2, a block diagram of the Sensor Unit in accordance with an exemplary embodiment is illustrated. TheSensor Unit102 can be located on the body or equipment suitable for a given physical activity. For instance, for skiing theSensor Unit102 can be located in the ski boots, on the insole, and on or in the skis and/or bindings. For running, aSensor Unit102 can be placed in one or both of the shoes or insoles. For example, the sensor could be built into the shoe, placed below the insole, built into the insole, placed above the insole but below the foot, etc. For horse riding, aSensor Unit102 can be placed in one or both of the shoes or stirrups. TheSensor Unit102 can include one or more sensors. For example, the sensors can include, but are not limited to, anAccelerometer204,Magnetometer206,Gyroscope208, Global Positioning System (218),Microphone220 andForce Sensors216. The sensors can be internal sensors and/or external sensors. TheForce Sensors216 can be either capacitive or resistive sensors that are coupled to theProcessing Unit210 usingForce Sensor Electronics202. Data from the sensors is processed on theProcessing Unit210. The processed data can be sent to theStorage Unit222 or the Radio Frequency Unit for transmission via theAntenna214 by a wireless protocol such as Wi-Fi, Bluetooth or other Radio Frequency. TheForce Sensor Electronics202 can be coupled with theForce Sensors216 to prepare force measurements for theProcessing Unit210. TheForce Sensors216 can include a capacitive sensing circuit with a digital output and/or a resistive circuit and one or more analog to digital converters. TheProcessing Unit210 can provide several functions, including but not limited to; sensor fusion of data from theAccelerometer204,Magnetometer206 andGyroscope208, to calculate the orientation of the unit; the integration of orientation measurements andAccelerometer204 measurement to provide dead-reckoning position estimates; the combination of GPS308 with motion data; signal processing of sensors; and processing of data for compression in the Storage302 or for transmission. Orientation can be computed in theprocessing unit210 by combining the data from204,206 and208 using a sensor fusion algorithm. Examples of possible sensor fusion algorithms that could be used include direct cosine matrix (See for instance, Premerlani, William, and Paul Bizard. “Direction cosine matrix IMU: Theory.” DIY DRONE: USA (2009): 13-15), an extended Kalman filter (See for instance, Marins, Joao Luis, et al. “An extended Kalman filter for quaternion-based orientation estimation using MARG sensors.” Intelligent Robots and Systems, 2001. Proceedings. 2001 IEEE/RSJ International Conference on. Vol. 4. IEEE, 2001) or other appropriate motion fusion algorithms. TheSensor Unit100 can communicate with theData Collection Unit102 via theAntenna214, where data is sent through theProcessing Unit210, to theRadio Frequency Unit212 for transmission by theAntenna214.

Referring toFIG.3, a block diagram of the Data Collection Unit in accordance with an exemplary embodiment is illustrated. TheData Collection Unit104 can contain aSensor Unit102 and/or theFeedback Delivery Unit108. Data from theSensor Units102 can be received through theantenna214 by a wireless protocol such as Wi-Fi, Bluetooth or other radio frequencies. The data from theRadio unit212 and the GPS Unit308 can be sent to the Storage Unit302 to generate adetailed user analysis408 using theLocal Analysis Unit306 and later to a Website and Web BasedAnalysis410. The data from302 and the GPS Unit308 can be sent to the Real-TimeAnalysis Processing Unit304. The Real-TimeAnalysis Processing Unit304 can calculate metrics to measure the performance of the activity of the user. The processed data can be sent to theFeedback Delivery Unit108 either through theantenna214 by a wireless protocol such as Wi-Fi, Bluetooth or other Radio Frequency or is embedded in the Data Collection Unit.

Referring toFIG.4, a flowchart for a method for communicating data via a wide area network, in accordance with an exemplary embodiment is illustrated. Theexemplary method400 is provided by way of example, as there are a variety of ways to carry out the method. Themethod400 described below can be carried out using the configurations illustrated inFIGS.1-3 by way of example, and various elements of this figure are referenced in explainingexemplary method400. Each block shown inFIG.4 represents one or more processes, methods or subroutines, carried out in theexemplary method400. Theexemplary method400 can begin at block402.

At block402, data is gathered. For example, theData Collection Unit104 gathers data from the Sensor Unit(s)102. After gathering data, themethod400 proceeds to block404. Atblock404, data is transmitted to the server. For example, theData Collection Unit104 transmits data to theServer106 usingconnection108. Afterblock404 themethod400 proceeds to block406. Atblock406, the data is stored. For example, the data transmitted through theconnection108 is stored onServer106. Afterblock406 themethod400 proceeds to block408 and416. Atblock416 the data is processed. For example, theserver106 processes the data for the web based analysis platform. Afterblock416method400 proceeds to418.

At block418 the data is converted or transformed and stored. For example, the data is converted or transformed by theserver106 into a form suitable for in-depth web analysis and then stored. After block418 themethod400 proceeds to block420. Atblock420 presents the data to allow for analysis. For example, the data is accessed onserver106 by external users for analysis of performance. Atblock408 optimization for the machine learning algorithms takes place in the case where the provided data is from a user with a pre-classified (either manually or automatic) skill type. Example approaches could focus on choice of model parameters for metric calculation to distinguish between good and bad technique. Each performance metric is assigned a weighting then the users receive a weighted score. The metric weighting can be chosen so as to maximize the statistical significance of the difference between the groups of users with good and bad technique. For instance, in the case of a Weighted Least Squares classifier the introduced classified user data could be used to adjust metric weightings in order to maximize the squared sum of differences between the good and bad groups of users.

Atblock410 metric processing is performed to calculate metrics related to the user's performance. The choice of metrics and their calculation is based on a model specific to the physical activity. For instance, in the case of skiing where sensor units are placed on each boot or ski of the skier as well as the body of the skier, performance metrics could include: (1) Transition point, defined as the point in the turn of the skier where the sum of the skier's center of mass (defined as the distance weighted sum of force measured by sensors) transfers from the left to the right foot (or right to the left). Where the point of turn for a given turn is defined, for instance, as the percentage of the whole turn that the skier turns through to complete the turn. For example, if the turn is 30 degrees (from furthest clockwise to furthest anti-clockwise direction) then 15 degrees through the turn corresponds to a 50 percent point (the transition point) of turn. The metric for transition point could be the average transition point across all of the turns (Average Across Turns) of the user or for a user's session. (2) Forward/back position, the percentage time that the skier's center of mass spends forward of a given threshold. (3) Left/right bias, the left/right distance the skiers time or distance averaged center of mass is away from the skier's center of mass when standing still. (4) The correlation coefficient between the time series forward/back center of mass and the speed of the skier. (5) The correlation coefficient between the forward/back center of mass and the steepness of the slope. (6) The percent of time or distance skied that the skis yaw of the skis pointing towards each other above a given threshold. (7) The correlation coefficient between the difference between the yaw of skis and the speed of the skier. (8) The correlation coefficient of the yaw difference between skis and the speed of the skier conditioned for when the skis are divergent (9) the percentage of time or distance skied that the skiers are diverging above a given threshold. (10) The correlation coefficient between the yaw difference of the skis and speed conditioned for when the skis are divergent. (11) The average of the maximum roll of the skis for turns. (12) The average difference in roll of the skis across turns. (13) The correlation coefficient between the maximum roll for the skis and the speed of the skier or steepness of the slope. (14) The correlation coefficient in the difference in roll of the skis and the speed of the skier or steepness of the slope. (15) The average speed of the skier. (16) The time of the skier for a given course. (17) The speed of the skier for given type of terrain. (18) The correlation coefficient for the yaw (or other angle) of the upper body and the sensors units on the skis. (19) The correlation coefficient between the yaw of the upper body and the lower body with a lead or lag relationship for a given time. Afterblock410method400 proceeds to block412,422 and414.

At block422 the user's performance metrics are stored. The user's performance metrics are stored in the storage unit302.

Atblock412 the metrics for a given user are compared against pre-classified groups of users to classify a given user by skill level of type of skill. For instance in the case of skiing, if there are two groups of skiers known as good and bad skiers then a simple Of Least Squares classifier would calculate the sum of squared metric differences between the good and the bad groups. The group that the given user has the lowest sum of squared metric difference is the group the user is classified as. It should be obvious to a person with ordinary skill in the art that similar machine learning approaches can be used such as Generalized Least Squares, Least Squares, Artificial Neural Network, Bayesian Statistics or Support Vector Machines, Expectation-maximization algorithm, Vector Quantization, Generative Topographic map, or Information Bottleneck method. The system then transmits the user ranking to theFeedback Delivery Unit108. Afterblock412method400 proceeds to block414.

Referring toFIG.5A, an exposed view of an insole having the Force Sensors either wholly or partly embodying the Sensor Unit in accordance with an exemplary embodiment. As shown, theForce Sensors216 are circular in form based on standard sensors available for manufacture. Thebattery unit502 contained in (or on, or below) theinsole506 is used to power the system. Theelectronics unit504 in theinsole506 wholly or partly contains thesensor unit102. As shown, the insole500aincludes a first Force Sensor216a, a second Force Sensor216band a third force sensor216c.

Referring toFIG.5B, an exposed view of the Force Sensors either wholly or partly embodying the Sensor Unit in accordance with an exemplary embodiment. TheSensor Unit102 includes Force Sensors216a,216band216cof an organic shape that is form fitted to the foot. As shown, the insole500bcomprises three multi-component Force Sensors216a,216b,216cof rectangular shapes.

Referring toFIG.5C, an exposed view of an insole having circular Force Sensors embedded therein in (i.e., built into the insole itself) accordance with a third exemplary embodiment as illustrated. As shown, the insole500cincludes a first Force Sensor216a, a second Force Sensor216band a third force sensor216c.

Referring toFIG.6, an exposed view of an alternative design for the Sensor Unit. The depicted view is for the example implementation in a ski boot, but a similar design could be used for a variety of physical activities. TheSensor Unit102 includes anInsole506, aMain Housing600, aHousing Strap602 and one ormore wires604. TheSensor Unit102 collects pressure data usingForce Sensors216 embedded in theInsole506 which is transmitted through the one ormore wires604 to theMain Housing600 for processing, storage and transmission. While in certain embodiments, the data collected can be actual pressure data, in other embodiments the pressure data can be pressure related data. Pressure related data can be data related to pressure, but obtained indirectly. For example, theSensor Unit102 can be configured to identify capacitance values which change based on how much pressure is being applied to theSensor Unit102. The capacitance values can have a linear or non-linear relationship to pressure, and therefore the data collected by theSensor Unit102 is “pressure related data.” Using such data, estimations of pressure can be calculated. Although such estimations may or may not be directly tied to standard calculations of pressure, such estimations can be referred to as “pressure.” The separation of theInsole506 and Main Housing allows for aninsole506 with a preferred thickness range of 0.2-2 mm, but is not limited to this range, to avoid discomfort to the user. The one ormore wires604 can have a 0.1-0.5 mm thickness, but are not limited to this range, to ensure the user does not notice the one ormore wires604 in the boot. The wire can either run inside the inner part of the ski boot or be taken between the inner boot and outer boot. To ensure suitability for the cable, defined as a collection of wires encased in a housing material, to run inside the boot the cable can have a high friction on the side in contract with the boot compared with a lower friction on the side in contact with the leg of the user.

Referring toFIG.7, of the exposed view of an alternative design for theSensor Unit102 with a pressure sensing band (FIG.16 will describe pressure sensors in greater detail).FIGS.7A and7B depict different views of the same design. The depicted view is for the example implementation in a ski boot, but a similar design could be used for a variety of physical activities. TheSensor Unit102 includes aforce sensing band706 and amain housing710. Theforce sensing band706 consists of theforce sensors216. Thehousing710 consists of the various other components of theSensor Unit102.

Referring toFIG.7A, an isometric exposed view of the sensor unit placed on the user's ski boot in accordance with an exemplary embodiment. Ahousing710 can be attached to the back of the boot. Aforce sensing band702 can be placed between the user's leg and the boot.

Referring toFIG.7B, a top down exposed view of the sensor unit placed on the user's ski boot in accordance with an exemplary embodiment. Ahousing710 can be placed on the back of the boot. Aforce sensing band702 can be placed on the inside of the boot between the user's foot and the boot.

Referring toFIG.7C, an exposed view of the pressure sensing band in accordance with an exemplary embodiment. Theforce sensing band702 can be coupled with thehousing710 through theconnection708 which consists of wires encased in a housing material. Thepressure sensors704 can monitor the pressure applied in a given direction. Data from the pressure sensors is gathered by thesensor unit102. The pressure sensing band is closed by theconnector706 which is used to surround the users leg or attach to the inside of the ski boot. Theconnector706 may be made of but not limited to an adhesive, hook and loop fastener or magnets.

Referring toFIGS.8A and8B, a flowchart for the User Interface and User experience on the Feedback Delivery Unit, in accordance with an exemplary embodiment is illustrated. Theexemplary method800 is provided by way of example, as there are a variety of ways to carry out the method. Themethod800 described below can be carried out using the configurations illustrated inFIGS.1-7 and8-12 by way of example, and various elements of this figure are referenced in explainingexemplary method800. Each block shown inFIG.8 represents one or more processes, methods or subroutines, carried out in theexemplary method800. Theexemplary method800 can begin atblock802. Themethod800 also depicts the interaction of the user with the lessons offered by the system. A lesson is defined as the training of the user to achieve a target value for a single performance metric or for several performance metrics.

For example, in skiing, the user may choose to have a lesson to improve their performance metric related to skiing with skis in parallel. Such a lesson will monitor the performance of the user, usingSensor Units102, in relation to the performance metric and provide them with corrective feedback through theFeedback Delivery Unit108 in Real-Time. A further example could include a lesson to train the user to have the weighting on the outside ski for each turn, such a lesson would target the left/right bias metric. In boxing, the user may choose lessons to work on footwork, with performance measured based on how the user uses foot pressure and how fast/accurately the user moves their feet.

In this case real-time feedback through headphones could be delivered through theFeedback Delivery Unit108 whenever the user completes a turn with a large left or right pressure bias. Another example lesson could be based around the forward/backward lean metric. When the user is leaning too far backward for a given threshold of turns theFeedback Delivery Unit108 reminds the user through the headphones to keep their weight forward in their boots. Another example could include the lead/lag relationship between the skis (where a sensor unit is placed on and/or in each ski and/or boot) and a second sensor associated with the user (either on a ski pole, in a glove, on a watch, in a phone, etc.). In the case where the second sensor is on the upper body, the user would receive positive feedback that the upper body is not leading the turns of the skier. For the case of the physical activity of dancing a lesson could focus on a particular pattern of steps. Each time the user takes the lesson the users path is compared to a known path using the Of Least Squares method, then the user receives feedback from theFeedback Delivery Unit108 about the parts of their dance step significantly different to the known path.

Atblock802, the existence of a user profile is determined. For example, the system determines if the user has a profile or needs to create a user profile. If a user profile exists, thesystem800 proceeds to block808. Otherwise the system proceeds to block804.

At block804, the system allows for the creation of a user profile. The profile consists of the username, first and last name, and email address of the user. For example, if the user decides to create a profile, the system proceeds to block806, otherwise the system proceeds to block810.

Atblock806 the user creates a profile (if necessary). For example, the user enters the details to be associated with their profile and this is stored. This can include data such as height, weight, age, foot/shoe size, boot and/or ski model and manufacturer, and/or even more precise data points such as tibia length, femur length, torso length, chest size, waist size, etc. Themethod800 then proceeds to block808.

Atblock808 the users profile is loaded. For example, the performance metrics associated with the user are loaded. After this, themethod800 then proceeds to block810.

Atblock810 the calibration state of the Sensor Unit(s)102 is checked. The system requests the user to position the sensors in a predetermined orientation, position or location and checks to see if the data correlates with these predetermined configurations. For example, in skiing, the system may ask the user to keep their skis parallel and next to each other and then check if the orientation data from the sensors shows that the skis are parallel. If not, themethod800 initiates a recalibration of the sensors. If calibration is required the method proceeds to block812. Otherwise the method proceeds to block814.

Atblock814 the system prepares for the recording of a session. For example, thesensor units102 are switched on and the data begins to be gathered by theData Collection Unit104. At this point the user can see the sensor data in real-time. Afterblock814, themethod800 proceeds to block816.

Atblock816 the user is presented with the option of recording the session when ready. TheFeedback Delivery Unit108 presents the option of recording the session to the user. If the user selects yes then the method proceeds to block818. Otherwise themethod800 proceeds to block814.

Atblock818 the system records the data. Sensor data from the sensors in the Sensor Unit(s) is saved to storage on theSensor Unit102 or theData Collection Unit104. Atblock818, data is gathered until the system receives a stop command. When the stop command is received, themethod800 proceeds to block820.

Atblock822 the system renders performance metrics calculated from the previous session. For example, the system renders the metrics such as the left/right weight bias of the user or the average and maximum roll of the skis. Afterblock822 themethod800 then proceeds to block816.

Atblock824 the user is given the option to start a lesson. The lesson mode can give the user real-time feedback on an aspect of performance. If the user selects “start lesson” then themethod800 proceeds to block826.

Atblock826 the user is presented with a number of lessons to aid in the improvement of technique associated with a performance metric. Upon selecting a given lesson the instructions are displayed to the user. The method then proceeds to block828.

Atblock828 the user is presented with instructions on how to improve the selected performance metric. For example, in skiing, the user may have chosen to improve their parallel skiing technique in which case the user is given advice to improve their parallel skiing technique. In snowboarding, the user may have selected to improve turning with their left foot trailing. After this, themethod800 proceeds to block830.

Inblock830 the user is given the option to initiate the lesson. If the user selects “Yes” then the method proceeds to832. Otherwise the system returns to the select lesson block826.

Inblock832 the method can record the lesson and can provide real-time feedback through the Feedback delivery unit. To use an example for a skiing lesson, the skier could be having a lesson where they are focusing on leaning forward in the boots. During the lesson theforce sensors216 would collect data relating to the position of the skier. This data would then be processed in theSensor Unit100,Data Collection Unit102, orFeedback Delivery Unit108. When the skier has the correct lean (defined as a threshold front of foot pressure measurement less the back of foot pressure sensor measurement) the feedback can be delivered by108 this can be done in real-time through the use of headphones, haptic feedback (for embodiments where haptic units are included in the Feedback Delivery Unit(s)108 and there is the appropriate mechanical contact with the user. The Haptic feedback can be located on the upper body, the insoles, residing in the footwear and/or in the Sensor Unit(s)102) and/or near eye displays.

As another example, if the user were running, the pressure sensors could detect that the user is performing a heel strike, with an excessive amount of pressure landing on the heel and light pressure on the mid-foot or ball of the foot, and provide real-time feedback to either take shorter strides or land on the ball of the foot. Likewise for runners, if the runner is placing insufficient pressure on the heel, the feedback can provide instructions or otherwise indicate that the user should let the heel slightly touch the ground.

At block834 the user is given the option to finish the session, if the user selects yes the recording of data and real-time feedback is stopped, then the method proceeds to block834. Otherwise, the system continues to record data and deliver real-time feedback.

Atblock836 the user is presented with feedback on the lesson. Here the user can understand whether they made progress on the aspect of technique they were trying to focus on. The user can choose to view their progress over different timescales such as the last run, the last day, last week, last ski season or over the lifetime of the product. The value of the performance metric of the user before the lesson is compared with the value of the performance metric after. For example, the left\right bias of the skier is compared before and after the lesson. If the bias has decreased, the technique of the skier has been shown to have improved after the lesson.

Referring toFIG.9, a flowchart for the processing of sensor data to generate performance metrics, in accordance with an exemplary embodiment is illustrated. Theexemplary method900 is provided by way of example, as there are a variety of ways to carry out the method. Themethod900 described below can be carried out using the configurations illustrated inFIGS.1-8 and10-13 by way of example, and various elements of this figure are referenced in explainingexemplary method900. Each block shown inFIG.9 represents one or more processes, methods or subroutines, carried out in theexemplary method900. Theexemplary method900 can begin block902.

Atblock902 the system gathers data. For example, data from multiple sensors including204,206,208,216,218,220 and222 across a plurality ofsensor units102 are gathered by theData Collection Unit104. AfterBlock902, themethod900 proceeds to block904.

Atblock904 the surface is identified. For example, the system uses the data from thesensors220 and204 to identify thesurface using method1000. The calculation to identify the surface can be done by theprocessor210 of thedata collection unit104 and/or by theserver106. After identifying the surface, themethod900 proceeds to block906.

Atblock906 the turns of the user are identified to give context to performance metrics. The turns are identified using sensor data from sensors including204,206,208, and222 using the process described inFIG.12.Block904 proceeds to block908, block910 and block912.

Atblock908 turn pressure is analyzed. The analysis of the pressure along the turn is done using the methods inblock410. Afterblock908, themethod900 proceeds to block914.

Atblock910 the system analyses the data gathered related to the weight. Methods stated inblock410 are used. Afterblock908, themethod900 proceeds to block914. Atblock912 the gathered data is used to model the upper body. An example embodiment for a skier could be to compare the yaw of the skis to that of the upper body. This embodiment would allow the calculation of the magnitude of covariance of upper body yaw compared to ski yaw as well as the measurement of lead/lag relationship listed in examples of possible performance metrics. Afterblock912, themethod900 proceeds to block914. Atblock914 the metrics are stored. For example, the metrics calculated from the previous steps are stored in thestorage unit300.

Referring toFIG.10, a graph of the motion or audio data in the time and frequency domain in accordance with an exemplary embodiment is illustrated. Using the Fourier Transform, or any such similar transform, the user performance metrics can be classified by the surface. For example, in the case of cycling, the method can be used to differentiate between when the cyclist is on a rough or smooth road. Calculating performance metrics for both these two contexts allows the cyclist to understand the effect of road surface on performance. To give another example, in the case of skiing, contextual information is important to differentiate performance between skiing on powder, ice, packed snow and granular ice. The method can take place in the processing components of theSensor Unit102, theData Collection Unit104 orServer106. The method can be based on data from theAccelerometer204,Magnetometer206,Gyroscope208, andBarometer218,High Frequency Microphone220 orForce Sensors216 or any combination thereof. For the example case given inFIG.10 of an accelerometer, the data for a rough and smooth surface is denoted by1000 and1004, respectively. Once the Fourier transform is taken the peak can be found by finding the maximum in the frequency domain. To classify the surface the peak can be compared to frequency domain peaks for known surfaces. The peak is classified by calculating the squared difference between peaks of known surfaces. The classified surface will be that with the lowest squared difference in peak frequency. AlthoughFIG.10 provides a specific example of a Fourier transform combined with an Of Least Squares classifier other combinations could equally be applied such as Fourier or other time-series features with supervised (For instance artificial neural network, Bayesian statistics or support vector machines) or unsupervised learning (Such as expectation-maximization algorithm, vector quantization, generative topographic map, or the information bottleneck method).

Referring toFIG.11, a screenshot showing foot pressure and upper body orientation at different points during a path of a user, in accordance with an exemplary embodiment. As shown, the path is for a skier making a turn. One of ordinary skill in the art would understand that other user activities can be displayed, such as, but not limited to, climbing a hill on a bicycle, a jump in ballet or the foot pressure at different points in the steps of a dance. The illustration depicted inFIG.11 is suitable for many activities where the foot pressure varies at different points of the path of the user.FIG.11 depicts the pressure of different points in the turn of a skier, but the same technique could also be for other activities. To generate the image, data from theAccelerometer204,Magnetometer206,Gyroscope208 and process orientation calculation as well as Global Positioning System (GPS)222 or308 to give a representation of the path of the user can be combined. Overlaid on the path of the user is the foot pressure recorded from theForce Sensors216. The components of the turn where the user is placing greater pressure have a darker color. For the path depicted inFIG.11 the user places a stronger pressure at the center of the turn for theinside foot1108, denoted by the darker shading. For theoutside foot1106 the pressure is slighter with lighter shading.

FIG.11 also contains representations of the user's upper body labelled1100 and1104. The data for the representation is done by comparing processed data gathered by theData Collection Unit104 from theSensor Units102 placed on the Upper Body and the Insoles of the user. The data from themotion sensors310 and204/206/208 are processed to give the orientation of the users body using a Sensor Fusion algorithm. The yaw of the skiers body can be taken from the output of the Sensor Fusion and is used to display the representations of the upper body shown in1100 and1104. In1100 the user's body is facing the direction of the lower body, in1104 the user is keeping their body facing directly forward.

Referring toFIGS.12A and12B, a skier's path is shown in accordance with an exemplary embodiment and a screenshot showing the pressure and motion data in accordance with an exemplary embodiment is illustrated. The feature to identify turns is important to give the context to the pressure and motion data. The average yaw of the user's feet is calculated using a motion fusion algorithm using the motion data gathered by theData Collection Unit104 from theSensor Units102 containing thesensors204,206,208,218, and222.FIG.12A shows the path of auser1200 and acorresponding graph1202 of the average yaw of the user's feet is shown inFIG.12B. The portrayed embodiment measures thepeaks1204 andtroughs1206 of the yaw as the beginning and end of the turn. An equivalent technique could be applied for the roll and/or pitch of the user's feet with the identification of the same or different data features. The method can be conducted in the processing components of theSensor Unit100,Data Collection Unit102,Server104 orFeedback Delivery Unit110.

Referring toFIG.13, a flowchart for suggested interactions when using the system with a human coach, in accordance with an exemplary embodiment is illustrated. The coach system consists of an application running on a mobile phone or tablet PC. The coach system allows the coach to track a user's performance by viewing their performance metrics over time. The coach system also allows the coach to record a video of the user and synchronize the video with the data from the user. Theexemplary method1300 is provided by way of example, as there are a variety of ways to carry out the method. Themethod1300 described below can be carried out using the configurations illustrated inFIGS.1-12 by way of example, and various elements of this figure are referenced in explainingexemplary method1300. Each block shown inFIG.13 represents one or more processes, methods or subroutines, carried out in theexemplary method1300. Theexemplary method1300 can beginblock1302.

Atblock1302, the system is prepared. For example, the user attaches theSensor Units102 to various points on the body and equipment. For example, for a skier,Sensor Units102 could be placed on the each of the boots and the upper body of the user. In other embodiments, aSensor Unit102 could be included in the skier's ski poles. If the user were a boxer, aSensor Unit102 could be placed in the boxing gloves. As the user hits, data can be collected regarding (1) how hard the user has hit and (2) where on the user's hand the contact was initiated. After being prepared, thesystem1300 proceeds to block1304

Atblock1304, the system is activated. For example, the user switches on theSensor Units102.Block1304 proceeds to block1306.

Atblock1306, the system is internally coupled, meaning theSensor Units102 are coupled with theData Collection Unit104, such that the Data Collection Unit records the data being collected at the sensor. Such coupling can occur automatically, or can be user initiated. For example, a user can select to couple theSensor Units102 with theData Collection Unit104 via Bluetooth or other mechanisms. In other configurations, the sensor can automatically couple to a nearby data collection unit. Afterblock1306,method1300 proceeds to block1308.

Atblock1308, the system is calibrated. For example, a user engages theData Collection Unit104 to calibrate theSensor Units102. This could include certain movements or requiring the user to place the sensor units in a certain direction. Afterblock1308,method1300 proceeds to block1310.

Atblock1310, the user engages in the physical activity. For example, for skiing, the user would begin to ski. For horse riding, the user could begin to ride. At this time the method proceeds to block1312 and1318.

Atblock1312, data is gathered, processed and feedback is presented by the system. For example, theData Collection Unit104 gathers data from theSensor Units102. This is processed by theData Collection Unit104 and then presented by theFeedback Delivery Unit108. Afterblock1312, thesystem1300 proceeds to block1314.

Atblock1314, the activity is completed. For example, the data would stop being gathered. Afterblock1314, thesystem1300 proceeds to block1316.

Atblock1316, detailed analysis is rendered and presented by the system. For example, theData Collection Unit104 would process the gathered data and present detailed analysis through the Feedback Delivery Unit.

At block1318 the Coach System and User system are coupled, such that the activity coach identifies the user and has access to the data of theData Collection Unit104.Method1300 then proceeds to block1320.

At block1320 the coach system records the user. For example, the coach records the user with a video camera. This could be a traditional camera, or a smartphone or tablet PC camera. When the user finishes theactivity method1300 proceeds from1320 to1322.

Atblock1322, the data from the system is transferred to the Coach System. For example, the gathered and processed data from theData Collection Unit104, is transmitted to the Coach System through a wired connection or wireless connection via Wi-Fi, Bluetooth or any other Radio frequency. Afterblock1322, thesystem1300 proceeds to1324.

Atblock1324 the filmed data from the instructor is synchronized with the data of the user. The data can be synchronized by displaying a timestamp from theData Collection Unit104 to the video camera, using an audio signal or using a wireless signal such as Bluetooth, Wi-Fi between the coach's smartphone or tablet PC. Alternatively, the synchronization can be conducted using an internet clock or a global positioning system. Once synchronization is completed themethod1300 can continue to block1326.

Atblock1326 thefeedback delivery unit108 renders a detailed level of analysis with the recorded video footage of the user performing the activity. The recorded footage of the user can be augmented with, or alongside, sensor and processed sensor data.

FIG.14 provides a brief introductory description of a basic general purpose system or computing device which can be employed to practice the concepts is disclosed herein. Theexemplary system1400 includes a general-purpose computing device, including a processing unit (CPU or processor)1410 and asystem bus1405 that couples various system components including thesystem memory1415 such as read only memory (ROM)1420 and random access memory (RAM)1435 to theprocessor1410. Thesystem1400 can include acache1412 of high speed memory connected directly with, in close proximity to, or integrated as part of theprocessor1410. Thesystem1400 copies data from thememory1415 and/or thestorage device1430 to thecache1412 for quick access by theprocessor1410. In this way, the cache provides a performance boost that avoidsprocessor1410 delays while waiting for data. These and other modules can control or be configured to control theprocessor1410 to perform various actions.Other system memory1415 may be available for use as well. Thememory1415 can include multiple different types of memory with different performance characteristics. It can be appreciated that the disclosure may operate on acomputing device1400 with more than oneprocessor1410 or on a group or cluster of computing devices networked together to provide greater processing capability. Theprocessor1410 can include any general purpose processor and a hardware module or software module, such asmodule11432,module21434, andmodule31436 stored instorage device1430, configured to control theprocessor1410 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. Theprocessor1410 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, etc. A multi-core processor may be symmetric or asymmetric.

Thesystem bus1405 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. A basic input/output (BIOS) stored inROM1420 or the like, may provide the basic routine that helps to transfer information between elements within thecomputing device1400, such as during start-up. Thecomputing device1400 further includesstorage devices1430 such as a hard disk drive, a magnetic disk drive, an optical disk drive, or tape drive. Thestorage device1430 can includesoftware modules1432,1434,1436 for controlling theprocessor1410. Other hardware or software modules are contemplated. Thestorage device1430 is connected to thesystem bus1405 by a drive interface. The drives and the associated computer-readable storage media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for thecomputing device1400. In one aspect, a hardware module that performs a particular function includes the software component stored in a tangible computer-readable storage medium in connection with the necessary hardware components, such as theprocessor1410,bus1405,display1435, and so forth, to carry out the function. In another aspect, the system can use a processor and computer-readable storage medium to store instructions which, when executed by the processor, cause the processor to perform a method or other specific actions. The basic components and appropriate variations are contemplated depending on the type of device, such as whether thedevice1400 is a small, handheld computing device, a desktop computer, or a computer server.

Although the exemplary embodiment described herein employs thehard disk1430, other types of computer-readable media which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, digital versatile disks, cartridges, random access memories (RAMs)1435, and read only memory (ROM)1420, may also be used in the exemplary operating environment. Tangible computer-readable storage media, computer-readable storage devices, or computer-readable memory devices, expressly exclude media such as transitory waves, energy, carrier signals, electromagnetic waves, and signals per se.

To enable user interaction with thecomputing device1400, aninput device1445 represents any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech and so forth. Anoutput device1435 can also be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems enable a user to provide multiple types of input to communicate with thecomputing device1400. Thecommunications interface1440 generally governs and manages the user input and system output. There is no restriction on operating on any particular hardware arrangement and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

For clarity of explanation, the illustrative system embodiment is presented as including individual functional blocks including functional blocks labeled as a “processor” orprocessor1410. The functions these blocks represent may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software and hardware, such as aprocessor1410, that is purpose-built to operate as an equivalent to software executing on a general purpose processor. For example the functions of one or more processors presented inFIG.14 may be provided by a single shared processor or multiple processors. Illustrative embodiments may include microprocessor and/or digital signal processor (DSP) hardware, read-only memory (ROM)1420 for storing software performing the operations described below, and random access memory (RAM)1435 for storing results. Very large scale integration (VLSI) hardware embodiments, as well as custom VLSI circuitry in combination with a general purpose DSP circuit, may also be provided.

The logical operations of the various embodiments are implemented as: (1) a sequence of computer implemented steps, operations, or procedures running on a programmable circuit within a general use computer, (2) a sequence of computer implemented steps, operations, or procedures running on a specific-use programmable circuit; and/or (3) interconnected machine modules or program engines within the programmable circuits. Thesystem1400 shown inFIG.14 can practice all or part of the recited methods, can be a part of the recited systems, and/or can operate according to instructions in the recited tangible computer-readable storage media. Such logical operations can be implemented as modules configured to control theprocessor1410 to perform particular functions according to the programming of the module. For example,FIG.14 illustrates threemodules Mod11432,Mod21434 andMod31436 which are modules configured to control theprocessor1410. These modules may be stored on thestorage device1430 and loaded intoRAM1435 ormemory1415 at runtime or may be stored in other computer-readable memory locations.

Having disclosed some components of a computing system, the disclosure now turns toFIG.15, which illustrates an exemplary method embodiment. For the sake of clarity, the method is described in terms of anexemplary system1400 as shown inFIG.14 configured to practice the method. The steps outlined herein are exemplary and can be implemented in any combination thereof, including combinations that exclude, add, or modify certain steps.

Thesystem1400 receives data at a sever, wherein the data is received from at least one sensor embedded in an insole of footwear of a user (1502). In other embodiments, the data can be received from at least one sensor in footwear of a user, where the at least one sensor is located between a foot of the user and a sole of the footwear. Exemplary footwear of the user can include a ski boot, a snowboarding boot, a water skiing boot, a wakeboarding boot, a tennis shoe, an ice skate, a golf shoe, a rowing shoe, and a horse riding boot. Thesystem1400 processes the data to evaluate a performance of the user, to yield processed data (1504), and transmits the processed data to a feedback device, wherein the feedback device provides feedback to the user regarding the performance of the user using the processed data (1506).

In certain configurations, processing the data to evaluate the performance of the user can further include: identifying a pressure applied to the at least one sensor by a foot of the user; identifying a position and an orientation of the user; and comparing the pressure, the position, and the orientation to a best practice threshold. In other configurations, processing the data to evaluate the performance of the user can also include: combining the processed data with a video analysis of a recording of the performance of the user.

While thesystem1400 can operate entirely using sensors in the user's footwear, the user can have additional sensors on their person, which are not associated with the footwear, and thesystem1400 can receive additional data from these additional sensors, then utilize this additional data during the processing of the data/evaluation of the user's performance. For example, an additional sensor can be embedded in a glove, goggles, a ski pole, boxing glove, golf club, rowing oar, etc.

Certain configurations of thesystem1400 allow for the user to select lessons to hone specific aspects of performance. In addition, thesystem1400 can suggest lessons to the user based on their performance. Such suggestions can be based on previous performances which may or may not be associated with lessons. For example, if the user has taken a lesson and excelled in a subsequent performance, thesystem1400 can suggest a higher-level lesson, whereas if the user's subsequent performance is poor then thesystem1400 can suggest returning to a lower-level/basic lesson before returning to the current lesson level. Thesystem1400 can compare performances (and scores/rankings based on performances) after lessons.

The feedback device can provide the feedback while the user continues the performance. Exemplary feedback devices can include an audio feedback device, a haptic feedback device, and near-eye display feedback. One non-limiting example of feedback is a user-interface with an infographic, where the infographic is color coded along a path of the user according to pressure being applied. As a first example, the path could be colored red on those portions of the path where the user is applying/applied incorrect pressure and green on those portions of the path where the user applied correct pressure. As a second example, the path could be colored red where the user applied heavy pressure and green where pressure was lightly applied.

FIG.16 illustrates anexemplary sensor embodiment1600. More specifically,FIG.16 illustrates a cross-sectional view of an insole or insert either wholly or partly embodying theSensor Unit102. As shown, theSensor Unit102 is in part made up of a composite including awaterproof fabric cover1602, aconductive layer1604 that could be embodied by a conductive fabric, a layer of copper, printed conductive ink or deposited conductive material, anelastomer1606, a Flexible Printed Circuit (FPC)1610, anotherconductive layer1612 and anotherwaterproof fabric cover1614. In thisembodiment1600, the layers are bonded together with an adhesive1608. TheFPC1610 can have capacitance sensors soldered, molded, or otherwise built into theFPC1610. TheFPC1610 can have, for example, local processing chips for processing data collected by theSensor Unit102. The cover layers1602,1614 and the dielectric1606 can, in some configurations, be larger than theconductive fabric layers1604,1612 and theFPC1610, allowing for trimming back or forming thesensor1600 to specific requirements.

The composite forms a pressure sensing device by measuring the capacitance between pads on theFPC1610 to the groundedconductive fabric1604,1612. As orientated inFIG.16, vertical pressure on the composite causes theelastomer1606 to compress, increasing the capacitance between theFPC1610 and theconductive fabric1612. In one variation of1600, the adhesive1608 may be formed by a heated weld which bonds theelastomer1606 and both waterproof fabric covers1602,1614.

FIG.17 illustrates anexemplary sensor embodiment1700 similar to the sensor illustrated inFIG.16, but in this instance the illustrated sensor is connected to awire1702. Here, the capacitance sensing pads on theFPC1610 are connected to microchips on theFPC1610 that measure the capacitance and convert it to a digital signal. These microchips are placed as close as possible to the pads so as to reduce the effect of any stray capacitance on the tracks connecting to the pads. The cable/wire1702 connecting thesensor1600 to the microchips can be insulated, having anovermolded cable guide1704. Theconductive center1706 of thewire1702 connects with the FPC1610 (i.e., there is no insulation where the data is communicated with thewire1702 from the sensor). The cables can be connected to the insole using theovermolded cable guide1704. An alternative design can use a molded cable guide that is bonded to theFPC1610 using adhesive and to thefabric cover1602,1614 using a heated weld. The two layers ofconductive fabric1604,1612 can be electrically connected to ground through theFPC1610 by a conductive adhesive.

FIG.18 illustrates an exemplary array of sensors for deployment infootwear1800. As illustrated, the array has a plurality ofsensors1802 connected bycables1804 to microchips. Such an array can, for example, be inserted into a shoe above or below the standard insole of a shoe or boot, and allows collection of pressure related data as discussed above.

Embodiments within the scope of the present disclosure may also include tangible and/or non-transitory computer-readable storage media for carrying or having computer-executable instructions or data structures stored thereon. Such tangible computer-readable storage media can be any available media that can be accessed by a general purpose or special purpose computer, including the functional design of any special purpose processor as described above. By way of example, and not limitation, such tangible computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions, data structures, or processor chip design. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or combination thereof) to a computer, the computer properly views the connection as a computer-readable medium. Thus, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of the computer-readable media.

Computer-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Computer-executable instructions also include program modules that are executed by computers in stand-alone or network environments. Generally, program modules include routines, programs, components, data structures, objects, and the functions inherent in the design of special-purpose processors, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.

Other embodiments of the disclosure may be practiced in network computing environments with many types of computer system configurations, including personal computers, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. Embodiments may also be practiced in distributed computing environments where tasks are performed by local and remote processing devices that are linked (either by hardwired links, wireless links, or by a combination thereof) through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

The embodiments shown and described above are only examples. Therefore, many details are neither shown nor described. Even though numerous characteristics and advantages of the present technology have been set forth in the foregoing description, together with details of the structure and function of the present disclosure, the disclosure is illustrative only, and changes may be made in the detail, especially in matters of shape, size and arrangement of the parts within the principles of the present disclosure to the full extent indicated by the broad general meaning of the terms used in the attached claims. It will therefore be appreciated that the embodiments described above may be modified within the scope of the appended claims.

The various embodiments described above are provided by way of illustration only and should not be construed to limit the scope of the disclosure. For example, the principles herein can be applied to pressure measurements in a wide variety of sports, activities, and performances. Various modifications and changes may be made to the principles described herein without following the example embodiments and applications illustrated and described herein, and without departing from the spirit and scope of the disclosure.

Statement Bank

Statement 1: A method comprising: measuring data using at least one sensor, wherein the data is received from the at least one sensor in footwear of a user, the at least one sensor being located between a foot of the user and a sole of the footwear; processing the data to evaluate a performance of the user, to yield processed data; and transmitting the processed data to a feedback device, wherein the feedback device provides feedback to the user regarding the performance of the user using the processed data.

Statement 2: The method ofStatement 1, wherein the footwear of the user comprises a snow ski boot.

Statement 3: The method ofStatement 1 or 2, wherein the footwear of the user comprises one of a snowboarding boot, a water skiing boot, a wakeboarding boot, an ice skate, a golf shoe, a rowing shoe, a horse riding boot, and a cycling shoe.

Statement 4: The method of any of the above statements, wherein the processing of the data to evaluate the performance of the user further comprises: identifying a pressure applied to the at least one sensor by a foot of the user; identifying a position and an orientation of the user; and comparing the pressure, the position, and the orientation to a best practice threshold.

Statement 5: The method of any of the above statements, wherein the processing of the data to evaluate the performance of the user further comprises: combining the processed data with a video of the performance of the user.

Statement 6: The method of any of the above statements, further comprising receiving second data from a second sensor associated with the user, wherein the second sensor is not associated with the footwear; and utilizing the second data during the processing of the data.

Statement 7: The method of any of the above statements, wherein the second sensor is embedded in one of a ski pole, a helmet, a snow glove, and a goggle.

Statement 8: The method of any of the above statements, further comprising: suggesting a lesson to the user based on the performance of the user.

Statement 9: The method of any of the above statements, wherein the feedback device provides the feedback while the user continues the performance.

Statement 10: The method of any of the above statements, wherein the feedback comprises at least one of audio feedback, haptic feedback, and near-eye display feedback.

Statement 11: The method of any of the above statements, wherein the processing of the data to evaluate the performance of the user further comprises: determining, using the processed data, a category of a surface on which the user is conducting the performance.

Statement 12: The method of any of the above statements, wherein the data is recorded by the at least one sensor as the user is skiing, wherein each ski turn is detected using a peak detection on the processed data, and wherein the processed data is averaged across each ski turn to form a new performance metric for use in evaluating of the performance.

Statement 13: The method of any of the above statements, wherein the feedback device displays to the user a traced travel path of the user with changes in color describing changes in the processed data.

Statement 14: The method of any of the above statements, wherein the feedback device is one of a smartphone and a tablet.

Statement 15: The method of any of the above statements, wherein the at least one sensor measures capacitance, the capacitance varying as the user applies pressure within the footwear.

Statement 16: A system comprising: capacitance sensor in footwear of a user, the capacitance sensor being located between a foot cavity of the footwear and a sole of the footwear; a processor; and a computer-readable storage medium having instructions stored which, when executed by the processor, perform operations comprising: identifying, at the capacitance sensor, pressure related data associated with a performance of the user; and transmitting the pressure related data to a feedback device.

Statement 17: The system of Statement 16, wherein the footwear comprises one of a snow ski boot and a snowboard boot.

Statement 18: The system of Statement 16, wherein the footwear of the user comprises one of a water skiing boot, a wakeboarding boot, a tennis shoe, an ice skate, a golf shoe, a rowing shoe, a horse riding boot, and a cycling shoe.

Statement 19: The system of any of Statements 16 to 18, wherein the transmitting of the pressure related data occurs through one of a Bluetooth transmission, a Wi-Fi transmission, and radio frequency transmission.

Statement 20: The system of any of Statements 16 to 19, wherein the capacitance sensor comprises: an accelerometer; a magnetometer; and a gyroscope.

Statement 21: The system of any of Statements 16 to 20, wherein the sensor is coupled via a cable to the processor, and wherein the transmitting of the pressure related data occurs using a Radio Frequency signal.

Statement 22: The system of any of Statements 16 to 21, wherein the sensor identifies the pressure related data by measuring a change in capacitance between constituent components of the sensor.

Statement 23: The system of any of Statements 16 to 22, further comprising: a feedback device; and the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising: receiving, in response to the transmitting of the pressure related data, feedback data from the data processing device; providing, during a continuation of the performance, feedback via the feedback device to the user regarding the performance based on the feedback data, wherein the feedback comprises at least one of audio feedback, haptic feedback, and near-eye display feedback.

Statement 24: The system of any of Statements 16 to 23, wherein the sensor comprises: a flexible printed circuit; an elastomeric layer; and a conductive layer.

Statement 25: The system of any of Statements 16 to 24, further comprising: a data collection unit; and the computer-readable storage medium having additional instructions stored which, when executed by the processor, cause the processor to perform operations comprising: storing the pressure related data in a data collection unit prior to the transmitting of the pressure related data.

Statement 26: The system of any of Statements 16 to 25, wherein the data collection unit stores performance metrics from a previous performance.

Statement 27: A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising: receiving data at the computing device, wherein the data is received from at least one sensor in footwear of a user, the at least one sensor being located between a foot a foot cavity of the footwear and a sole of the footwear; processing the data to evaluate a performance of the user, to yield processed data; and transmitting the processed data to a feedback device, wherein the feedback device provides feedback to the user regarding the performance of the user using the processed data.

Statement 28: A method comprising: receiving data at a server, wherein the data is received from at least one sensor in footwear of a user, the at least one sensor being located between a foot of the user and a sole of the footwear; calculating, using the data, a weight distribution of the user over the footwear; evaluating a performance of the user using the center of mass, to yield processed data; and transmitting the processed data to a feedback device, wherein the feedback device provides feedback to the user regarding the performance of the user using the processed data.

Statement 27: The method of Statement 26, wherein the footwear of the user comprises a snow ski boot.

Statement 28: The method of Statements 26 or 27, wherein the data is recorded by the at least one sensor as the user is skiing, wherein each ski turn is detected, and wherein the processed data is averaged across each ski turn to form a new performance metric for use in the evaluating of the performance.

Statement 29: The method of any of Statements 26 to 28, wherein the processed data is displayed by the feedback device and shows a traced travel path of the user with changes in color describing changes in the processed data.

Statement 30: A system comprising: a sensor in a band which wraps around a leg of a user, the sensor being located between the leg and a snow ski boot; a processor; and a computer-readable storage medium having instructions stored which, when executed by the processor, perform operations comprising: identifying, at the sensor, pressure data.

Statement 31: The system ofStatement 30, wherein the sensor is a capacitive sensor which measures capacitance, the capacitance being modified as the user applies pressure within the snow ski boot.

Claims (20)

The invention claimed is:

1. A method for providing feedback to a skier, the method comprising:

receiving, by a feedback computer, sensor data from a plurality of sensors located in a ski boot of the skier, the sensor data representing a movement of the skier during a ski run;

receiving, by the feedback computer, video data representing a video recording of the skier during the ski run;

after the ski run, synchronizing, by the feedback computer, the sensor data and the video data; and

simultaneously displaying, on a display screen in electrical communication with the feedback computer, the video recording and the sensor data.

2. The method ofclaim 1, wherein the sensor data is displayed alongside the video recording.

3. The method ofclaim 1, wherein the sensor data and the video data are synchronized using time stamps.

4. The method ofclaim 1, wherein the sensor data and the video data are synchronized using wireless signals transmitted between the feedback computer and a data collection computer, the data collection computer in electrical communication with the sensors.

5. The method ofclaim 1, wherein the sensor data and the video data are synchronized using an internet clock or a global positioning system.

6. The method ofclaim 1, wherein the sensor data includes processed data that indicates a performance of the skier during the ski run.

7. The method ofclaim 6, wherein the processed data includes:

a pressure applied to at least one force sensor by a foot of the skier,

a position of the skier, and

an orientation of the skier.

8. The method ofclaim 7, wherein the processed data includes a comparison of the pressure, the position, and the orientation to a best practice threshold.

9. The method ofclaim 6, wherein:

the ski run includes a plurality of ski turns, and

the processed data includes, for each ski turn:

a detection of a ski turn using a peak condition, and

a performance metric relating to the ski turn.

10. The method ofclaim 1, further comprising:

collecting, with a data collection computer in electrical communication with the sensors, the sensor data; and

transferring the sensor data from the data collection computer to the feedback computer.

11. A system for providing feedback to a skier, the system comprising:

a display screen;

a processor in electrical communication with the display screen; and

a computer-readable storage medium have instructions stored thereon which, when executed by the processor, cause the processor to:

receive sensor data from a plurality of sensors located in a ski boot of the skier, the sensor data representing a movement of the skier during a ski run;

receive video data representing a video recording of the skier during the ski run;

after the ski run, synchronize the sensor data and the video data; and

simultaneously display, on the display screen, the video recording and the sensor data.

12. The system ofclaim 11, wherein the instructions further cause the processor to display the sensor data alongside the video recording.

13. The system ofclaim 11, wherein the instructions further cause the processor to synchronize the sensor data and the video data using time stamps.

14. The system ofclaim 11, wherein the instructions further cause the processor to synchronize the sensor data and the video data using wireless signals transmitted between the system and a data collection computer, the data collection computer in electrical communication with the sensors.

15. The system ofclaim 11, wherein the instructions further cause the processor to synchronize the sensor data and the video data using an internet clock or a global positioning system.

16. A method for providing feedback to a skier, the method comprising:

receiving, by a first computer, sensor data from a plurality of sensors located between a foot of the skier and a sole of a ski boot worn by the skier, the sensor data representing a movement of the skier during a ski run;

processing the sensor data with the first computer to evaluate a performance of the skier to yield processed data;

transmitting, after the ski run, the processed data to a second computer, the second computer storing video data representing a video recording of the skier during the ski run;

synchronizing, by the second computer, the processed data and the video data; and

simultaneously displaying, on a display screen in electrical communication with the second computer, the video recording and the processed data.

17. The method ofclaim 16, wherein:

the ski run includes a plurality of ski turns, and

for each ski turn, processing the sensor data comprises:

detecting a ski turn using a peak condition; and

determining a performance metric relating to the ski turn.

18. The method ofclaim 17, further comprising transmitting the processed data from the first computer to a feedback device, wherein the feedback device provides feedback to the skier regarding the performance of the skier for the ski turns, wherein for each ski turn the performance metric is transmitted to the feedback device once the skier completes the ski turn.

19. The method ofclaim 16, wherein:

the ski run includes a plurality of ski turns, and

for each ski turn, processing the sensor data comprises:

detecting a ski turn using a peak condition; and

determining a transition point relating to the ski turn.

20. The method ofclaim 19, further comprising transmitting the processed data from the first computer to a feedback device, wherein the feedback device provides feedback to the skier regarding the performance of the skier for the ski turns, wherein for each ski turn the transition point is transmitted to the feedback device once the skier completes the ski turn.

Images