CN113625873A - Interactive learning method and system based on audio recognition and multi-track sequence representation - Google Patents

Abstract

The invention provides an interactive learning method and system based on audio recognition and multi-track sequence representation, wherein the method comprises the following steps: step S1: acquiring MIDI files generated by playing musical instruments by users; step S2: analyzing the MIDI file, and extracting note information and characteristic information; step S3: acquiring a plurality of preset audio tracks, and representing corresponding note sequences on the audio tracks based on note information; step S4: and performing teaching marking on the audio track based on the characteristic information for the user to learn. According to the interactive learning method and system based on audio identification and multi-track sequence representation, a user can play and check the audio track at the same time to perform learning, the user does not need to go out, the convenience is high, meanwhile, one-to-one interactive learning is achieved, and the learning effect is improved.

Description

Interactive learning method and system based on audio recognition and multi-track sequence representation

Technical Field

The invention relates to the technical field of musical instrument learning, in particular to an interactive learning method based on audio recognition and multi-track sequence representation.

Background

At present, most of the study of musical instrument playing is class system, the effect is poor, and in addition, students need to go to the field by themselves to study, and the convenience is not enough;

therefore, a solution is needed.

Disclosure of Invention

One of the purposes of the invention is to provide an interactive learning method based on audio identification and multi-track sequence representation, so that a user can play and check the audio track at the same time to perform learning, the user does not need to go out, the convenience is high, meanwhile, one-to-one interactive learning is realized, and the learning effect is improved.

The embodiment of the invention provides an interactive learning method based on audio identification and multi-track sequence representation, which comprises the following steps:

step S1: acquiring MIDI files generated by playing musical instruments by users;

step S2: analyzing the MIDI file, and extracting note information and characteristic information;

step S3: acquiring a plurality of preset audio tracks, and representing corresponding note sequences on the audio tracks based on note information;

step S4: and performing teaching marking on the audio track based on the characteristic information for the user to learn.

Preferably, in step S3, the step of representing the corresponding note sequence on the track based on the note information includes:

extracting a plurality of note sets in the note information, the note sets including: a plurality of notes, note attributes of the notes, and a production time;

determining animation notes corresponding to the note attributes based on a preset note template library;

obtaining a first type of source of the note set, and representing the animation notes on a position corresponding to the generation time on a track with a second type which is the same as the first type;

when all the notes are expressed, a note sequence is formed on the track, and the expression is finished.

Preferably, in step S4, the teaching instruction is performed on the audio track based on the characteristic information, and the teaching instruction includes:

extracting a plurality of characteristic items in the characteristic information, wherein the plurality of characteristic items comprise: chord information, rhythm information, and key information;

obtaining a preset chord judgment model, inputting chord information into the chord judgment model, and obtaining first judgment information, wherein the first judgment information comprises: at least one first excellent item and a first deficient item;

acquiring a preset rhythm evaluation model, inputting rhythm information into the rhythm evaluation model, and acquiring second evaluation information, wherein the second evaluation information comprises: at least one second excellent term and a second deficient term;

acquiring a preset tonality evaluation model, inputting tonality information into the tonality evaluation model, and acquiring third evaluation information, wherein the third evaluation information comprises: at least one third excellent item and a third deficient item;

and displaying on the audio track based on the first occurrence time of each of the first excellent item, the first lack item, the second excellent item, the second lack item, the third excellent item and the third lack item to finish teaching marking.

Preferably, the interactive learning method based on audio recognition and multi-track sequence representation further includes:

step S5: acquiring a learning record of a user, determining a piece of music suitable for playing the user based on the learning record, and pushing the piece of music to the user;

wherein, the step of determining the proper playing music of the user based on the learning record comprises the following steps:

extracting a plurality of fourth excellent items and fourth defective items in the learning record;

establishing a first time axis, and representing a fourth excellent item and a fourth defective item on the first time axis based on a second occurrence time of the fourth excellent item and the fourth defective item to obtain a second time axis;

acquiring a preset selection direction, and starting to select along the selection direction on a second time axis;

when the fourth excellent item is selected, taking the selected fourth excellent item as a fifth excellent item;

analyzing the fifth excellent item to obtain a first excellent reason and a first excellent value;

selecting a fourth excellent item appearing after the fifth excellent item on the second time axis and taking the fourth excellent item as a sixth excellent item;

analyzing the sixth excellent item to obtain a second excellent reason and a second excellent value;

if the second excel cause is the same as the first priority cause, determining a first distance between the corresponding sixth excel item and the fifth excel item on the second time axis, and calculating a first difference between the corresponding second excel value and the first excel value;

determining a first score corresponding to the first distance and the first difference together based on a preset first score library;

selecting a fourth defect item appearing after the fifth excellent item on the second time axis and taking the fourth defect item as a fifth defect item;

analyzing the fifth defect item to obtain a defect reason and a defect value;

determining whether the first defect reason and the first excellent reason are mutually exclusive based on a semantic understanding technology, if so, determining a second distance between a corresponding fifth defect item and a fifth excellent item on a second time axis, and meanwhile, calculating a second difference value between a corresponding first excellent value and a defect value;

determining a second score corresponding to the second distance and the second difference together based on a preset second score library;

summing the first score and the second score to obtain a sum of scores, and associating the sum with a first characteristic of a fifth excellent item, the first characteristic comprising: one of a first chord characteristic, a first tempo characteristic, and a first key characteristic;

removing a fifth excellent item on a second time axis, a corresponding sixth excellent item with the same second excellent reason and first priority reason, and a corresponding fifth defective item with mutually exclusive first defective reason and first defective reason, continuing to select after removing, and determining the score sum associated with the second characteristic of the next selected fifth excellent item;

sorting the first chord characteristics based on the associated scores and from big to small to obtain a first sequence, and selecting the first n first chord characteristics in the first sequence as second chord characteristics;

sorting the first rhythm characteristics based on the associated scores and from big to small to obtain a second sequence, and selecting the first n first rhythm characteristics in the second sequence as second rhythm characteristics;

sorting the first tonal characteristics based on the associated scores and from the size to obtain a third sequence, and selecting the first n first tonal characteristics in the third sequence as second tonal characteristics;

freely combining the second chord characteristic, the second rhythm characteristic and the second tone characteristic to obtain a plurality of combinations;

determining music corresponding to the combination based on a preset music library;

and summarizing the music to obtain a first music album, and taking the first music album as the music suitable for the user to play to finish the determination.

Preferably, the interactive learning method based on audio recognition and multi-track sequence representation further includes:

step S6: matching a battle player for the user, and after the match is successful, constructing a playing battle space for fighting;

wherein, establish and play the battle space, fight, include:

acquiring a second music set suitable for the player to play;

avoiding opening the first music set and the second music set based on a preset game music library, and determining a plurality of game music;

acquiring a preset virtual space;

determining at least one first virtual instrument corresponding to the match track based on a preset virtual instrument library, arranging the first virtual instrument in a player seat in a virtual space to serve as a second virtual instrument, and arranging the first virtual instrument in a battle player seat in the virtual space to serve as a third virtual instrument;

determining a virtual music score corresponding to a match track based on a preset virtual music score library, and respectively arranging the virtual music score in a user seat and a fighting player seat;

acquiring a first virtual character selected by a user from a preset virtual character table, arranging the first virtual character in a user seat, acquiring a second virtual character selected by a competitor from the virtual character table, and arranging the second virtual character in the competitor seat;

arranging a first virtual camera at a first head of a first virtual character to be in butt joint with first VR glasses of a user, and arranging a second virtual camera at a second head of a second virtual character to be in butt joint with second VR glasses of a competitor;

playing the match tracks in a mute manner, and starting the battle;

acquiring a plurality of first operations input by a user based on first VR operation equipment, operating a second virtual musical instrument based on the first operations, acquiring a plurality of second operations input by a competitor player based on the second VR operation equipment, and operating a third virtual musical instrument based on the second operations;

acquiring a standard note item played by a current music piece in a virtual music score, acquiring a first note item fed back by a second virtual musical instrument in real time, and simultaneously acquiring a second note item fed back by a third virtual musical instrument in real time;

performing conformity analysis on the first note item based on the standard note items to obtain a first conformity, and performing conformity analysis on the second note item to obtain a second conformity;

determining a first distance corresponding to the first conformity based on a preset moving distance library, moving the user seat to the competitor seat by the first distance, simultaneously determining a second distance corresponding to the second conformity, and moving the competitor seat away from the user seat by the second distance;

if the first conformity and the second conformity are both larger than or equal to a preset conformity threshold, triggering a preset special effect item, wherein the special effect item comprises: a special effect material sequence;

acquiring a first lens alignment area of a first virtual camera, and acquiring a second lens alignment area of a second virtual camera;

when a first intersection area exists between the first lens alignment area and the second lens alignment area, displaying the special effect materials in the special effect material sequence in the first intersection area;

when the first lens alignment area and the second lens alignment area have a second intersection area again, displaying a next special effect material in the special effect material sequence in the second intersection area;

and after the game tracks are all played, the battle is finished.

The embodiment of the invention provides an interactive learning system based on audio identification and multi-track sequence representation, which comprises:

the acquisition module is used for acquiring MIDI files generated by playing musical instruments by users;

the analysis module is used for analyzing the MIDI file and extracting note information and characteristic information;

the representation module is used for acquiring a plurality of preset music tracks and representing corresponding note sequences on the music tracks based on the note information;

and the marking module is used for teaching and marking on the audio track based on the characteristic information so as to provide the user for learning.

Preferably, the representation module performs the following operations:

extracting a plurality of note sets in the note information, the note sets including: a plurality of notes, note attributes of the notes, and a production time;

determining animation notes corresponding to the note attributes based on a preset note template library;

obtaining a first type of source of the note set, and representing the animation notes on a position corresponding to the generation time on a track with a second type which is the same as the first type;

when all the notes are expressed, a note sequence is formed on the track, and the expression is finished.

Preferably, the marking module performs the following operations:

extracting a plurality of characteristic items in the characteristic information, wherein the plurality of characteristic items comprise: chord information, rhythm information, and key information;

obtaining a preset chord judgment model, inputting chord information into the chord judgment model, and obtaining first judgment information, wherein the first judgment information comprises: at least one first excellent item and a first deficient item;

acquiring a preset rhythm evaluation model, inputting rhythm information into the rhythm evaluation model, and acquiring second evaluation information, wherein the second evaluation information comprises: at least one second excellent term and a second deficient term;

acquiring a preset tonality evaluation model, inputting tonality information into the tonality evaluation model, and acquiring third evaluation information, wherein the third evaluation information comprises: at least one third excellent item and a third deficient item;

and displaying on the audio track based on the first occurrence time of each of the first excellent item, the first lack item, the second excellent item, the second lack item, the third excellent item and the third lack item to finish teaching marking.

Preferably, the interactive learning system based on audio recognition and multi-track sequence representation further comprises:

the pushing module is used for acquiring the learning record of the user, determining the music suitable for playing of the user based on the learning record and pushing the music to the user;

the push module performs the following operations:

extracting a plurality of fourth excellent items and fourth defective items in the learning record;

establishing a first time axis, and representing a fourth excellent item and a fourth defective item on the first time axis based on a second occurrence time of the fourth excellent item and the fourth defective item to obtain a second time axis;

acquiring a preset selection direction, and starting to select along the selection direction on a second time axis;

when the fourth excellent item is selected, taking the selected fourth excellent item as a fifth excellent item;

analyzing the fifth excellent item to obtain a first excellent reason and a first excellent value;

selecting a fourth excellent item appearing after the fifth excellent item on the second time axis and taking the fourth excellent item as a sixth excellent item;

analyzing the sixth excellent item to obtain a second excellent reason and a second excellent value;

if the second excel cause is the same as the first priority cause, determining a first distance between the corresponding sixth excel item and the fifth excel item on the second time axis, and calculating a first difference between the corresponding second excel value and the first excel value;

determining a first score corresponding to the first distance and the first difference together based on a preset first score library;

selecting a fourth defect item appearing after the fifth excellent item on the second time axis and taking the fourth defect item as a fifth defect item;

analyzing the fifth defect item to obtain a defect reason and a defect value;

determining whether the first defect reason and the first excellent reason are mutually exclusive based on a semantic understanding technology, if so, determining a second distance between a corresponding fifth defect item and a fifth excellent item on a second time axis, and meanwhile, calculating a second difference value between a corresponding first excellent value and a defect value;

determining a second score corresponding to the second distance and the second difference together based on a preset second score library;

summing the first score and the second score to obtain a sum of scores, and associating the sum with a first characteristic of a fifth excellent item, the first characteristic comprising: one of a first chord characteristic, a first tempo characteristic, and a first key characteristic;

removing a fifth excellent item on a second time axis, a corresponding sixth excellent item with the same second excellent reason and first priority reason, and a corresponding fifth defective item with mutually exclusive first defective reason and first defective reason, continuing to select after removing, and determining the score sum associated with the second characteristic of the next selected fifth excellent item;

sorting the first chord characteristics based on the associated scores and from big to small to obtain a first sequence, and selecting the first n first chord characteristics in the first sequence as second chord characteristics;

sorting the first rhythm characteristics based on the associated scores and from big to small to obtain a second sequence, and selecting the first n first rhythm characteristics in the second sequence as second rhythm characteristics;

sorting the first tonal characteristics based on the associated scores and from the size to obtain a third sequence, and selecting the first n first tonal characteristics in the third sequence as second tonal characteristics;

freely combining the second chord characteristic, the second rhythm characteristic and the second tone characteristic to obtain a plurality of combinations;

determining music corresponding to the combination based on a preset music library;

and summarizing the music to obtain a first music album, and taking the first music album as the music suitable for the user to play to finish the determination.

Preferably, the interactive learning system based on audio recognition and multi-track sequence representation further comprises:

the fighting module is used for matching one fighting player for the user, and after the matching is successful, a playing fighting space is constructed for fighting;

wherein, establish and play the battle space, fight, include:

acquiring a second music set suitable for the player to play;

avoiding opening the first music set and the second music set based on a preset game music library, and determining a plurality of game music;

acquiring a preset virtual space;

determining at least one first virtual instrument corresponding to the match track based on a preset virtual instrument library, arranging the first virtual instrument in a player seat in a virtual space to serve as a second virtual instrument, and arranging the first virtual instrument in a battle player seat in the virtual space to serve as a third virtual instrument;

determining a virtual music score corresponding to a match track based on a preset virtual music score library, and respectively arranging the virtual music score in a user seat and a fighting player seat;

acquiring a first virtual character selected by a user from a preset virtual character table, arranging the first virtual character in a user seat, acquiring a second virtual character selected by a competitor from the virtual character table, and arranging the second virtual character in the competitor seat;

arranging a first virtual camera at a first head of a first virtual character to be in butt joint with first VR glasses of a user, and arranging a second virtual camera at a second head of a second virtual character to be in butt joint with second VR glasses of a competitor;

playing the match tracks in a mute manner, and starting the battle;

acquiring a plurality of first operations input by a user based on first VR operation equipment, operating a second virtual musical instrument based on the first operations, acquiring a plurality of second operations input by a competitor player based on the second VR operation equipment, and operating a third virtual musical instrument based on the second operations;

acquiring a standard note item played by a current music piece in a virtual music score, acquiring a first note item fed back by a second virtual musical instrument in real time, and simultaneously acquiring a second note item fed back by a third virtual musical instrument in real time;

performing conformity analysis on the first note item based on the standard note items to obtain a first conformity, and performing conformity analysis on the second note item to obtain a second conformity;

determining a first distance corresponding to the first conformity based on a preset moving distance library, moving the user seat to the competitor seat by the first distance, simultaneously determining a second distance corresponding to the second conformity, and moving the competitor seat away from the user seat by the second distance;

if the first conformity and the second conformity are both larger than or equal to a preset conformity threshold, triggering a preset special effect item, wherein the special effect item comprises: a special effect material sequence;

acquiring a first lens alignment area of a first virtual camera, and acquiring a second lens alignment area of a second virtual camera;

when a first intersection area exists between the first lens alignment area and the second lens alignment area, displaying the special effect materials in the special effect material sequence in the first intersection area;

when the first lens alignment area and the second lens alignment area have a second intersection area again, displaying a next special effect material in the special effect material sequence in the second intersection area;

and after the game tracks are all played, the battle is finished.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:

FIG. 1 is a flowchart of an interactive learning method based on audio recognition and multi-track sequence representation according to an embodiment of the present invention;

FIG. 2 is a flowchart of another interactive learning method based on audio recognition and multi-track sequence representation according to an embodiment of the present invention;

fig. 3 is a schematic diagram of an interactive learning system based on audio recognition and multi-track sequence representation according to an embodiment of the present invention.

Detailed Description

The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.

The embodiment of the invention provides an interactive learning method based on audio identification and multi-track sequence representation, as shown in fig. 1, comprising the following steps:

step S1: acquiring MIDI files generated by playing musical instruments by users;

step S2: analyzing the MIDI file, and extracting note information and characteristic information;

step S3: acquiring a plurality of preset audio tracks, and representing corresponding note sequences on the audio tracks based on note information;

step S4: and performing teaching marking on the audio track based on the characteristic information for the user to learn.

The working principle and the beneficial effects of the technical scheme are as follows:

the user connects the musical instrument equipment to the MIDI interface of the system, and the MIDI file can be obtained when playing the musical instrument; based on the audio recognition technology, the MIDI file is parsed, and note information is extracted [ for example: pitch, duration, dynamics, etc. of notes and characteristic information (rhythm, chord, tonality, etc.); the corresponding note sequence is represented on a preset track, and meanwhile, teaching notation is carried out [ for example: insufficient dynamics, dislocation and the like, a user can play and check the audio track at the same time to perform learning;

according to the embodiment of the invention, a user can play and check the audio track at the same time, playing and learning are carried out, the user does not need to go out, the convenience is high, meanwhile, one-to-one interactive learning is realized, and the learning effect is improved.

An embodiment of the present invention provides an interactive learning method based on audio recognition and multi-track sequence representation, as shown in fig. 2, in step S3, representing a corresponding note sequence on a track based on note information, including:

step S201: extracting a plurality of note sets in the note information, the note sets including: a plurality of notes, note attributes of the notes, and a production time;

step S202: determining animation notes corresponding to the note attributes based on a preset note template library;

step S203: obtaining a first type of source of the note set, and representing the animation notes on a position corresponding to the generation time on a track with a second type which is the same as the first type;

step S204: when all the notes are expressed, a note sequence is formed on the track, and the expression is finished.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset note template library specifically comprises: a database, in which animation notes corresponding to different note attributes (pitch, duration, dynamics, etc.) are stored; labeling the animated notes on the track based on the time of note generation, the type of note source; types of audio tracks are classified into guitar, piano, drum, and the like;

the invention represents the note sequence that the user is playing on the audio track, which is convenient for the user to view.

The embodiment of the invention provides an interactive learning method based on audio identification and multi-track sequence representation, in step S4, teaching marking is performed on a sound track based on characteristic information, which includes:

extracting a plurality of characteristic items in the characteristic information, wherein the plurality of characteristic items comprise: chord information, rhythm information, and key information;

obtaining a preset chord judgment model, inputting chord information into the chord judgment model, and obtaining first judgment information, wherein the first judgment information comprises: at least one first excellent item and a first deficient item;

acquiring a preset rhythm evaluation model, inputting rhythm information into the rhythm evaluation model, and acquiring second evaluation information, wherein the second evaluation information comprises: at least one second excellent term and a second deficient term;

acquiring a preset tonality evaluation model, inputting tonality information into the tonality evaluation model, and acquiring third evaluation information, wherein the third evaluation information comprises: at least one third excellent item and a third deficient item;

and displaying on the audio track based on the first occurrence time of each of the first excellent item, the first lack item, the second excellent item, the second lack item, the third excellent item and the third lack item to finish teaching marking.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset chord judgment model specifically comprises the following steps: using machine learning algorithms for a large number of people [ for example: professional music teachers ] record chord evaluation and generate a model after learning; the preset rhythm judging model and the tone judging model are the same as the above ones, and are not described in detail;

the embodiment of the invention judges chord information, rhythm information and tone information based on a pre-trained model to obtain excellent items (rhythm is controlled to be good, excellent value 88) and defect items (rhythm is disordered and defect value-32), and the excellent items and the defect items are represented on corresponding positions of audio tracks for a user to check and know the successful position and the deficiency of self playing.

The embodiment of the invention provides an interactive learning method based on audio identification and multi-track sequence representation, which further comprises the following steps:

step S5: acquiring a learning record of a user, determining a piece of music suitable for playing the user based on the learning record, and pushing the piece of music to the user;

wherein, the step of determining the proper playing music of the user based on the learning record comprises the following steps:

extracting a plurality of fourth excellent items and fourth defective items in the learning record;

establishing a first time axis, and representing a fourth excellent item and a fourth defective item on the first time axis based on a second occurrence time of the fourth excellent item and the fourth defective item to obtain a second time axis;

acquiring a preset selection direction, and starting to select along the selection direction on a second time axis;

when the fourth excellent item is selected, taking the selected fourth excellent item as a fifth excellent item;

analyzing the fifth excellent item to obtain a first excellent reason and a first excellent value;

selecting a fourth excellent item appearing after the fifth excellent item on the second time axis and taking the fourth excellent item as a sixth excellent item;

analyzing the sixth excellent item to obtain a second excellent reason and a second excellent value;

if the second excel cause is the same as the first priority cause, determining a first distance between the corresponding sixth excel item and the fifth excel item on the second time axis, and calculating a first difference between the corresponding second excel value and the first excel value;

determining a first score corresponding to the first distance and the first difference together based on a preset first score library;

selecting a fourth defect item appearing after the fifth excellent item on the second time axis and taking the fourth defect item as a fifth defect item;

analyzing the fifth defect item to obtain a defect reason and a defect value;

determining whether the first defect reason and the first excellent reason are mutually exclusive based on a semantic understanding technology, if so, determining a second distance between a corresponding fifth defect item and a fifth excellent item on a second time axis, and meanwhile, calculating a second difference value between a corresponding first excellent value and a defect value;

determining a second score corresponding to the second distance and the second difference together based on a preset second score library;

summing the first score and the second score to obtain a sum of scores, and associating the sum with a first characteristic of a fifth excellent item, the first characteristic comprising: one of a first chord characteristic, a first tempo characteristic, and a first key characteristic;

removing a fifth excellent item on a second time axis, a corresponding sixth excellent item with the same second excellent reason and first priority reason, and a corresponding fifth defective item with mutually exclusive first defective reason and first defective reason, continuing to select after removing, and determining the score sum associated with the second characteristic of the next selected fifth excellent item;

sorting the first chord characteristics based on the associated scores and from big to small to obtain a first sequence, and selecting the first n first chord characteristics in the first sequence as second chord characteristics;

sorting the first rhythm characteristics based on the associated scores and from big to small to obtain a second sequence, and selecting the first n first rhythm characteristics in the second sequence as second rhythm characteristics;

sorting the first tonal characteristics based on the associated scores and from the size to obtain a third sequence, and selecting the first n first tonal characteristics in the third sequence as second tonal characteristics;

freely combining the second chord characteristic, the second rhythm characteristic and the second tone characteristic to obtain a plurality of combinations;

determining music corresponding to the combination based on a preset music library;

and summarizing the music to obtain a first music album, and taking the first music album as the music suitable for the user to play to finish the determination.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset selecting direction is specifically as follows: from the start to the end of the time axis; the preset first score library specifically comprises: the database stores first scores corresponding to different distances and difference values together; the preset second score library is the same as the preset second score library; the preset music library specifically comprises: a database, in which music corresponding to different chord, rhythm and tone combinations are stored;

when the music suitable for the user is determined, selecting the fifth excellent item on the second time axis, and analyzing and determining the first excellent reason and the first excellent value; if a sixth excellent item occurs after that, the second excellent cause and the first excellent cause are the same, see if there is a difference between the second excellent value and the first excellent value, i.e. calculate the difference [ for example: subtracting the former from the latter, if the difference is negative, the score will be reduced if the difference is compared before taking, and meanwhile, the distance between the corresponding sixth excellent item and the fifth excellent item is determined (the smaller the distance is, the step is reduced in a short time), and the score will be reduced); if after that, the defect cause of the fifth defect item appears mutually exclusive from the first priority cause [ for example: mutual exclusion is not easily mastered by the cooperative rhythm of the drum and the guitar and the cooperative rhythm of the drum and the guitar, the difference between a defect value and a first excellent value is seen (the difference is larger, the step back is larger, and the score is reduced), and meanwhile, the distance between a fifth defect item and a fifth excellent item is determined (the distance is smaller, the step back is very large in a short time, and the score is reduced); summarizing the first score and the second score to obtain a score sum; ranking and selecting suitable music pieces based on the scores and the sums;

according to the embodiment of the invention, the music suitable for playing by the user is intelligently determined based on the time axis, and is pushed for the user, so that the user experience is improved; meanwhile, a plurality of score libraries are arranged, corresponding scores are determined based on reasonable influence conditions of the distances and the difference corresponding scores, sorting is carried out based on the scores, and the working efficiency of the system is improved.

The embodiment of the invention provides an interactive learning method based on audio identification and multi-track sequence representation, which further comprises the following steps:

allocating a proper online teacher to the user for online teaching;

wherein assigning a suitable online teacher to the user comprises:

extracting the first chord characteristic except the second chord characteristic in the first sequence as a third chord characteristic;

extracting the first rhythm characteristic except the second rhythm characteristic in the second sequence as a third rhythm characteristic;

extracting the first tonal characteristics in the third sequence except the second tonal characteristic as third tonal characteristics;

performing feature extraction on the third chord characteristic, the third chord characteristic and the third key characteristic to obtain a plurality of first features;

acquiring a preset online teacher list, and randomly selecting a first teacher from the online teacher list;

obtaining adequacy information of the first teacher, the adequacy information including: a plurality of first excel items;

acquiring a verification value of the first excel item, and simultaneously extracting the first excel item of which the verification value is greater than a preset verification value threshold from the first excel item as a second excel item;

performing feature extraction on the second excel item to obtain a plurality of second features;

matching the first characteristic and the second characteristic in sequence, and acquiring a third coincidence degree of matching coincidence when matching coincidence is realized each time;

summarizing the third conformity, obtaining a conformity sum, and associating the conformity sum with the corresponding first teacher;

extracting the first teacher corresponding to the conformity and the maximum value, and using the first teacher as a second teacher;

assigning the second teacher to the user;

wherein obtaining a verification value for the first excel item comprises:

acquiring additional information of the first teacher, wherein the additional information comprises: a plurality of acquisition nodes;

obtaining teaching records of the first teacher through the obtaining node, wherein the teaching records comprise: a plurality of entries;

performing feature extraction on the first excel item to obtain a plurality of third features;

determining a plurality of fourth features associated with the third features based on a preset associated feature library;

performing feature extraction on the record items to obtain a plurality of fifth features;

matching the third feature with the fifth feature in sequence, and acquiring a fourth coincidence degree of matching coincidence when matching coincidence is performed each time;

matching the fourth feature with the fifth feature in sequence, and acquiring a fifth matching degree of matching coincidence when matching coincidence is achieved each time;

obtaining the credit degree of the obtaining node, and calculating the verification value of the first excellence item based on the credit degree, the fourth conformity and the fifth conformity, wherein the calculation formula is as follows:

wherein σ is the verification value, n is the total number of the acquisition nodes, F_iIs an intermediate variable, α_i，t，xThe x-th fourth conformity degree, beta, generated when the third characteristic is matched with the fifth characteristic of the t-th record item in the teaching record acquired from the ith acquisition node_i，t，xThe x-th fifth conformity degree, C, generated when the fourth feature associated with the third feature is matched with the fifth feature of the t-th record item in the teaching record acquired from the ith acquisition node_tThe total number of fifth conformity degrees, L, generated when the fourth feature associated with the third feature is matched with the fifth feature of the tth record item in the teaching record acquired from the ith acquisition node_tThe total number of fourth conformity degrees, O, generated when the third characteristic is matched with the fifth characteristic of the t-th record item in the teaching record acquired from the ith acquisition node_iFor the total number of entries in the tutorial record obtained from the ith said acquisition node, J_iCredit, f, for the ith said acquisition node₀Is a preset comparison threshold.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset online teacher list specifically includes: a list comprising a plurality of online teachers; the preset verification value threshold specifically comprises: for example, 96; the preset associated feature library specifically comprises: a database, in which associated features corresponding to different features are stored [ the same characterization meaning is different due to different expression forms, and the extracted features are also different, so that an associated feature library is set to avoid omission ]; the preset comparison value threshold specifically comprises: for example, 50;

when recruiting teachers, requiring the teachers to upload adequacy information, and determining students suitable for teaching based on the adequacy information; however, the authenticity of the adequacy information cannot be verified, so that a teacher is required to upload additional information, the additional information comprises a plurality of nodes (corresponding to a teaching platform), teaching records are obtained through the nodes, and whether the teacher is actually adequacy is judged based on the teaching records (namely, an authentication value is calculated, and the authentication value is greater than or equal to an authentication value threshold value, namely, adequacy);

obtain the credit of each acquisition node [ e.g.: based on whether records of malicious provided records exist in the node or not, verification values are calculated based on the credibility, the condition of feature matching and the condition of associated feature matching, the authenticity of the good item provided by the teacher is judged quickly, the fact that the good item which is finally determined to be suitable for the students is reliable is ensured, the accuracy of distributing suitable online teachers to the students is improved to the maximum extent, and meanwhile, the working efficiency of the system is also improved;

in the formula, the greater the credit, the greater the fourth conformity and the greater the fifth conformity,

is greater than the comparison threshold f₀If the first good item is smaller than the second good item, the authenticity of the first good item is not good, and the value is assigned to be 0.138.

The embodiment of the invention provides an interactive learning method based on audio identification and multi-track sequence representation, which further comprises the following steps:

step S6: matching a battle player for the user, and after the match is successful, constructing a playing battle space for fighting;

wherein, establish and play the battle space, fight, include:

acquiring a second music set suitable for the player to play;

avoiding opening the first music set and the second music set based on a preset game music library, and determining a plurality of game music;

acquiring a preset virtual space;

determining at least one first virtual instrument corresponding to the match track based on a preset virtual instrument library, arranging the first virtual instrument in a player seat in a virtual space to serve as a second virtual instrument, and arranging the first virtual instrument in a battle player seat in the virtual space to serve as a third virtual instrument;

determining a virtual music score corresponding to a match track based on a preset virtual music score library, and respectively arranging the virtual music score in a user seat and a fighting player seat;

acquiring a first virtual character selected by a user from a preset virtual character table, arranging the first virtual character in a user seat, acquiring a second virtual character selected by a competitor from the virtual character table, and arranging the second virtual character in the competitor seat;

arranging a first virtual camera at a first head of a first virtual character to be in butt joint with first VR glasses of a user, and arranging a second virtual camera at a second head of a second virtual character to be in butt joint with second VR glasses of a competitor;

playing the match tracks in a mute manner, and starting the battle;

acquiring a plurality of first operations input by a user based on first VR operation equipment, operating a second virtual musical instrument based on the first operations, acquiring a plurality of second operations input by a competitor player based on the second VR operation equipment, and operating a third virtual musical instrument based on the second operations;

acquiring a standard note item played by a current music piece in a virtual music score, acquiring a first note item fed back by a second virtual musical instrument in real time, and simultaneously acquiring a second note item fed back by a third virtual musical instrument in real time;

performing conformity analysis on the first note item based on the standard note items to obtain a first conformity, and performing conformity analysis on the second note item to obtain a second conformity;

determining a first distance corresponding to the first conformity based on a preset moving distance library, moving the user seat to the competitor seat by the first distance, simultaneously determining a second distance corresponding to the second conformity, and moving the competitor seat away from the user seat by the second distance;

if the first conformity and the second conformity are both larger than or equal to a preset conformity threshold, triggering a preset special effect item, wherein the special effect item comprises: a special effect material sequence;

acquiring a first lens alignment area of a first virtual camera, and acquiring a second lens alignment area of a second virtual camera;

when a first intersection area exists between the first lens alignment area and the second lens alignment area, displaying the special effect materials in the special effect material sequence in the first intersection area;

when the first lens alignment area and the second lens alignment area have a second intersection area again, displaying a next special effect material in the special effect material sequence in the second intersection area;

and after the game tracks are all played, the battle is finished.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset match track library specifically comprises: a database, in which a plurality of match tracks are stored; the preset virtual musical instrument library specifically comprises the following steps: the database stores virtual musical instruments corresponding to different competition tracks; the preset virtual music score library specifically comprises the following steps: a database, in which virtual music scores corresponding to different match tracks are stored; the preset virtual character list specifically comprises the following steps: a list including a plurality of virtual characters; the preset moving distance library specifically comprises the following steps: a database, which stores the corresponding distance of different conformity; the preset conformity threshold specifically includes: for example, 85;

a user wears VR glasses, holds VR operating equipment such as a hand or foot, selects favorite virtual characters, and the system automatically selects competition music to ensure fairness and avoid the strong music of both sides; at the beginning of the game, the note items fed back by the virtual instruments are collected [ for example: pitch, duration, etc., of notes, obtaining corresponding standard notes currently played in the score, and performing conformity analysis [ for example: matching is carried out; the larger the first conformity is, the better the user plays against the music score is, and the user seat is moved to the battle player seat; the larger the second conformity is, the better the fighter is played, and the fighter seat is moved back to the user seat; meanwhile, the larger the conformity is, the larger the moving distance is; if the user performs well and the battle players do not perform well, the user sees that the user is closer to the battle seat in the VR visual angle of the user, the difference between the user and the opponent is clearly identified to be reduced, and the substitution feeling of the user is improved; meanwhile, if the conformity of the two parties is greater than the conformity threshold value, triggering a special effect, only when the two parties look at the same area, triggering the special effect, and then when the two parties look at the same area, triggering the next special effect, so that the user knows that the user and the other party view the same special effect, and the experience is better; because the sequence of the special effect sequences is sequenced according to the optimal viewing effect, the special effect can be played to a great extent.

An embodiment of the present invention provides an interactive learning system based on audio recognition and multi-track sequence representation, as shown in fig. 3, including:

theacquisition module 1 is used for acquiring MIDI files generated by playing musical instruments by users;

theanalysis module 2 is used for analyzing the MIDI file and extracting note information and characteristic information;

therepresentation module 3 is used for acquiring a plurality of preset audio tracks and representing corresponding note sequences on the audio tracks based on note information;

and themarking module 4 is used for teaching and marking on the audio track based on the characteristic information so as to provide the user for learning.

The working principle and the beneficial effects of the technical scheme are as follows:

the user connects the musical instrument equipment to the MIDI interface of the system, and the MIDI file can be obtained when playing the musical instrument; based on the audio recognition technology, the MIDI file is parsed, and note information is extracted [ for example: pitch, duration, dynamics, etc. of notes and characteristic information (rhythm, chord, tonality, etc.); the corresponding note sequence is represented on a preset track, and meanwhile, teaching notation is carried out [ for example: insufficient dynamics, dislocation and the like, a user can play and check the audio track at the same time to perform learning;

according to the embodiment of the invention, a user can play and check the audio track at the same time, playing and learning are carried out, the user does not need to go out, the convenience is high, meanwhile, one-to-one interactive learning is realized, and the learning effect is improved.

The embodiment of the invention provides an interactive learning system based on audio identification and multi-track sequence representation, wherein arepresentation module 3 executes the following operations:

extracting a plurality of note sets in the note information, the note sets including: a plurality of notes, note attributes of the notes, and a production time;

determining animation notes corresponding to the note attributes based on a preset note template library;

obtaining a first type of source of the note set, and representing the animation notes on a position corresponding to the generation time on a track with a second type which is the same as the first type;

when all the notes are expressed, a note sequence is formed on the track, and the expression is finished.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset note template library specifically comprises: a database, in which animation notes corresponding to different note attributes (pitch, duration, dynamics, etc.) are stored; labeling the animated notes on the track based on the time of note generation, the type of note source; types of audio tracks are classified into guitar, piano, drum, and the like;

the invention represents the note sequence that the user is playing on the audio track, which is convenient for the user to view.

The embodiment of the invention provides an interactive learning system based on audio identification and multi-track sequence representation, wherein amarking module 4 executes the following operations:

extracting a plurality of characteristic items in the characteristic information, wherein the plurality of characteristic items comprise: chord information, rhythm information, and key information;

obtaining a preset chord judgment model, inputting chord information into the chord judgment model, and obtaining first judgment information, wherein the first judgment information comprises: at least one first excellent item and a first deficient item;

acquiring a preset rhythm evaluation model, inputting rhythm information into the rhythm evaluation model, and acquiring second evaluation information, wherein the second evaluation information comprises: at least one second excellent term and a second deficient term;

acquiring a preset tonality evaluation model, inputting tonality information into the tonality evaluation model, and acquiring third evaluation information, wherein the third evaluation information comprises: at least one third excellent item and a third deficient item;

and displaying on the audio track based on the first occurrence time of each of the first excellent item, the first lack item, the second excellent item, the second lack item, the third excellent item and the third lack item to finish teaching marking.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset chord judgment model specifically comprises the following steps: using machine learning algorithms for a large number of people [ for example: professional music teachers ] record chord evaluation and generate a model after learning; the preset rhythm judging model and the tone judging model are the same as the above ones, and are not described in detail;

the embodiment of the invention judges chord information, rhythm information and tone information based on a pre-trained model to obtain excellent items (rhythm is controlled to be good, excellent value 88) and defect items (rhythm is disordered and defect value-32), and the excellent items and the defect items are represented on corresponding positions of audio tracks for a user to check and know the successful position and the deficiency of self playing.

The embodiment of the invention provides an interactive learning system based on audio recognition and multi-track sequence representation, which further comprises:

the pushing module is used for acquiring the learning record of the user, determining the music suitable for playing of the user based on the learning record and pushing the music to the user;

the push module performs the following operations:

extracting a plurality of fourth excellent items and fourth defective items in the learning record;

establishing a first time axis, and representing a fourth excellent item and a fourth defective item on the first time axis based on a second occurrence time of the fourth excellent item and the fourth defective item to obtain a second time axis;

acquiring a preset selection direction, and starting to select along the selection direction on a second time axis;

when the fourth excellent item is selected, taking the selected fourth excellent item as a fifth excellent item;

analyzing the fifth excellent item to obtain a first excellent reason and a first excellent value;

selecting a fourth excellent item appearing after the fifth excellent item on the second time axis and taking the fourth excellent item as a sixth excellent item;

analyzing the sixth excellent item to obtain a second excellent reason and a second excellent value;

if the second excel cause is the same as the first priority cause, determining a first distance between the corresponding sixth excel item and the fifth excel item on the second time axis, and calculating a first difference between the corresponding second excel value and the first excel value;

determining a first score corresponding to the first distance and the first difference together based on a preset first score library;

selecting a fourth defect item appearing after the fifth excellent item on the second time axis and taking the fourth defect item as a fifth defect item;

analyzing the fifth defect item to obtain a defect reason and a defect value;

determining whether the first defect reason and the first excellent reason are mutually exclusive based on a semantic understanding technology, if so, determining a second distance between a corresponding fifth defect item and a fifth excellent item on a second time axis, and meanwhile, calculating a second difference value between a corresponding first excellent value and a defect value;

determining a second score corresponding to the second distance and the second difference together based on a preset second score library;

summing the first score and the second score to obtain a sum of scores, and associating the sum with a first characteristic of a fifth excellent item, the first characteristic comprising: one of a first chord characteristic, a first tempo characteristic, and a first key characteristic;

removing a fifth excellent item on a second time axis, a corresponding sixth excellent item with the same second excellent reason and first priority reason, and a corresponding fifth defective item with mutually exclusive first defective reason and first defective reason, continuing to select after removing, and determining the score sum associated with the second characteristic of the next selected fifth excellent item;

sorting the first chord characteristics based on the associated scores and from big to small to obtain a first sequence, and selecting the first n first chord characteristics in the first sequence as second chord characteristics;

sorting the first rhythm characteristics based on the associated scores and from big to small to obtain a second sequence, and selecting the first n first rhythm characteristics in the second sequence as second rhythm characteristics;

sorting the first tonal characteristics based on the associated scores and from the size to obtain a third sequence, and selecting the first n first tonal characteristics in the third sequence as second tonal characteristics;

freely combining the second chord characteristic, the second rhythm characteristic and the second tone characteristic to obtain a plurality of combinations;

determining music corresponding to the combination based on a preset music library;

and summarizing the music to obtain a first music album, and taking the first music album as the music suitable for the user to play to finish the determination.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset selecting direction is specifically as follows: from the start to the end of the time axis; the preset first score library specifically comprises: the database stores first scores corresponding to different distances and difference values together; the preset second score library is the same as the preset second score library; the preset music library specifically comprises: a database, in which music corresponding to different chord, rhythm and tone combinations are stored;

when the music suitable for the user is determined, selecting the fifth excellent item on the second time axis, and analyzing and determining the first excellent reason and the first excellent value; if a sixth excellent item occurs after that, the second excellent cause and the first excellent cause are the same, see if there is a difference between the second excellent value and the first excellent value, i.e. calculate the difference [ for example: subtracting the former from the latter, if the difference is negative, the score will be reduced if the difference is compared before taking, and meanwhile, the distance between the corresponding sixth excellent item and the fifth excellent item is determined (the smaller the distance is, the step is reduced in a short time), and the score will be reduced); if after that, the defect cause of the fifth defect item appears mutually exclusive from the first priority cause [ for example: mutual exclusion is not easily mastered by the cooperative rhythm of the drum and the guitar and the cooperative rhythm of the drum and the guitar, the difference between a defect value and a first excellent value is seen (the difference is larger, the step back is larger, and the score is reduced), and meanwhile, the distance between a fifth defect item and a fifth excellent item is determined (the distance is smaller, the step back is very large in a short time, and the score is reduced); summarizing the first score and the second score to obtain a score sum; ranking and selecting suitable music pieces based on the scores and the sums;

according to the embodiment of the invention, the music suitable for playing by the user is intelligently determined based on the time axis, and is pushed for the user, so that the user experience is improved; meanwhile, a plurality of score libraries are arranged, corresponding scores are determined based on reasonable influence conditions of the distances and the difference corresponding scores, sorting is carried out based on the scores, and the working efficiency of the system is improved.

The embodiment of the invention provides an interactive learning system based on audio recognition and multi-track sequence representation, which further comprises:

the distribution module is used for distributing a proper online teacher for the user to perform online teaching;

the allocation module performs the following operations:

extracting the first chord characteristic except the second chord characteristic in the first sequence as a third chord characteristic;

extracting the first rhythm characteristic except the second rhythm characteristic in the second sequence as a third rhythm characteristic;

extracting the first tonal characteristics in the third sequence except the second tonal characteristic as third tonal characteristics;

performing feature extraction on the third chord characteristic, the third chord characteristic and the third key characteristic to obtain a plurality of first features;

acquiring a preset online teacher list, and randomly selecting a first teacher from the online teacher list;

obtaining adequacy information of the first teacher, the adequacy information including: a plurality of first excel items;

acquiring a verification value of the first excel item, and simultaneously extracting the first excel item of which the verification value is greater than a preset verification value threshold from the first excel item as a second excel item;

performing feature extraction on the second excel item to obtain a plurality of second features;

matching the first characteristic and the second characteristic in sequence, and acquiring a third coincidence degree of matching coincidence when matching coincidence is realized each time;

summarizing the third conformity, obtaining a conformity sum, and associating the conformity sum with the corresponding first teacher;

extracting the first teacher corresponding to the conformity and the maximum value, and using the first teacher as a second teacher;

assigning the second teacher to the user;

wherein obtaining a verification value for the first excel item comprises:

acquiring additional information of the first teacher, wherein the additional information comprises: a plurality of acquisition nodes;

obtaining teaching records of the first teacher through the obtaining node, wherein the teaching records comprise: a plurality of entries;

performing feature extraction on the first excel item to obtain a plurality of third features;

determining a plurality of fourth features associated with the third features based on a preset associated feature library;

performing feature extraction on the record items to obtain a plurality of fifth features;

matching the third feature with the fifth feature in sequence, and acquiring a fourth coincidence degree of matching coincidence when matching coincidence is performed each time;

matching the fourth feature with the fifth feature in sequence, and acquiring a fifth matching degree of matching coincidence when matching coincidence is achieved each time;

obtaining the credit degree of the obtaining node, and calculating the verification value of the first excellence item based on the credit degree, the fourth conformity and the fifth conformity, wherein the calculation formula is as follows:

wherein σ is the verification value, n is the total number of the acquisition nodes, F_iIs an intermediate variable, α_i，t，xThe x-th fourth conformity degree, beta, generated when the third characteristic is matched with the fifth characteristic of the t-th record item in the teaching record acquired from the ith acquisition node_i，t，xThe x-th fifth conformity degree, C, generated when the fourth feature associated with the third feature is matched with the fifth feature of the t-th record item in the teaching record acquired from the ith acquisition node_tThe total number of fifth conformity degrees, L, generated when the fourth feature associated with the third feature is matched with the fifth feature of the tth record item in the teaching record acquired from the ith acquisition node_tThe total number of fourth conformity degrees, O, generated when the third characteristic is matched with the fifth characteristic of the t-th record item in the teaching record acquired from the ith acquisition node_iFor the total number of entries in the tutorial record obtained from the ith said acquisition node, J_iCredit, f, for the ith said acquisition node₀Is a preset comparison threshold.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset online teacher list specifically includes: a list comprising a plurality of online teachers; the preset verification value threshold specifically comprises: for example, 96; the preset associated feature library specifically comprises: a database, in which associated features corresponding to different features are stored [ the same characterization meaning is different due to different expression forms, and the extracted features are also different, so that an associated feature library is set to avoid omission ]; the preset comparison value threshold specifically comprises: for example, 50;

when recruiting teachers, requiring the teachers to upload adequacy information, and determining students suitable for teaching based on the adequacy information; however, the authenticity of the adequacy information cannot be verified, so that a teacher is required to upload additional information, the additional information comprises a plurality of nodes (corresponding to a teaching platform), teaching records are obtained through the nodes, and whether the teacher is actually adequacy is judged based on the teaching records (namely, an authentication value is calculated, and the authentication value is greater than or equal to an authentication value threshold value, namely, adequacy);

obtain the credit of each acquisition node [ e.g.: based on whether records of malicious provided records exist in the node or not, verification values are calculated based on the credibility, the condition of feature matching and the condition of associated feature matching, the authenticity of the good item provided by the teacher is judged quickly, the fact that the good item which is finally determined to be suitable for the students is reliable is ensured, the accuracy of distributing suitable online teachers to the students is improved to the maximum extent, and meanwhile, the working efficiency of the system is also improved;

in the formula, the greater the credit, the greater the fourth conformity and the greater the fifth conformity,

is greater than the comparison threshold f₀If the first good item is smaller than the second good item, the authenticity of the first good item is not good, and the value is assigned to be 0.138.

The embodiment of the invention provides an interactive learning system based on audio recognition and multi-track sequence representation, which further comprises:

the fighting module is used for matching one fighting player for the user, and after the matching is successful, a playing fighting space is constructed for fighting;

wherein, establish and play the battle space, fight, include:

acquiring a second music set suitable for the player to play;

avoiding opening the first music set and the second music set based on a preset game music library, and determining a plurality of game music;

acquiring a preset virtual space;

determining at least one first virtual instrument corresponding to the match track based on a preset virtual instrument library, arranging the first virtual instrument in a player seat in a virtual space to serve as a second virtual instrument, and arranging the first virtual instrument in a battle player seat in the virtual space to serve as a third virtual instrument;

determining a virtual music score corresponding to a match track based on a preset virtual music score library, and respectively arranging the virtual music score in a user seat and a fighting player seat;

acquiring a first virtual character selected by a user from a preset virtual character table, arranging the first virtual character in a user seat, acquiring a second virtual character selected by a competitor from the virtual character table, and arranging the second virtual character in the competitor seat;

arranging a first virtual camera at a first head of a first virtual character to be in butt joint with first VR glasses of a user, and arranging a second virtual camera at a second head of a second virtual character to be in butt joint with second VR glasses of a competitor;

playing the match tracks in a mute manner, and starting the battle;

acquiring a plurality of first operations input by a user based on first VR operation equipment, operating a second virtual musical instrument based on the first operations, acquiring a plurality of second operations input by a competitor player based on the second VR operation equipment, and operating a third virtual musical instrument based on the second operations;

acquiring a standard note item played by a current music piece in a virtual music score, acquiring a first note item fed back by a second virtual musical instrument in real time, and simultaneously acquiring a second note item fed back by a third virtual musical instrument in real time;

performing conformity analysis on the first note item based on the standard note items to obtain a first conformity, and performing conformity analysis on the second note item to obtain a second conformity;

determining a first distance corresponding to the first conformity based on a preset moving distance library, moving the user seat to the competitor seat by the first distance, simultaneously determining a second distance corresponding to the second conformity, and moving the competitor seat away from the user seat by the second distance;

if the first conformity and the second conformity are both larger than or equal to a preset conformity threshold, triggering a preset special effect item, wherein the special effect item comprises: a special effect material sequence;

acquiring a first lens alignment area of a first virtual camera, and acquiring a second lens alignment area of a second virtual camera;

when a first intersection area exists between the first lens alignment area and the second lens alignment area, displaying the special effect materials in the special effect material sequence in the first intersection area;

when the first lens alignment area and the second lens alignment area have a second intersection area again, displaying a next special effect material in the special effect material sequence in the second intersection area;

and after the game tracks are all played, the battle is finished.

The working principle and the beneficial effects of the technical scheme are as follows:

the preset match track library specifically comprises: a database, in which a plurality of match tracks are stored; the preset virtual musical instrument library specifically comprises the following steps: the database stores virtual musical instruments corresponding to different competition tracks; the preset virtual music score library specifically comprises the following steps: a database, in which virtual music scores corresponding to different match tracks are stored; the preset virtual character list specifically comprises the following steps: a list including a plurality of virtual characters; the preset moving distance library specifically comprises the following steps: a database, which stores the corresponding distance of different conformity; the preset conformity threshold specifically includes: for example, 85;

a user wears VR glasses, holds VR operating equipment such as a hand or foot, selects favorite virtual characters, and the system automatically selects competition music to ensure fairness and avoid the strong music of both sides; at the beginning of the game, the note items fed back by the virtual instruments are collected [ for example: pitch, duration, etc., of notes, obtaining corresponding standard notes currently played in the score, and performing conformity analysis [ for example: matching is carried out; the larger the first conformity is, the better the user plays against the music score is, and the user seat is moved to the battle player seat; the larger the second conformity is, the better the fighter is played, and the fighter seat is moved back to the user seat; meanwhile, the larger the conformity is, the larger the moving distance is; if the user performs well and the battle players do not perform well, the user sees that the user is closer to the battle seat in the VR visual angle of the user, the difference between the user and the opponent is clearly identified to be reduced, and the substitution feeling of the user is improved; meanwhile, if the conformity of the two parties is greater than the conformity threshold value, triggering a special effect, only when the two parties look at the same area, triggering the special effect, and then when the two parties look at the same area, triggering the next special effect, so that the user knows that the user and the other party view the same special effect, and the experience is better; because the sequence of the special effect sequences is sequenced according to the optimal viewing effect, the special effect can be played to a great extent.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (10)

1. An interactive learning method based on audio recognition and multi-track sequence representation is characterized by comprising the following steps:

step S1: acquiring MIDI files generated by playing musical instruments by users;

step S2: analyzing the MIDI file, and extracting note information and characteristic information;

step S3: acquiring a plurality of preset audio tracks, and representing corresponding note sequences on the audio tracks based on the note information;

step S4: and performing teaching marking on the audio track based on the characteristic information for the user to learn.

2. The interactive learning method based on audio recognition and multi-track sequence representation as claimed in claim 1, wherein in step S3, representing the corresponding note sequence on the audio track based on the note information comprises:

extracting a plurality of note sets in the note information, the note sets comprising: a plurality of notes, note attributes of the notes, and a production time;

determining animation notes corresponding to the note attributes based on a preset note template library;

obtaining a first type of source of said note set, representing said animated note on a location on said track corresponding to said production time having a second type identical to said first type;

and when the notes are all represented, forming a note sequence on the audio track to finish the representation.

3. The interactive learning method based on audio recognition and multi-track sequence representation as claimed in claim 1, wherein in step S4, performing teaching marking on the audio track based on the characteristic information comprises:

extracting a plurality of characteristic items in the characteristic information, the plurality of characteristic items including: chord information, rhythm information, and key information;

acquiring a preset chord judgment model, inputting the chord information into the chord judgment model, and acquiring first judgment information, wherein the first judgment information comprises: at least one first excellent item and a first deficient item;

acquiring a preset rhythm judging model, inputting the rhythm information into the rhythm judging model, and acquiring second judging information, wherein the second judging information comprises: at least one second excellent term and a second deficient term;

acquiring a preset tonality evaluation model, inputting the tonality information into the tonality evaluation model, and acquiring third evaluation information, wherein the third evaluation information comprises: at least one third excellent item and a third deficient item;

and displaying on the audio track based on the first occurrence time of each of the first excellent item, the first lack item, the second excellent item, the second lack item, the third excellent item and the third lack item, and finishing teaching indication.

4. The interactive learning method based on audio recognition and multi-track sequence representation as claimed in claim 1, further comprising:

step S5: acquiring a learning record of a user, determining a piece of music suitable for playing of the user based on the learning record, and pushing the piece of music to the user;

wherein determining the user's suitable playing melody based on the learning record includes:

extracting a plurality of fourth excellent items and fourth defective items in the learning record;

establishing a first time axis, and representing the fourth excellent item and the fourth defective item on the first time axis based on a second occurrence time of the fourth excellent item and the fourth defective item to obtain a second time axis;

acquiring a preset selection direction, and starting to select along the selection direction on the second time axis;

when the fourth excellent item is selected, taking the selected fourth excellent item as a fifth excellent item;

analyzing the fifth excellent item to obtain a first excellent reason and a first excellent value;

selecting the fourth excellent item appearing after the fifth excellent item on the second time axis as a sixth excellent item;

analyzing the sixth excellent item to obtain a second excellent reason and a second excellent value;

if the second excel cause is the same as the first priority cause, determining a first distance between the corresponding sixth excel item and the fifth excel item on the second time axis, and calculating a first difference between the corresponding second excel value and the corresponding first excel value;

determining a first score corresponding to the first distance and the first difference together based on a preset first score library;

selecting the fourth defect item appearing after the fifth excellent item on the second time axis as a fifth defect item;

analyzing the fifth defect item to obtain a defect reason and a defect value;

determining whether the first defect cause and the first excellent cause are mutually exclusive based on a semantic understanding technology, if so, determining a second distance between the fifth defect item and the fifth excellent item on the second time axis, and meanwhile, calculating a second difference value between the corresponding first excellent value and the defect value;

determining a second score corresponding to the second distance and the second difference together based on a preset second score library;

summing the first score and the second score to obtain a sum of scores and associating with a first characteristic of the fifth excellent item, the first characteristic comprising: one of a first chord characteristic, a first tempo characteristic, and a first key characteristic;

removing the fifth excellent item, the sixth excellent item corresponding to the same second excellent reason and the first priority reason on the second time axis, and the fifth defective item corresponding to the mutually exclusive first defective reason and the first defective reason, and after removing, continuing to select, and determining the score sum associated with the second characteristic of the next selected fifth excellent item;

sorting the first chord characteristics based on the associated scores and from the size to obtain a first sequence, and selecting the first n first chord characteristics in the first sequence as second chord characteristics;

sorting the first rhythm characteristics based on the associated scores and from big to small to obtain a second sequence, and selecting the first n first rhythm characteristics in the second sequence as second rhythm characteristics;

sorting the first tonal characteristics based on the associated scores and from the size to obtain a third sequence, and selecting the first n first tonal characteristics in the third sequence as second tonal characteristics;

freely combining the second chord characteristic, the second rhythm characteristic and the second tone characteristic to obtain a plurality of combinations;

determining a music corresponding to the combination based on a preset music library;

and summarizing the music to obtain a first music album, and taking the first music album as the music suitable for playing of the user to finish the determination.

5. The interactive learning method based on audio recognition and multi-track sequence representation as claimed in claim 4, further comprising:

step S6: matching a battle player for the user, and after the match is successful, constructing a playing battle space for fighting;

wherein, establish and play the battle space, fight, include:

acquiring a second music set suitable for being played by the competitor;

based on a preset game track library, avoiding opening the first music set and the second music set, and determining a plurality of game tracks;

acquiring a preset virtual space;

determining at least one first virtual musical instrument corresponding to the match track based on a preset virtual musical instrument library, arranging the first virtual musical instrument on a player seat in the virtual space to serve as a second virtual musical instrument, and arranging the first virtual musical instrument on a battle player seat in the virtual space to serve as a third virtual musical instrument;

determining a virtual music score corresponding to the match track based on a preset virtual music score library, and respectively arranging the virtual music score in the user seat and the fighting player seat;

acquiring a first virtual character selected by the user from a preset virtual character table, arranging the first virtual character in the user seat, acquiring a second virtual character selected by the competitor from the virtual character table, and arranging the second virtual character in the competitor seat;

arranging a first virtual camera at a first head of the first virtual character to be in butt joint with first VR glasses of the user, and arranging a second virtual camera at a second head of the second virtual character to be in butt joint with second VR glasses of the competitor;

playing the match tracks in a mute manner, and starting the battle;

acquiring a plurality of first operations input by the user based on a first VR operation device, operating the second virtual musical instrument based on the first operations, and simultaneously acquiring a plurality of second operations input by the competitor player based on a second VR operation device, and operating the third virtual musical instrument based on the second operations;

acquiring a standard note item played by the current music in the virtual music score, acquiring a first note item fed back by the second virtual musical instrument in real time, and acquiring a second note item fed back by the third virtual musical instrument in real time;

performing conformity analysis on the first note item based on the standard note item to obtain a first conformity, and performing conformity analysis on the second note item to obtain a second conformity;

determining a first distance corresponding to the first conformity based on a preset moving distance library, moving the user seat to the competitor seat by the first distance, determining a second distance corresponding to the second conformity, and moving the competitor seat away from the user seat by the second distance;

if the first conformity degree and the second conformity degree are both larger than or equal to a preset conformity degree threshold value, triggering a preset special effect item, wherein the special effect item comprises: a special effect material sequence;

acquiring a first lens alignment area of the first virtual camera, and acquiring a second lens alignment area of the second virtual camera;

when a first intersection region exists between the first lens alignment region and the second lens alignment region, displaying the special effect materials in the special effect material sequence in the first intersection region;

when a second intersection region exists in the first lens alignment region and the second lens alignment region again, displaying a next special effect material in the special effect material sequence in the second intersection region;

and after the game tracks are all played, the battle is finished.

6. An interactive learning system based on audio recognition and multi-track sequence representation, comprising:

the acquisition module is used for acquiring MIDI files generated by playing musical instruments by users;

the analysis module is used for analyzing the MIDI file and extracting note information and characteristic information;

the representing module is used for acquiring a plurality of preset music tracks and representing corresponding note sequences on the music tracks based on the note information;

and the marking module is used for performing teaching marking on the audio track based on the characteristic information so as to provide for the learning of a user.

7. The interactive learning system based on audio recognition and multi-track sequence representation of claim 6, wherein the representation module performs the following operations:

extracting a plurality of note sets in the note information, the note sets comprising: a plurality of notes, note attributes of the notes, and a production time;

determining animation notes corresponding to the note attributes based on a preset note template library;

obtaining a first type of source of said note set, representing said animated note on a location on said track corresponding to said production time having a second type identical to said first type;

and when the notes are all represented, forming a note sequence on the audio track to finish the representation.

8. The interactive learning system based on audio recognition and multi-track sequence representation of claim 6, wherein the labeling module performs the following operations:

extracting a plurality of characteristic items in the characteristic information, the plurality of characteristic items including: chord information, rhythm information, and key information;

acquiring a preset chord judgment model, inputting the chord information into the chord judgment model, and acquiring first judgment information, wherein the first judgment information comprises: at least one first excellent item and a first deficient item;

acquiring a preset rhythm judging model, inputting the rhythm information into the rhythm judging model, and acquiring second judging information, wherein the second judging information comprises: at least one second excellent term and a second deficient term;

acquiring a preset tonality evaluation model, inputting the tonality information into the tonality evaluation model, and acquiring third evaluation information, wherein the third evaluation information comprises: at least one third excellent item and a third deficient item;

and displaying on the audio track based on the first occurrence time of each of the first excellent item, the first lack item, the second excellent item, the second lack item, the third excellent item and the third lack item, and finishing teaching indication.

9. The interactive learning system based on audio recognition and multi-track sequence representation of claim 6, further comprising:

the pushing module is used for acquiring the learning record of the user, determining the music suitable for playing of the user based on the learning record and pushing the music to the user;

the push module performs the following operations:

extracting a plurality of fourth excellent items and fourth defective items in the learning record;

establishing a first time axis, and representing the fourth excellent item and the fourth defective item on the first time axis based on a second occurrence time of the fourth excellent item and the fourth defective item to obtain a second time axis;

acquiring a preset selection direction, and starting to select along the selection direction on the second time axis;

when the fourth excellent item is selected, taking the selected fourth excellent item as a fifth excellent item;

analyzing the fifth excellent item to obtain a first excellent reason and a first excellent value;

selecting the fourth excellent item appearing after the fifth excellent item on the second time axis as a sixth excellent item;

analyzing the sixth excellent item to obtain a second excellent reason and a second excellent value;

if the second excel cause is the same as the first priority cause, determining a first distance between the corresponding sixth excel item and the fifth excel item on the second time axis, and calculating a first difference between the corresponding second excel value and the corresponding first excel value;

determining a first score corresponding to the first distance and the first difference together based on a preset first score library;

selecting the fourth defect item appearing after the fifth excellent item on the second time axis as a fifth defect item;

analyzing the fifth defect item to obtain a defect reason and a defect value;

determining whether the first defect cause and the first excellent cause are mutually exclusive based on a semantic understanding technology, if so, determining a second distance between the fifth defect item and the fifth excellent item on the second time axis, and meanwhile, calculating a second difference value between the corresponding first excellent value and the defect value;

determining a second score corresponding to the second distance and the second difference together based on a preset second score library;

summing the first score and the second score to obtain a sum of scores and associating with a first characteristic of the fifth excellent item, the first characteristic comprising: one of a first chord characteristic, a first tempo characteristic, and a first key characteristic;

removing the fifth excellent item, the sixth excellent item corresponding to the same second excellent reason and the first priority reason on the second time axis, and the fifth defective item corresponding to the mutually exclusive first defective reason and the first defective reason, and after removing, continuing to select, and determining the score sum associated with the second characteristic of the next selected fifth excellent item;

sorting the first chord characteristics based on the associated scores and from the size to obtain a first sequence, and selecting the first n first chord characteristics in the first sequence as second chord characteristics;

sorting the first rhythm characteristics based on the associated scores and from big to small to obtain a second sequence, and selecting the first n first rhythm characteristics in the second sequence as second rhythm characteristics;

sorting the first tonal characteristics based on the associated scores and from the size to obtain a third sequence, and selecting the first n first tonal characteristics in the third sequence as second tonal characteristics;

freely combining the second chord characteristic, the second rhythm characteristic and the second tone characteristic to obtain a plurality of combinations;

determining a music corresponding to the combination based on a preset music library;

and summarizing the music to obtain a first music album, and taking the first music album as the music suitable for playing of the user to finish the determination.

10. The interactive learning system based on audio recognition and multi-track sequence representation of claim 6, further comprising:

the fighting module is used for matching one fighting player for the user, and after the matching is successful, a playing fighting space is constructed for fighting;

wherein, establish and play the battle space, fight, include:

acquiring a second music set suitable for being played by the competitor;

based on a preset game track library, avoiding opening the first music set and the second music set, and determining a plurality of game tracks;

acquiring a preset virtual space;

determining at least one first virtual musical instrument corresponding to the match track based on a preset virtual musical instrument library, arranging the first virtual musical instrument on a player seat in the virtual space to serve as a second virtual musical instrument, and arranging the first virtual musical instrument on a battle player seat in the virtual space to serve as a third virtual musical instrument;

determining a virtual music score corresponding to the match track based on a preset virtual music score library, and respectively arranging the virtual music score in the user seat and the fighting player seat;

acquiring a first virtual character selected by the user from a preset virtual character table, arranging the first virtual character in the user seat, acquiring a second virtual character selected by the competitor from the virtual character table, and arranging the second virtual character in the competitor seat;

arranging a first virtual camera at a first head of the first virtual character to be in butt joint with first VR glasses of the user, and arranging a second virtual camera at a second head of the second virtual character to be in butt joint with second VR glasses of the competitor;

playing the match tracks in a mute manner, and starting the battle;

acquiring a plurality of first operations input by the user based on a first VR operation device, operating the second virtual musical instrument based on the first operations, and simultaneously acquiring a plurality of second operations input by the competitor player based on a second VR operation device, and operating the third virtual musical instrument based on the second operations;

acquiring a standard note item played by the current music in the virtual music score, acquiring a first note item fed back by the second virtual musical instrument in real time, and acquiring a second note item fed back by the third virtual musical instrument in real time;

performing conformity analysis on the first note item based on the standard note item to obtain a first conformity, and performing conformity analysis on the second note item to obtain a second conformity;

determining a first distance corresponding to the first conformity based on a preset moving distance library, moving the user seat to the competitor seat by the first distance, determining a second distance corresponding to the second conformity, and moving the competitor seat away from the user seat by the second distance;

if the first conformity degree and the second conformity degree are both larger than or equal to a preset conformity degree threshold value, triggering a preset special effect item, wherein the special effect item comprises: a special effect material sequence;

acquiring a first lens alignment area of the first virtual camera, and acquiring a second lens alignment area of the second virtual camera;

when a first intersection region exists between the first lens alignment region and the second lens alignment region, displaying the special effect materials in the special effect material sequence in the first intersection region;

when a second intersection region exists in the first lens alignment region and the second lens alignment region again, displaying a next special effect material in the special effect material sequence in the second intersection region;

and after the game tracks are all played, the battle is finished.

Images