A kind of method and device thereof of robot identification student's state auxiliary classroom instructionTechnical field
The invention belongs to robot applications to assist classroom in teaching field more particularly to a kind of robot identification student's stateThe method and device thereof of teaching.
Background technique
Classroom instruction behavior refers to behavior expression of the teacher in teaching.Under specific Teaching Circumstances, teacher is according to certainlyThen oneself Specialized Quality selection teaching pattern and oneself role carry out teaching and produce teaching behavior, be mainlyClassroom give lessons people come with oneself feel impart knowledge to students to course.
Robot assistant instruction, which refers to, provides the machine of service as instructional media and tool for the learning aid activity carried outDevice people can play assistant, learn companion, environment or the intelligent equipment for feeling classroom atmosphere.Educational robot is extracurricular as schoolMovable carrier can make extracurricular activities have scientific and interest, while the initiative spirit of energy training student, integrated practiceAbility and collaboration capabilities.And in lesson application, classroom environment can be monitored, teacher is helped to assist student's study, monitor classroomThe also seldom report such as behavior.
The monitoring classroom instruction behavior of reasonable utilization robot can make us more understand Classroom Information, more for teacherThere is critically important reference value in terms of control classroom instruction.So robot has important application valence for classroom instructionValue.
In conclusion problem of the existing technology is：
Existing classroom teaching, class-teaching of teacher cannot understand completely student's state, not using robot toRaw state is first handled after being identified, assists teachers ' teaching；The learning efficiency of student cannot more effectively be improved.
Summary of the invention
In view of the problems of the existing technology, the present invention provides a kind of robot identification student's states to assist classroom instructionMethod and device thereof.
The invention is realized in this way a kind of method of robot identification student's state auxiliary classroom instruction, the machinePeople identify student's state auxiliary classroom instruction method include：
Knowledge accumulation and deep learning are carried out for student's state, teaching behavior, construct student's state behavior knowledge numberAccording to library；
After the characteristic value for extracting each student's state, the characteristic value is subjected to Classification and Identification by SVM classifier, andThe perception state method of weighting is obtained with mathematical statistics method；It identifies to obtain true state weight a% by visual sensor, actIt is b% that sensor, which obtains time of day probability, and eeg sensor identifies to obtain true state weight c%, and hearing transducer obtainsIt is d% to time of day probability；
The multiple sensors of computation vision sensor, action sensor, eeg sensor, hearing transducer obtain true againTotal probability is when perception state：F=m (a%+b%+c%+d%), wherein F is total probability, and m is that multiple sensors perceive shapeThe value of weight when state, m obtains in training study.
Further, teaching behavior knowledge accumulation includes：The classroom that teacher makes under the real-time status of cognition student adjustsDatabase；The classroom adjustment includes the adjustment of classroom rhythm, word speed voice.
Further, the method for robot identification student's state auxiliary classroom instruction further includes：
Student's mood data is perceived out by eeg sensor, visual sensor perceives out student's human face data, movement passesSensor perceives out student's gesture and attitude data, hearing transducer perception student's voice data；
Obtained student's mood data, student's human face data, student's gesture and attitude data, student's voice data are carried outThe state of student is analyzed and is identified in processing；
The data that eeg sensor, visual sensor, action sensor, hearing transducer obtain first are filtered classificationIdentification；Then it obtains each class library, carries out deep learning.
Further, the method for robot identification student's state auxiliary classroom instruction further includes：
Robot carries out the control program that analysis decision goes out teaching behavior to the state of student, and the control program includes shapeState knowledge accumulation, teaching behavior knowledge accumulation, feedback conduct programming, human-computer interaction.
Further, the method for robot identification student's state auxiliary classroom instruction further includes：
Robot is gone to realize corresponding instruction by corresponding mechanism after obtaining analysis decision；Corresponding instruction includes religionTeacher had proposed the repetition of knowledge, had been exported outward by voice, display shows knowledge, points out problem to teacher.
Another object of the present invention is to provide a kind of robot identification student's states to assist classroom instruction device, including：
Student's state aware module, for perceiving out student's mood data by eeg sensor, visual sensor perceivesStudent's human face data, action sensor perceive out student's gesture and attitude data, hearing transducer perception student's voice data out；
Student's state recognition module, the data for obtaining to student's state aware module carry out processing analysis and identifyThe state of student；The data that each sensor obtains first are filtered Classification and Identification, then obtain each class library, are carried out deepDegree study；The obtained data include that visual sensor show that human face expression characteristic value, eeg sensor obtain emotional characteristicsValue；
It makes decisions on one's own module, carries out the control program that analysis decision goes out teaching behavior for state of the robot to student,The control program includes State Knowledge accumulation, teaching behavior knowledge accumulation, feedback conduct programming, human-computer interaction；
Deep learning module, for carrying out knowledge accumulation and machine deep learning, building for student's state, teaching behaviorStudent's state behavior knowledge data base out；
State recognition training study is carried out after the characteristic value for extracting each student's state；Characteristic value is passed through SVM pointsClass device carries out Classification and Identification, and obtains the perception state method of weighting with mathematical statistics method；It identifies to obtain by visual sensor trueReal state weight a%, it is b% that action sensor, which obtains time of day probability, and eeg sensor identifies to obtain true stateWeight c%, it is d% that hearing transducer, which obtains time of day probability,；
The multiple sensors of computation vision sensor, action sensor, eeg sensor, hearing transducer obtain the sense of reality againKnow that probability total when state is：F=m (a%+b%+c%+d%), wherein F is total probability, and m is that multiple sensors perceive stateWhen weight, the value of m obtains in training study；Teaching behavior knowledge accumulation includes teacher under the real-time status of cognition studentThe database for the classroom adjustment made, specifically there is classroom rhythm, the adjustment of word speed voice；
Executing agency is gone to realize corresponding instruction by corresponding mechanism after obtaining analysis decision for robot；Including religionTeacher had proposed the repetition of knowledge, had been exported outward by voice, display shows knowledge, points out problem to teacher.
Further, robot identification student's state auxiliary classroom instruction device includes：
CCD camera constantly takes pictures and to form acquisition student's human face expression, movement visual signal input is passed to inter-processIn device, visual information is obtained；
377A02 sound transducer constantly acquires student's voice signal of attending class and is input in robot processor, obtains soundSound signal；
WISEIMA attitude transducer constantly acquires sitting posture of student state on stool, is input in internal processor, obtainsTo sitting posture signal；
Brain dateline helmet, input EEG signals, which are input in processor, obtains mood signal；
The state of processor, comprehensive analysis student makes decisions；
Display shows final recognition result；
Further, robot identification student's state auxiliary classroom instruction device further includes：
Loudspeaker when current internal state behavior knowledge library can be handled, carry out repeating class offerings；When its internal state behaviorWhen knowledge base cannot be handled, is shown by display and prompt to be handled by teacher.
Another object of the present invention is to provide a kind of computer readable storage mediums, including instruction, when it is in computerWhen upper operation, so that computer executes above-mentioned method.
Advantages of the present invention and good effect are：
The invention proposes a set of monitored using robot student state assisted teacher classroom instruction method, helpTeacher understands student's state, and the efficiency of classroom instruction can be improved.
The present invention solves student's state out that single-sensor cannot accurately identify, and utilizes perception number using multiple sensorsMore accurate student's state can be obtained according to method of weighting analysis student's status data.
Detailed description of the invention
Fig. 1 is the method flow diagram of robot identification student's state auxiliary classroom instruction provided in an embodiment of the present invention.
Fig. 2 is the schematic diagram of student's state aware module provided in an embodiment of the present invention.
Fig. 3 is the schematic diagram of device of robot identification student's state auxiliary classroom instruction provided in an embodiment of the present invention.
Fig. 4 is the schematic device of robot identification student's state auxiliary classroom instruction provided in an embodiment of the present invention.
In figure：1, CCD camera；2,377A02 sound transducer；3, loudspeaker；4, WISEIMA attitude transducer；5, brain electricityThe helmet；6, processor；7, display.
Fig. 5 is robot identification student's state auxiliary classroom instruction device block diagram provided in an embodiment of the present invention.
In figure：8, student's state aware module；9, student's state recognition module；10, it makes decisions on one's own module；11, depthPractise module；12, executing agency.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to embodiments, to the present inventionIt is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not used toLimit the present invention.
With reference to the accompanying drawing and specific embodiment is further described application principle of the invention.
As depicted in figs. 1 and 2, the side of robot identification student's state auxiliary classroom instruction provided in an embodiment of the present inventionMethod, including：
Student's mood data is perceived out by eeg sensor, visual sensor perceives out student's human face data, movement passesSensor perceives out student's gesture and attitude data, hearing transducer perception student's voice data；
Obtained student's mood data, student's human face data, student's gesture and attitude data, student's voice data are carried outThe state of student is analyzed and is identified in processing；
The data that eeg sensor, visual sensor, action sensor, hearing transducer obtain first are filtered classificationIdentification；Then it obtains each class library, carries out deep learning.
Knowledge accumulation and deep learning are carried out for student's state, teaching behavior, construct student's state behavior knowledge numberAccording to library；
After the characteristic value for extracting each student's state, the characteristic value is subjected to Classification and Identification by SVM classifier, andThe perception state method of weighting is obtained with mathematical statistics method；It identifies to obtain true state weight a% by visual sensor, actIt is b% that sensor, which obtains time of day probability, and eeg sensor identifies to obtain true state weight c%, and hearing transducer obtainsIt is d% to time of day probability；
The multiple sensors of computation vision sensor, action sensor, eeg sensor, hearing transducer obtain true againTotal probability is when perception state：F=m (a%+b%+c%+d%), wherein F is total probability, and m is that multiple sensors perceive shapeThe value of weight when state, m obtains in training study.
Teaching behavior knowledge accumulation includes：The data for the classroom adjustment that teacher makes under the real-time status of cognition studentLibrary；The classroom adjustment includes the adjustment of classroom rhythm, word speed voice.
Robot carries out the control program that analysis decision goes out teaching behavior to the state of student, and the control program includes shapeState knowledge accumulation, teaching behavior knowledge accumulation, feedback conduct programming, human-computer interaction.
Robot is gone to realize corresponding instruction by corresponding mechanism after obtaining analysis decision；Corresponding instruction includes religionTeacher had proposed the repetition of knowledge, had been exported outward by voice, display shows knowledge, points out problem to teacher.
Fig. 3, Fig. 4, robot identification student's state provided in an embodiment of the present invention assist classroom instruction device, including：
CCD camera (1) constantly takes pictures to form acquisition student's human face expression, and the visual signals input such as movement is passed to insideIn processor (6), visual information is obtained；377A02 sound transducer (2) constantly acquisition student voice signal of attending class is input to machineIn device people processor (6), voice signal is obtained；WISEIMA attitude transducer (4) constantly acquires sitting posture of student shape on stoolState is input in internal processor (6), obtains sitting posture signal, while brain dateline helmet (5) input EEG signals are input to processor(6) mood signal is obtained in, is made decisions by the state of processor (6) comprehensive analysis student, is shown finally by display (7)Recognition result；When current internal state behavior knowledge library can be handled, such as repetition class offerings can transfer to executing agency's loudspeaker (3)It completes；Transferring to display (7) to show when its internal state behavior knowledge library cannot be handled transfers to teacher to handle.
Such as Fig. 5, robot assisted classroom instruction behavior device provided in an embodiment of the present invention, mainly by student's state awareThe composition such as module 8, student's state recognition module 9, module of making decisions on one's own 10, deep learning module 11, executing agency 12.
Student's state aware module 8, which refers to, obtains the data of student's state by a series of sensors；It is specificRefer to that eeg sensor perceives out student's mood data, visual sensor perceives out student's human face data, action sensor perceptionStudent's gesture and attitude data, hearing transducer perceive student's voice data out.
Student's state recognition module 9, which refers to, carries out processing analysis in the data obtained to student's state aware moduleAnd identify the state of student.The data that each sensor obtains first are filtered Classification and Identification, as visual sensor obtainsHuman face expression characteristic value is identified, eeg sensor obtains emotional characteristics value identification etc., is then obtained each class library, is carried outDeep learning.
The module 10 of making decisions on one's own refers to that robot carries out the control that analysis decision goes out teaching behavior to the state of studentScheme processed specifically includes State Knowledge accumulation, teaching behavior knowledge accumulation, feedback conduct programming, human-computer interaction etc..
The deep learning module 11 refers to the engineering for the accumulation of student's State Knowledge, teaching behavior knowledge accumulationIt practises；Construct state behavior knowledge data base.State recognition training study be after the characteristic value for extracting each state, will be specialValue indicative carries out Classification and Identification by SVM classifier, and obtains the perception state method of weighting with mathematical statistics method, is such as passed by visionSensor identifies to obtain true state weight a%, and it is b%, eeg sensor identification that action sensor, which obtains time of day probability,True state weight c% is obtained, it is d% that hearing transducer, which obtains time of day probability, then obtains in the multiple sensors of calculatingTotal probability is when true perception state：F=m (a%+b%+c%+d%), wherein F is total probability, and m is multiple sensor sensesKnow that weight when state, the value of m can obtain in training study.Teaching behavior knowledge accumulation refers to teacher in the reality of cognition studentWhen state under the database of classroom adjustment made, the adjustment such as classroom rhythm, the problems such as word speed voice.
The executing agency 12 refers to be gone to realize corresponding instruction by corresponding mechanism after robot obtains analysis decision.It if teacher had proposed the repetition of knowledge, is exported outward by voice, display shows knowledge, while pointing out to ask to teacherTopic.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or any combination thereof realIt is existing.When using entirely or partly realizing in the form of a computer program product, the computer program product include one orMultiple computer instructions.When loading on computers or executing the computer program instructions, entirely or partly generate according toProcess described in the embodiment of the present invention or function.The computer can be general purpose computer, special purpose computer, computer networkNetwork or other programmable devices.The computer instruction may be stored in a computer readable storage medium, or from oneComputer readable storage medium is transmitted to another computer readable storage medium, for example, the computer instruction can be from oneA web-site, computer, server or data center pass through wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)Or wireless (such as infrared, wireless, microwave etc.) mode is carried out to another web-site, computer, server or data centerTransmission).The computer-readable storage medium can be any usable medium or include one that computer can accessThe data storage devices such as a or multiple usable mediums integrated server, data center.The usable medium can be magnetic JieMatter, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or semiconductor medium (such as solid state hard disk SolidState Disk (SSD)) etc..
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the inventionMade any modifications, equivalent replacements, and improvements etc., should all be included in the protection scope of the present invention within mind and principle.