CN112331305A - Computer-assisted autism child emotion social rehabilitation training system - Google Patents
Computer-assisted autism child emotion social rehabilitation training systemDownload PDFInfo
- Publication number
- CN112331305A CN112331305ACN202011189221.XACN202011189221ACN112331305ACN 112331305 ACN112331305 ACN 112331305ACN 202011189221 ACN202011189221 ACN 202011189221ACN 112331305 ACN112331305 ACN 112331305A
- Authority
- CN
- China
- Prior art keywords
- training
- pictures
- expression
- interface
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Developmental Disabilities (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Child & Adolescent Psychology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Hospice & Palliative Care (AREA)
- Psychiatry (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Translated fromChinese
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a computer-assisted autism emotion social rehabilitation training system.
Background
Autistic children have different degrees of emotional social disorders, which are more serious in specific social life scenes due to lack of sight, lack of sight with people, difficulty in judging facial expressions of other people or expressing self emotion, difficulty in judging identities of people, incapability of remembering people, and the like.
However, at present, computer-aided training systems for autistic children mainly aim at training cognitive competence or expression recognition of autistic children, and an emotional social competence training system for autistic children is still lacking, so that the computer-aided training systems can assist autistic children to gradually perform sight attention, expression recognition and expression, identity recognition and memory with social characters as core characteristics, and emotional social complex competence training combined with daily life scenes, improve emotional social competence, and solve the problem of insufficient functions of the traditional computer-aided training systems.
Disclosure of Invention
Aiming at the defects in the prior art, the computer-assisted autism child emotion social rehabilitation training system provided by the invention can assist autism children in performing sight line, expression, identity cognition and social scene emotion training which take social characters as core characteristics, and can improve the emotional social ability of autism children and improve the functions of the computer-assisted training system through a layer-by-layer progressive training mode.
A computer-assisted autism child emotion social rehabilitation training system is characterized by comprising an identity cognition training module, a sight line training module, an expression training module and a social scene emotion training module, and realizing the assistant training of emotion social rehabilitation of autism children;
preferably, the training modules are connected in series to form a layer-by-layer progressive rehabilitation training mode of 'character perception-sight communication-expression communication-situation communication' facing the autistic children emotional social disorder; the training modules can be used independently to perform independent distinguishing training of identity cognition, sight, expression and social scene emotional ability on the autistic children;
preferably, the sight line direction of the autism child is acquired and recognized through the sight line processing device, the facial expression of the child is acquired and recognized through the facial expression processing device, the voice of the child is acquired and recognized through the voice recognition processing device, and the interface action of the child is acquired and analyzed through the touch behavior processing device;
preferably, the acquired multi-channel feedback training results of sight line, expression, voice, interface action and the like of the autism child are used for guiding the setting of a training module, training content, training difficulty and training frequency which are accepted by the child next step, and the autism child emotion social rehabilitation training method and device which take social characters as core characteristics and emotion social ability as a training target are gradually formed, so that the emotion social ability of the autism child is comprehensively improved.
Preferably, the identity awareness training module comprises:
identity matching: training the identity matching ability of the autism children or training the identity characteristic cognition ability of the autism children based on the identity cognition training questions;
identity memory: training the ability of the autistic children to remember the identities of other people based on identity cognitive training questions;
based on the completion time and accuracy of the training task of the autistic child in the identity matching or identity memorizing module (the accuracy is the ratio of the number of questions answered by the child in the identity matching or identity memorizing module to the total number of the questions), the training module (identity matching or identity memorizing) which the child receives next step, the training content (for example, the presentation material of the questions), the training difficulty (for example, three difficulties of low, medium and high), and the training frequency (for example, 2 times per day) are determined.
Preferably, the sight line training module specifically includes:
and (3) visual line identification: training the ability of the autistic children to judge the sight direction of other people based on sight training questions, or training the autistic children to judge the ability of the other people to see the eyes of the children;
and (3) visual line output: training the autistic children to control the sight attention of the autistic children to the object or the character based on the sight training questions;
based on the training task completion time and the accuracy (the accuracy is the ratio of the number of questions answered by the child in the sight line recognition or sight line output module to the total number of the questions) of the autistic child in the sight line recognition or sight line output module, the training module (the sight line recognition or sight line output) accepted by the child next step, the training content (for example, the presentation material of the questions), the training difficulty (for example, three difficulties of low, medium and high), and the training frequency (for example, 2 times per day) are determined.
Preferably, the expression training module comprises:
and (3) expression recognition: training the ability of the autistic child to interpret the facial expressions of the other person or different parts of the face based on the expression training subjects, or training the ability of the autistic child to remember the facial expressions of the other person;
expression simulation: training the expression imitating and expressing ability of the autism child based on the expression training questions;
based on the completion time and the accuracy (the accuracy is the ratio of the number of questions answered by the child in the expression recognition or expression simulation module to the total number of questions) of the autistic child in the expression recognition or expression simulation module, the training module (expression recognition or expression simulation) accepted by the child next step, the training content (for example: presentation material of the questions), the training difficulty (for example: three difficulties of low, medium and high), and the training frequency (for example: 2 times per day) are determined.
Preferably, the social scene emotion training module specifically includes:
training comprehensive understanding and application capacity of sight line, expression and identity of the autistic child in a social situation based on the social scene emotion training questions, and determining comprehensive training content (for example: presentation material of the questions) and training frequency (for example: 2 times per day) accepted by the child next step based on completion time and accuracy (accuracy is a ratio of number of questions answered by the child in the comprehensive training task of the social scene emotion training to total number of questions) of the autistic child in a comprehensive training task of the social scene emotion training module.
Preferably, the layer-by-layer progressive rehabilitation training mode is characterized in that the identity recognition module trains the character perception ability of the autistic children, sight training and expression training are performed on the basis of character perception, the sight communication and expression communication ability of the autistic children is improved, comprehensive training of emotional social ability is performed by combining specific social scenes on the basis of sight and expression training, and finally the layer-by-layer progressive rehabilitation training mode of character perception, sight communication, expression communication and situation communication is formed.
Preferably, the autism children sight line processing device captures the sight line direction of the children through a desktop type or glasses type camera arranged in front of the children, and judges that the children control the sight line attention of the children to objects or people according to the sight line direction of the children with autism.
Preferably, the facial expression processing device for the autism child captures the facial expression of the child through a camera arranged in front of the child, identifies the facial expression, and judges the facial expression imitating and expressing capability of the autism child according to the identification result.
Preferably, the autism child touch behavior processing device obtains an input behavior action in an interactive interface operated and controlled by a child through an input device to obtain a training result of the autism child.
Preferably, the method and apparatus for emotional social rehabilitation training of autistic children comprise a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are connected to each other, the memory is used for storing a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions and execute the following steps:
setting a plurality of training subjects, wherein the training subjects are used for carrying out identity cognition training, sight line training, expression training or scene emotion training on autistic children; each training question comprises at least one picture and an operation instruction;
sequentially displaying pictures of the selected training questions through an interactive interface, and playing or displaying corresponding operation instructions;
receiving a multi-channel training feedback result of the autism child, wherein the multi-channel training feedback result comprises the sight line, the expression, the voice and the interface action of the autism child;
and according to the training results of all modules, guiding to set the next training module, training content, training difficulty and training frequency of the children.
Preferably, the identity cognition training questions comprise a first identity matching question, a second identity matching question, an identity matching question and an identity memorizing question;
the pictures in the first identity matching questions comprise a plurality of pairs of matching pictures arranged on the interactive interface, and each pair of matching pictures consists of two identical figure pictures; the operation instruction corresponding to the first identity matching question is used for indicating the autistic children to select the same figure pictures for matching until all the figure pictures on the interactive interface are matched;
the pictures in the second identity matching questions comprise a plurality of pairs of matching pictures arranged on the interactive interface, and each pair of matching pictures comprises two figure pictures with the same identity information and different collocation information; the operation instruction corresponding to the second identity matching question is used for indicating the autistic children to select the figure pictures with the same identity information for matching until all the figure pictures on the interactive interface are matched;
the pictures in the identity matching questions comprise target pictures and a plurality of picture fragments which are arranged on the interactive interface, wherein the plurality of picture fragments can be spliced into pictures with the same identity information of people in the target pictures; the operation instruction corresponding to the identity matching question is used for indicating the autistic children to select picture fragments and splicing the picture with the same identity information of the characters in the target picture;
the identity memorizing questions comprise a first interface and a second interface which are sequentially displayed according to a preset time interval, pictures in the first interface comprise target pictures arranged on the interactive interface, and pictures in the second interface comprise a plurality of character pictures arranged on the interactive interface; wherein the identity information of different figure pictures is different; the method comprises the steps that a person picture is identical to a target picture, or identity information of the person picture is identical to identity information in the target picture; and the operation instruction corresponding to the identity memorizing questions is used for indicating the autistic children to select the character picture which is the same as the target picture or the identity information in the target picture in the second interface.
Preferably, the sight line training questions comprise a first sight line identification question, a second sight line identification question and a sight line output question;
the pictures in the first sight line identification topic comprise a person picture and a plurality of article pictures which are arranged at different positions on the interactive interface, wherein the person picture indicates the sight line direction of a person, and one article picture is arranged in the sight line direction of the person picture; the operation instruction corresponding to the first sight line identification question is used for indicating the autistic child to select an article picture arranged in the sight line direction of the character in the character picture;
the pictures in the second sight line identification topic comprise a plurality of character pictures arranged on the interactive interface, and the sight line directions of characters in different character pictures are different; the operation instruction corresponding to the second sight line identification question is used for indicating the autistic children to select the figure picture in the designated figure sight line direction;
pictures in the sight line output title comprise a plurality of article pictures or figure pictures arranged on the interactive interface; and the operation instruction corresponding to the sight line output subject is used for indicating the autistic children to look at the designated article picture or the figure picture on the interactive interface.
Preferably, the expression training questions comprise a first expression recognition question, a second expression recognition question, a third expression recognition question and an expression imitation question;
the pictures in the first expression recognition topic comprise a plurality of pairs of matching pictures arranged on the interactive interface, and each pair of matching pictures consists of two figure pictures with the same expression; the operation instruction corresponding to the first expression identification question is used for indicating the autistic children to select the figure pictures with the same expression for matching until all the figure pictures on the interactive interface are matched;
the pictures in the second expression recognition topic comprise a target picture and a plurality of picture fragments which are arranged on the interactive interface, wherein the target picture can be spliced by the plurality of picture fragments; the operation instruction corresponding to the second expression recognition question is used for indicating the autistic children to select picture fragments and splicing the picture fragments into the target picture;
the third emotion recognition topic comprises a first interface and a second interface which are sequentially displayed according to a preset time interval, wherein the picture in the first interface comprises a target picture arranged on the interactive interface, and the picture in the second interface comprises a plurality of character pictures arranged on the interactive interface; the expressions of different character pictures in the second interface are different, and the expression of one character picture in the second interface is the same as the expression in the target picture; the operation instruction corresponding to the third emotion recognition question is used for indicating the autistic child to select a character picture with the same expression as that of the target picture in the second interface;
the pictures in the expression imitation questions comprise target pictures arranged on the interactive interface; and the operation instruction corresponding to the expression recognition question is used for indicating the autistic child to make the character expression in the target picture.
Preferably, the social scene emotion training topic comprises a scene interface and a plurality of question interfaces which are sequentially displayed according to a preset time interval;
the scene interface comprises at least one scene picture or scene video arranged on the interactive interface; the operation instruction corresponding to the problem interface is used for instructing the autistic children to perform expression training and identity cognitive training according to the scene interface;
the problem interface comprises an identity recognition interface, an expression judgment interface and an expression simulation interface;
the pictures in the identity recognition interface comprise at least one figure picture arranged on the interactive interface, the identity information of different figure pictures is different, and an operation instruction corresponding to the identity recognition interface is used for indicating the autistic children to select the figure picture with the designated identity information in the scene interface;
the pictures in the expression recognition interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression recognition interface is used for indicating that the autistic children select the expression or expression change of the designated character in the scene interface under the designated event in the scene interface;
the pictures in the expression judgment interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression recognition interface is used for indicating that the autistic children select the pictures as the expressions or expression changes of the designated characters in the scene interface under the designated event in the scene interface;
the operation instruction corresponding to the expression simulation interface is used for indicating the autism child to make a specified expression; or the pictures in the expression simulation interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression simulation interface is used for indicating the autistic children to make the expressions of the character pictures.
According to the technical scheme, the autistic child emotion social rehabilitation training system provided by the invention can gradually train the emotional social ability of the autistic child layer by integrating sight line training, expression training, identity cognition training and social scene emotion training which take social characters as core characteristics, and gradually form a set of autistic child social rehabilitation training method and system, so that the emotional social ability of the autistic child is comprehensively improved.
Drawings
Fig. 1 is a flow chart of an emotional social rehabilitation training method for autistic children.
FIG. 2 is a diagram of an autistic child emotional social rehabilitation training module and mode.
FIG. 3 is a block diagram of an emotional social rehabilitation training method and device for autistic children.
Fig. 4 is a flowchart of an embodiment of identity awareness training.
Fig. 5 is a flow chart of an embodiment of gaze training.
Fig. 6 is a flowchart of an expression training embodiment.
FIG. 7 is a flowchart of an embodiment of scene emotion training.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby. It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
The first embodiment is as follows:
a computer-aided autistic children emotional social rehabilitation training method, referring to fig. 1, comprising the following steps:
s1: the method comprises the steps that a plurality of training modules are arranged, each training module is composed of training subjects with different difficulty levels, the training subjects are used for carrying out emotional social rehabilitation training on autistic children, and each training subject comprises at least one picture and an operation instruction;
s2: displaying pictures and operation instructions in the training questions to the autistic children;
s3: receiving training feedback results of the autistic children, including vision, expressions, voice and interface actions of the autistic children;
s4: and guiding to set a training module, training content and training difficulty of the next step of the child according to a training result.
Referring to fig. 2, the emotional social rehabilitation training module includes an identity cognition training module, a sight line training module, an expression training module, and a social scene emotion training module. The identity cognition training module comprises simple identity cognition and generalized identity cognition, the sight training module comprises sight recognition and sight output, the expression training module comprises expression recognition and expression simulation, and the scene module emotion training comprises single picture scene training, comic scene training and three-dimensional animation scene training. The training takes social characters as core characteristics and aims to improve the emotional social ability of the autistic children. The identity recognition module trains the character perception ability of the autism child, visual line training and expression training are performed on the basis of character perception, the visual line communication and expression communication ability of the autism child is improved, comprehensive training of emotional social ability is performed by combining specific social scenes on the basis of visual line and expression training, and finally a layer-by-layer progressive rehabilitation training mode of character perception, visual line communication, expression communication and situation communication is formed.
The identity recognition training module comprises: identity matching: training the identity matching capability of the autism children, or training the identity feature cognitive capability of the autism children; identity memory: training the ability of the autistic children to remember the identities of other people.
The sight line training module specifically comprises: and (3) visual line identification: training the ability of the autistic children to judge the sight direction of other people, or training the autistic children to judge the ability of the other people to see the eyes of the autistic children; and (3) visual line output: training the autistic children to control the visual attention of the children to the object or the character.
The expression training module comprises: and (3) expression recognition: training the ability of the autistic child to interpret facial expressions of other people or different parts of the face, or training the ability of the autistic child to remember facial expressions of other people; expression simulation: training the expression imitation and expression ability of the autistic children.
The scene emotion training module specifically comprises: training the cognitive emotional complex ability of the autistic child in a specific situation.
The difficulty level of the training questions is determined by picture selection, operation instructions and training actions;
the autistic child emotion social rehabilitation training method can guide the design and development of an autistic child emotion social rehabilitation training system by integrating sight training, expression training, identity cognition training and social scene emotion training which take social characters as core characteristics, and gradually forms a set of autistic child social rehabilitation training method and device.
Example two:
a computer-aided autistic children emotion social rehabilitation training method and device, referring to fig. 3, comprises a processor 801, an input device 802, anoutput device 803 and amemory 804, wherein the processor 801, the input device 802, theoutput device 803 and thememory 804 are connected with each other through abus 805.
It should be understood that in the embodiment of the present invention, the input device may include a camera or a video camera (the user captures facial features including the sight line direction and facial expression of the autistic child), a mouse keyboard or a touch pad (for acquiring input action in the interactive interface manipulated by the child), a fingerprint sensor (for acquiring fingerprint information of the user and direction information of the fingerprint), a microphone, etc., and the output device may include a display (LCD, etc.), a speaker, etc.
The processor referred to may be a CPU central processing unit, but may also be other general purpose processors, DSP digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include both read-only memory and random access memory, and provides instructions and data to the processor. The portion of memory may also include non-volatile random access memory. For example, the memory may also store device type information.
Wherein thememory 804 is configured to store a computer program comprising program instructions, and the processor 801 is configured to call the program instructions to perform the following steps:
setting a plurality of training subjects, wherein the training subjects are used for performing sight training, expression training, identity cognition training or scene emotion training on autistic children; each training question comprises at least one picture and an operation instruction;
specifically, different training contents correspond to different training subjects, and the operation instruction is used for reminding the autistic children how to complete training through characters or voice.
Sequentially displaying pictures of the selected training questions through an interactive interface, and playing or displaying corresponding operation instructions;
specifically, the management user can preset different training subjects for the autism children according to the illness states of the different autism children, and the management user and the autism children can also select the training subjects for training. The interactive interface displays preset or selected training subjects in sequence to complete training.
Receiving a training action of an autistic child;
in particular, the training action may be a click action, a drag action, etc. of the autistic child in the trading interface. Or information collected by other devices on the autism emotional social rehabilitation training system, for example, facial information of the autism child is collected by a camera, voice information of the autism child is collected by a microphone, and the like.
And obtaining the training result of the autistic child according to the pictures, the operation instructions and the training actions in the training subjects.
Specifically, the training results include completion time and accuracy.
Example three:
third embodiment, based on the above embodiments, identity recognition training is performed. The training subjects of the identity recognition training comprise identity matching, identity spelling and identity memory, and refer to fig. 4.
The pictures in the identity matching questions comprise a plurality of pairs of matching pictures arranged on the interactive interface, and each pair of matching pictures consists of two or more identical figure pictures; or each pair of matched pictures comprises two figure pictures with the same identity information and different collocation information; and the operation instruction corresponding to the identity matching subject is used for indicating the autistic children to select the same figure pictures for matching until all the figure pictures on the interactive interface are matched.
The pictures in the identity spelling topics comprise target pictures and a plurality of picture fragments which are arranged on the interactive interface, wherein the plurality of picture fragments can be pieced into pictures with the same identity information of people in the target pictures; and the operation instruction corresponding to the identity spelling topic is used for indicating the autistic children to select the picture fragments and spelling the picture with the same identity information of the characters in the target picture.
The identity memorizing questions comprise a first interface and a second interface which are sequentially displayed according to a preset time interval, pictures in the first interface comprise target pictures arranged on the interactive interface, and pictures in the second interface comprise a plurality of character pictures arranged on the interactive interface; wherein the identity information of different figure pictures is different; the method comprises the steps that a person picture is identical to a target picture, or identity information of the person picture is identical to identity information in the target picture; and the operation instruction corresponding to the identity memorizing questions is used for indicating the autistic children to select the character picture which is the same as the target picture or the identity information in the target picture in the second interface.
Example four:
in the fourth embodiment, the sight line training is performed on the basis of the third embodiment. The training subjects for gaze training include gaze recognition and gaze output, see fig. 5.
The pictures in the sight line identification title comprise a person picture and a plurality of article pictures which are arranged at different positions on the interactive interface, wherein the person picture indicates the sight line direction of the person, and one article picture is arranged in the sight line direction of the person picture; and the operation instruction corresponding to the sight line identification subject is used for instructing the autistic child to select the article picture arranged in the sight line direction of the character in the character picture. The pictures in the sight line identification questions further comprise a plurality of character pictures arranged on the interactive interface, and the sight line directions of characters in different character pictures are different; the operation instruction corresponding to the sight line identification question is used for indicating the autistic children to select the figure picture in the designated figure sight line direction;
the sight line output subject presents a plurality of article pictures or figure pictures for the autistic children at the same time, the article pictures or the figure pictures are placed at different positions in the interactive interface, and the operation instruction indicates the autistic children to look at the specified article pictures or figure pictures on the interactive interface. During interaction, the sight direction of the autistic children is acquired through a preset camera in the system, so that whether the autistic children complete sight output tasks according to requirements or not is automatically judged. The sight line output questions can adjust the difficulty level of training by placing different numbers of object pictures or figure pictures at different positions in the interactive interface, and the autistic children can control the sight line attention of the objects or the figures to train. The training is completed by arranging a plurality of article pictures on the interactive interface, and the character display 'please see the appointed pictures according to the description'. The training is completed by arranging a plurality of character pictures on the interactive interface, and the character display 'please look at the appointed pictures according to the description'.
Example five:
fifth embodiment on the basis of the above embodiments, expression training is performed. The training subjects for expression training include expression recognition and expression simulation, see fig. 6.
The pictures in the expression recognition questions comprise a plurality of pairs of matched pictures arranged on the interactive interface, and each pair of matched pictures consists of two figure pictures with the same expression; the operation instruction corresponding to the expression identification question is used for indicating the autistic children to select the figure pictures with the same expression for matching until all the figure pictures on the interactive interface are matched; the pictures in the expression recognition questions further comprise target pictures and a plurality of picture fragments which are arranged on the interactive interface, wherein the target pictures can be spliced by the plurality of picture fragments; and the operation instruction corresponding to the expression recognition question is used for indicating the autistic children to select the picture fragments and splicing the picture fragments into the target picture. The expression recognition question further comprises a first interface and a second interface which are sequentially displayed according to a preset time interval, wherein the picture in the first interface comprises a target picture arranged on the interactive interface, and the picture in the second interface comprises a plurality of character pictures arranged on the interactive interface; the expressions of different character pictures in the second interface are different, and the expression of one character picture in the second interface is the same as the expression in the target picture; and the operation instruction corresponding to the third emotion recognition topic is used for indicating the autistic child to select the character picture with the same expression as that in the target picture in the second interface.
The pictures in the expression imitation questions comprise target pictures arranged on the interactive interface; and the operation instruction corresponding to the expression recognition question is used for indicating the autistic child to make the character expression in the target picture. Specifically, the expression modeling theme is obtained by presenting a picture of facial expressions of a person, i.e., a target picture, to an autistic child. The interactive interface gives expression imitating instructions through voice and characters. When the autism children use, the same expression as the target picture needs to be made according to the expression simulation instruction, the expression of the autism children is captured in real time through the camera, and whether the autism children make the correct expression or not is judged. The expression simulation question can adjust the difficulty of expression simulation by controlling the exposure time of facial expression, and trains the facial exaggerated expression simulation and expression ability of the autistic children. The 'please imitate the expression in the drawing' is displayed through the interactive interface, and the 'exaggeration expression imitation' and the 'natural expression imitation' are included.
Example six:
sixth embodiment scene emotion training is performed on the basis of the above embodiments. The training subjects for scene emotion training include single picture scene training, comic scene training, and three-dimensional animation scene training, as shown in fig. 7.
The training questions comprise a scene interface and a plurality of question interfaces which are sequentially displayed according to a preset time interval; the scene interface comprises at least one scene picture or scene video arranged on the interactive interface; and the operation instruction corresponding to the problem interface is used for instructing the autistic child to perform expression training and identity cognitive training according to the scene interface.
Specifically, the training subjects of the scene emotion training present one or more scene pictures for explaining daily life or a section of three-dimensional animation to the autistic children in a multi-channel mode. The scene interface comprises a plurality of characters (cartoon or real people) and rich, situational and various character emotions of the characters. The training subjects of the scene emotion training take a certain person in a scene interface as a core, the identity cognitive ability of the autism children is trained firstly, and then the emotional ability of the autism children is trained. When the autistic children use, the information transmitted by the scene interface needs to be learned, events, characters and corresponding character emotions in the scene interface need to be learned, and then training of the problem interface is completed according to the operation instruction. The training subjects of the scene emotion training can adjust the training difficulty level by setting the information amount in the scene interface and the task difficulty of the problem interface, and train the cognitive emotion comprehensive capacity of the autistic children in the situation presented by a single picture.
Preferably, the question interface comprises an identity recognition interface, an expression judgment interface and an expression imitation interface;
the pictures in the identity recognition interface comprise at least one figure picture arranged on the interactive interface, the identity information of different figure pictures is different, and an operation instruction corresponding to the identity recognition interface is used for indicating the autistic children to select the figure picture with the designated identity information in the scene interface;
the pictures in the expression recognition interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression recognition interface is used for indicating that the autistic children select the expression or expression change of the designated character in the scene interface under the designated event in the scene interface;
the operation instruction corresponding to the expression imitation interface is used for indicating the autism child to make a specified expression; or the pictures in the expression simulation interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression simulation interface is used for indicating the autistic children to make the expressions of the character pictures.
The scene interface comprises multimedia materials, and the materials can be a scene picture, can also be a comic displayed by a plurality of scene pictures, and can also be a section of three-dimensional animation. And sets a number of training questions based on the material content. Because the vision, expression, identity cognition and other abilities need to be applied and exerted in real life, the scene emotion training module presents an interactive situation reflecting daily life for the autistic children in a single scene picture mode, a comic mode and a three-dimensional animation mode respectively, and trains the comprehensive abilities of vision, expression and identity cognition in the interactive situation.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (5)
1. A computer-assisted autism child emotion social rehabilitation training system is characterized by comprising an identity cognition training module, a sight line training module, an expression training module, a social scene emotion training module and a multi-channel feedback unit, wherein the identity cognition training module, the sight line training module, the expression training module, the social scene emotion training module and the multi-channel feedback unit are arranged in the system, and the system comprises a computer, a computer
The multi-channel feedback unit comprises a sight line processing device, a facial expression processing device, a voice recognition processing device and a touch behavior processing device; the sight line processing device is used for acquiring and identifying the sight line direction of the autism child; the facial expression processing device is used for acquiring and identifying the facial expression of the autism child; the voice recognition processing device is used for acquiring and recognizing the voice of the autistic children; the touch behavior processing device is used for acquiring and analyzing the interface actions of the autistic children on the interactive interface;
the identity cognition training module is used for training autistic children based on set identity cognition training questions, acquiring identity cognition feedback information of the autistic children during training by utilizing the multi-channel feedback unit, calculating the completion time and accuracy of a training task based on the identity cognition feedback information, and determining the training content, the training difficulty and the training frequency of the sight line training module;
the sight training module is used for training autistic children based on set sight training subjects, collecting sight feedback information of the autistic children during training by using the multi-channel feedback unit, calculating training task completion time and accuracy based on the sight feedback information, and determining training content, training difficulty and training frequency of the expression training module;
the expression training module is used for training the autism children based on set expression training questions, acquiring expression feedback information of the autism children during training by using the multi-channel feedback unit, calculating the completion time and accuracy of a training task based on the expression feedback information, and determining the training content, the training difficulty and the training frequency of the social scene emotion training module; the social scene emotion training module is used for training autistic children based on set social scene emotion training questions, collecting sight, expression and identity feedback information of the autistic children during training by using the multi-channel feedback unit, calculating training task completion time and accuracy based on the feedback information, and determining comprehensive training content and training frequency.
2. The system of claim 1, wherein the identity-aware training topics include a first identity matching topic, a second identity matching topic, an identity matching topic, and an identity memory topic; the pictures in the first identity matching questions comprise a plurality of pairs of matching pictures arranged on the interactive interface, and each pair of matching pictures consists of two identical figure pictures; the operation instruction corresponding to the first identity matching question is used for indicating the autistic children to select the same figure pictures for matching until all the figure pictures on the interactive interface are matched; the pictures in the second identity matching questions comprise a plurality of pairs of matching pictures arranged on the interactive interface, and each pair of matching pictures comprises two figure pictures with the same identity information and different collocation information; the operation instruction corresponding to the second identity matching question is used for indicating the autistic children to select the figure pictures with the same identity information for matching until all the figure pictures on the interactive interface are matched; the pictures in the identity matching questions comprise target pictures and a plurality of picture fragments which are arranged on the interactive interface, wherein the plurality of picture fragments can be spliced into pictures with the same identity information of people in the target pictures; the operation instruction corresponding to the identity matching question is used for indicating the autistic children to select picture fragments and splicing the picture with the same identity information of the characters in the target picture; the identity memorizing questions comprise a first interface and a second interface which are sequentially displayed according to a preset time interval, pictures in the first interface comprise target pictures arranged on the interactive interface, and pictures in the second interface comprise a plurality of character pictures arranged on the interactive interface; the identity information of different character pictures is different, and one character picture is identical to the target picture, or the identity information of one character picture is identical to the identity information in the target picture; and the operation instruction corresponding to the identity memorizing questions is used for indicating the autistic children to select the character picture which is the same as the target picture or the identity information in the target picture in the second interface.
3. The system of claim 1, wherein the gaze training topic comprises a first gaze identification topic, a second gaze identification topic, and a gaze output topic; the pictures in the first sight line identification topic comprise a person picture and a plurality of article pictures which are arranged at different positions on the interactive interface, wherein the person picture indicates the sight line direction of a person, and one article picture is arranged in the sight line direction of the person picture; the operation instruction corresponding to the first sight line identification question is used for indicating the autistic child to select an article picture arranged in the sight line direction of the character in the character picture; the pictures in the second sight line identification topic comprise a plurality of character pictures arranged on the interactive interface, and the sight line directions of characters in different character pictures are different; the operation instruction corresponding to the second sight line identification question is used for indicating the autistic children to select the figure picture in the designated figure sight line direction; pictures in the sight line output title comprise a plurality of article pictures or figure pictures arranged on the interactive interface; and the operation instruction corresponding to the sight line output subject is used for indicating the autistic children to look at the designated article picture or the figure picture on the interactive interface.
4. The system of claim 1, wherein the expression training questions comprise a first expression recognition question, a second expression recognition question, a third expression recognition question, and an expression imitation question; the pictures in the first expression recognition topic comprise a plurality of pairs of matching pictures arranged on the interactive interface, and each pair of matching pictures consists of two figure pictures with the same expression; the operation instruction corresponding to the first expression identification question is used for indicating the autistic children to select the figure pictures with the same expression for matching until all the figure pictures on the interactive interface are matched; the pictures in the second expression recognition topic comprise a target picture and a plurality of picture fragments which are arranged on the interactive interface, wherein the target picture can be spliced by the plurality of picture fragments; the operation instruction corresponding to the second expression recognition question is used for indicating the autistic children to select picture fragments and splicing the picture fragments into the target picture; the third emotion recognition topic comprises a first interface and a second interface which are sequentially displayed according to a preset time interval, wherein the picture in the first interface comprises a target picture arranged on the interactive interface, and the picture in the second interface comprises a plurality of character pictures arranged on the interactive interface; the expressions of different character pictures in the second interface are different, and the expression of one character picture in the second interface is the same as the expression in the target picture; the operation instruction corresponding to the third emotion recognition question is used for indicating the autistic child to select a character picture with the same expression as that of the target picture in the second interface; the pictures in the expression imitation questions comprise target pictures arranged on the interactive interface; and the operation instruction corresponding to the expression recognition question is used for indicating the autistic child to make the character expression in the target picture.
5. The system of claim 1, wherein the social scene emotion training questions comprise a scene interface and a plurality of question interfaces that are displayed in sequence at preset time intervals; the scene interface comprises at least one scene picture or scene video arranged on the interactive interface; the operation instruction corresponding to the problem interface is used for instructing the autistic children to perform expression training and identity cognitive training according to the scene interface; the problem interface comprises an identity recognition interface, an expression judgment interface and an expression simulation interface; the pictures in the identity recognition interface comprise at least one figure picture arranged on the interactive interface, the identity information of different figure pictures is different, and an operation instruction corresponding to the identity recognition interface is used for indicating the autistic children to select the figure picture with the designated identity information in the scene interface; the pictures in the expression recognition interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression recognition interface is used for indicating that the autistic children select the expression or expression change of the designated character in the scene interface under the designated event in the scene interface; the pictures in the expression judgment interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression recognition interface is used for indicating that the autistic children select the pictures as the expressions or expression changes of the designated characters in the scene interface under the designated event in the scene interface; the operation instruction corresponding to the expression simulation interface is used for indicating the autism child to make a specified expression; or the pictures in the expression simulation interface comprise at least one character picture arranged on the interactive interface, the expressions of different character pictures are different, and the operation instruction corresponding to the expression simulation interface is used for indicating the autistic children to make the expressions of the character pictures.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011189221.XACN112331305A (en) | 2020-10-30 | 2020-10-30 | Computer-assisted autism child emotion social rehabilitation training system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011189221.XACN112331305A (en) | 2020-10-30 | 2020-10-30 | Computer-assisted autism child emotion social rehabilitation training system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112331305Atrue CN112331305A (en) | 2021-02-05 |
Family
ID=74297373
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011189221.XAPendingCN112331305A (en) | 2020-10-30 | 2020-10-30 | Computer-assisted autism child emotion social rehabilitation training system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112331305A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096805A (en)* | 2021-04-12 | 2021-07-09 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN114464297A (en)* | 2022-01-27 | 2022-05-10 | 杭州行熠科技有限公司 | A recommended method, system, equipment and medium for autism rehabilitation training subjects |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108899081A (en)* | 2018-06-14 | 2018-11-27 | 北京科技大学 | A kind of man-machine interactive system towards self-closing disease recovering aid |
| CN110866434A (en)* | 2019-08-07 | 2020-03-06 | 上海市精神卫生中心（上海市心理咨询培训中心） | Facial expression recognition training method and system for autism patient |
| CN110890140A (en)* | 2019-11-25 | 2020-03-17 | 上海交通大学 | Virtual reality-based autism rehabilitation training and capability assessment system and method |
- 2020
- 2020-10-30CNCN202011189221.XApatent/CN112331305A/enactivePending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108899081A (en)* | 2018-06-14 | 2018-11-27 | 北京科技大学 | A kind of man-machine interactive system towards self-closing disease recovering aid |
| CN110866434A (en)* | 2019-08-07 | 2020-03-06 | 上海市精神卫生中心（上海市心理咨询培训中心） | Facial expression recognition training method and system for autism patient |
| CN110890140A (en)* | 2019-11-25 | 2020-03-17 | 上海交通大学 | Virtual reality-based autism rehabilitation training and capability assessment system and method |
Non-Patent Citations (3)
| Title |
|---|
| 卜凡帅: "自闭症幼儿图画书社会信息注意特征及教育干预研究", 《中国博士学位论文全文数据库 社会科学Ⅱ辑》* |
| 恩启特教平台: "4个好方法，让自闭症孩子把"配对"用进生活里", 《HTTPS://WWW.SOHU.COM/A/132160627_648638》* |
| 陈顺森: "自闭症幼儿面孔加工特点的眼动研究：社会认知缺陷指标的探索", 《中国博士学位论文全文数据库 哲学与人文科学辑》* |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096805A (en)* | 2021-04-12 | 2021-07-09 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN113096805B (en)* | 2021-04-12 | 2024-02-13 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN114464297A (en)* | 2022-01-27 | 2022-05-10 | 杭州行熠科技有限公司 | A recommended method, system, equipment and medium for autism rehabilitation training subjects |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Behera et al. | Associating facial expressions and upper-body gestures with learning tasks for enhancing intelligent tutoring systems | |
| Seymour et al. | Actors, avatars and agents: Potentials and implications of natural face technology for the creation of realistic visual presence | |
| Yin | [Retracted] An Artificial Intelligent Virtual Reality Interactive Model for Distance Education | |
| WO2010093780A2 (en) | Communication and skills training using interactive virtual humans | |
| US20220198952A1 (en) | Assessment and training system | |
| CN117854341A (en) | General rule-based doctor-patient communication training assessment method based on virtual reality technology | |
| Halim et al. | Individualized virtual reality for increasing self-compassion: evaluation study | |
| CN112331305A (en) | Computer-assisted autism child emotion social rehabilitation training system | |
| Faisal et al. | Towards a reference model for sensor-supported learning systems | |
| Cinieri et al. | Eye tracking and speech driven human-avatar emotion-based communication | |
| Vihriӓlӓ et al. | Possibilities of intelligent textiles in AAC–perspectives of speech and language therapists | |
| Refoua et al. | The Next Frontier in Mindreading? Assessing Generative Artificial Intelligence (GAI)’s Social-Cognitive Capabilities using Dynamic Audiovisual Stimuli | |
| Ren et al. | Avatar-Based Picture Exchange Communication System Enhancing Joint Attention Training for Children With Autism | |
| CN115768535A (en) | Systems and methods for accessible computer-user interaction | |
| JP7440889B2 (en) | Learning support systems and programs | |
| DiMicco | Changing small group interaction through visual reflections of social behavior | |
| Pedro et al. | Towards higher sense of presence: a 3D virtual environment adaptable to confusion and engagement | |
| Utami et al. | A Brief Study of The Use of Pattern Recognition in Online Learning: Recommendation for Assessing Teaching Skills Automatically Online Based | |
| John et al. | Assistive robotics: Adaptive multimodal interaction improving people with communication disorders | |
| Jin et al. | An Investigation of In-Service Teachers' Perceptions and Development of Computational Thinking Skills in a Graduate Emerging Technologies Course. | |
| Jost et al. | Interactions in Multisensory Experiences: Toward a New Timeline Metaphor | |
| Laskowitz et al. | Between Reality and Fiction: Self-Representation as an Avatar and Its Effects on Self-Presence | |
| Gordon | Keeping it real: The potential for accessible mixed-reality usability testing environments | |
| Axelrod et al. | ‘Acted Reality’in Electronic Patient Record Research: A Bridge between Laboratory and Ethnographic Studies | |
| Kavitha et al. | Contemporary Trends and Usage of Augmented Reality Tools for Autistic Children |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20210205 | |
| RJ01 | Rejection of invention patent application after publication |