Summary of the invention
The purpose of the application is to solve the deficiencies in the prior art, provides a kind of original painting auxiliary neural network based colouring sideMethod and device can obtain the technical effect painted to the original painting line original text of input from dynamic auxiliary.
To achieve the goals above, the following technical solution is employed by the application:
Firstly, the application proposes a kind of original painting auxiliary painting methods neural network based.Method includes the following steps:
S100 the original painting line original text and original painting colouring original text of multiple original paintings colouring front and back) are obtained, and marks each subdivision area of original paintingThe colouring parameter in domain forms original painting data sample；
S200) using Keras initialization colouring neural network model, wherein colouring neural network model uses VGG model；
S300 it) imports original painting data sample and is used as training set, neural network model exercises supervision study to painting；
S400) the colouring neural network model finished using training carries out automatic colouring to the original painting line original text of input.
Further, in the above method of the application, the step S100 further includes following sub-step:
S101 the original painting line original text and original painting colouring original text of each original painting colouring front and back) are matched, and on original painting line original text and original paintingColor original text is registrated；
S102) original painting divide based on original painting line original text and form prime area；
S103) based on hue threshold, saturation degree threshold value and luminance threshold, to prime area carry out region segmentation, formedDivided area；
S104 tone, saturation degree and the average value of brightness in same divided area) are calculated, and by tone average value, saturationAverage value, average brightness and adjacent area number are spent as colouring parameter.
Still further, in the above method of the application, the colouring parameter at least further include on original painting line original text withThe adjacent original painting lines in divided area.
Further, in the above method of the application, original painting data sample is classified as according to the Color Style of original paintingMultiple sub- training sets, and corresponding multiple colouring neural network models are formed based on the sub- training set.
Further, in the above method of the application, the step S200 further includes following sub-step:
S201) using Keras establish and initialize VGG model convolutional layer and MAXpooling layers；
S202 evaluate to VGG model is configured) with real-time testing model training performance.
Still further, the step S300 further includes following sub-step in the above method of the application:
S301 the training set being made of original painting data sample) is imported into convolutional layer；
S302 it) is exercised supervision study using SqueezeNet convolutional neural networks to training set；
S303 the weight parameter of classifier in convolutional layer and MAXpooling layers) is assessed based on evaluate, and when detectionDeconditioning when weight parameter into classifier restrains.
Further, in the above method of the application, the step S400 includes following sub-step:
S401 the colouring neural network model that training finishes) is arranged in network server, and configures colouring neural networkThe Data entries of model；
S402 original painting) is uploaded into colouring neural network model by Data entries to execute automatic colouring.
Still further, the Data entries are the forms of webpage in the above method of the application.
Secondly, disclosed herein as well is a kind of original paintings neural network based to assist coloring means.Described device can wrapIt includes with lower module: obtaining module, for obtaining the original painting line original text and original painting colouring original text of multiple original paintings colouring front and back, and mark original paintingThe colouring parameter of each divided area forms original painting data sample；Initialization module, for using Keras initialization colouring mindThrough network model, wherein colouring neural network model uses VGG model；Training module, for importing original painting data sample conductTraining set exercises supervision study to colouring neural network model；Colouring module, the colouring neural network for being finished using trainingModel carries out automatic colouring to the original painting line original text of input.
Further, in the above-mentioned apparatus of the application, the acquisition module may include following submodule: matching mouldBlock, for matching the original painting line original text and original painting colouring original text of each original painting colouring front and back, and to original painting line original text and original painting paint original text intoRow registration；First division module forms prime area for divide to original painting based on original painting line original text；Second division module,For based on hue threshold, saturation degree threshold value and luminance threshold, to prime area carry out region segmentation, formed divided area；Mark module, for calculating tone in same divided area, saturation degree and the average value of brightness, and by tone average value, saturationAverage value, average brightness and adjacent area number are spent as colouring parameter.
Still further, in the above-mentioned apparatus of the application, the colouring parameter at least further include on original painting line original text withThe adjacent original painting lines in divided area.
Further, in the above-mentioned apparatus of the application, original painting data sample is classified as according to the Color Style of original paintingMultiple sub- training sets, and corresponding multiple colouring neural network models are formed based on the sub- training set.
Further, in the above-mentioned apparatus of the application, the initialization module can also include following submodule: establishModule, for established and initialized using Keras VGG model convolutional layer and MAXpooling layers；Configuration module, for configuringEvaluate to VGG model is with real-time testing model training performance.
Still further, the training module can also include following submodule in the above-mentioned apparatus of the application: importingModule, for the training set being made of original painting data sample to be imported into convolutional layer；Execution module, for using SqueezeNetConvolutional neural networks exercise supervision study to training set；Evaluation module, for based on evaluate assessment convolutional layer andThe weight parameter of classifier in MAXpooling layers, and the deconditioning when detecting the weight parameter convergence in classifier.
Further, in the above-mentioned apparatus of the application, the colouring module can also include following submodule: arrangement mouldBlock for the colouring neural network model finished will to be trained to be arranged in network server, and configures colouring neural network modelData entries；Uploading module, for original painting to be uploaded to colouring neural network model by Data entries to execute automatic colouring.
Still further, the Data entries are the forms of webpage in the above-mentioned apparatus of the application.
Finally, the application also proposes a kind of computer readable storage medium, it is stored thereon with computer instruction.Above-metioned instructionWhen being executed by processor, following steps are executed:
S100 the original painting line original text and original painting colouring original text of multiple original paintings colouring front and back) are obtained, and marks each subdivision area of original paintingThe colouring parameter in domain forms original painting data sample；
S200) using Keras initialization colouring neural network model, wherein colouring neural network model uses VGG model；
S300 it) imports original painting data sample and is used as training set, neural network model exercises supervision study to painting；
S400) the colouring neural network model finished using training carries out automatic colouring to the original painting line original text of input.
Further, when processor executes above-metioned instruction, the step S100 further includes following sub-step:
S101 the original painting line original text and original painting colouring original text of each original painting colouring front and back) are matched, and on original painting line original text and original paintingColor original text is registrated；
S102) original painting divide based on original painting line original text and form prime area；
S103) based on hue threshold, saturation degree threshold value and luminance threshold, to prime area carry out region segmentation, formedDivided area；
S104 tone, saturation degree and the average value of brightness in same divided area) are calculated, and by tone average value, saturationAverage value, average brightness and adjacent area number are spent as colouring parameter.
Still further, the colouring parameter at least further includes on original painting line original text when processor executes above-metioned instructionThe original painting lines adjacent with divided area.
Further, when processor executes above-metioned instruction, original painting data sample is classified according to the Color Style of original paintingFor multiple sub- training sets, and corresponding multiple colouring neural network models are formed based on the sub- training set.
Further, when processor executes above-metioned instruction, the step S200 further includes following sub-step:
S201) using Keras establish and initialize VGG model convolutional layer and MAXpooling layers；
S202 evaluate to VGG model is configured) with real-time testing model training performance.
Still further, the step S300 further includes following sub-step when processor executes above-metioned instruction:
S301 the training set being made of original painting data sample) is imported into convolutional layer；
S302 it) is exercised supervision study using SqueezeNet convolutional neural networks to training set；
S303 the weight parameter of classifier in convolutional layer and MAXpooling layers) is assessed based on evaluate, and when detectionDeconditioning when weight parameter into classifier restrains.
Further, when processor executes above-metioned instruction, the step S400 includes following sub-step:
S401 the colouring neural network model that training finishes) is arranged in network server, and configures colouring neural networkThe Data entries of model；
S402 original painting) is uploaded into colouring neural network model by Data entries to execute automatic colouring.
Still further, the Data entries are the forms of webpage when processor executes above-metioned instruction.
Having the beneficial effect that for the application carries out auxiliary colouring to original painting using neural network, to alleviate original painting colouringHeavy coloring work in the process conveniently and efficiently can be made and be modified to original painting.
Specific embodiment
It is carried out below with reference to technical effect of the embodiment and attached drawing to the design of the application, specific structure and generation clearChu, complete description, to be completely understood by the purpose, scheme and effect of the application.It should be noted that the case where not conflictingUnder, the features in the embodiments and the embodiments of the present application can be combined with each other.
It should be noted that unless otherwise specified, when a certain feature referred to as " fixation ", " connection " are in another feature,It can directly fix, be connected to another feature, and can also fix, be connected to another feature indirectly.In addition, thisThe descriptions such as upper and lower, left and right used in application are only the mutual alignment pass relative to each component part of the application in attached drawingFor system.In the application and the "an" of singular used in the attached claims, " described " and "the" also purportIt is including most forms, unless the context clearly indicates other meaning.
In addition, unless otherwise defined, the technology of all technical and scientific terms used herein and the artThe normally understood meaning of personnel is identical.Term used in the description is intended merely to description specific embodiment herein, withoutIt is to limit the application.Term as used herein "and/or" includes the arbitrary of one or more relevant listed itemsCombination.
It will be appreciated that though various elements may be described in this application using term first, second, third, etc., butThese elements should not necessarily be limited by these terms.These terms are only used to for same type of element being distinguished from each other out.For example, not taking offIn the case where the application range, first element can also be referred to as second element, and similarly, second element can also be referred to asFirst element.Depending on context, word as used in this " if " can be construed to " ... when " or " when ...When ".
Method flow diagram shown in referring to Fig.1, in one or more embodiments of the application, original neural network basedDrawing auxiliary painting methods may comprise steps of:
S100 the original painting line original text and original painting colouring original text of multiple original paintings colouring front and back) are obtained, and marks each subdivision area of original paintingThe colouring parameter in domain forms original painting data sample；
S200) using Keras initialization colouring neural network model, wherein colouring neural network model uses VGG model；
S300 it) imports original painting data sample and is used as training set, neural network model exercises supervision study to painting；
S400) the colouring neural network model finished using training carries out automatic colouring to the original painting line original text of input.
Wherein, Keras can rapidly construct corresponding colouring nerve net as a kind of simple high-rise neural network toolNetwork model is to be trained.Specifically, the tool that developer can be provided using Keras, the TensorFlow based on open sourceSystem establishes colouring neural network model.Further, the test verifying through current mainstream from image it is found that extract CNN spySign, being trained using VGG model is preference algorithm, is more suitable for that image color lump position is imitated and learnt.AbilityField technique personnel can be based on related tool and the specific colouring neural network model of algorithm principle structure configuration.
Referring to method flow diagram shown in Fig. 2, in one or more embodiments of the application, the step S100 includesFollowing sub-step:
S101 the original painting line original text and original painting colouring original text of each original painting colouring front and back) are matched, and on original painting line original text and original paintingColor original text is registrated；
S102) original painting divide based on original painting line original text and form prime area；
S103) based on hue threshold, saturation degree threshold value and luminance threshold, to prime area carry out region segmentation, formedDivided area；
S104 tone, saturation degree and the average value of brightness in same divided area) are calculated, and by tone average value, saturationAverage value, average brightness and adjacent area number are spent as colouring parameter.
Specifically, referring to schematic diagram shown in Fig. 3, as the original painting data sample for training colouring neural network modelThis, original painting line original text and original painting colouring original text will first be registrated, with accurately determined in subsequent sub-step each divided area withThe relative position that line original text Central Plains draws lines.Then, original painting line original text will divide prime area based on original painting lines.It draws prime areaThe mode of dividing can use the existing image segmentation algorithm based on edge (for example, by using original painting lines as edge), and will segmentationPrime area afterwards be added to original painting colouring original text, with determine prime area original painting colouring original text on corresponding region.It is then possible to baseIn including but not limited to hue threshold, saturation degree threshold value and luminance threshold, region segmentation is carried out to prime area, forms subdivisionRegion.Wherein, hue threshold, saturation degree threshold value and luminance threshold can be using least variance method or Two-dimensional Maximums in existing classThe image algorithms such as entropy partitioning algorithm determine.Those skilled in the art can be determined using appropriate algorithm above-mentioned as the case may beEach threshold value for being used to divide divided area, the application not limit this.It is formed by divided area finally, for segmentation,Calculate tone in corresponding divided area, saturation degree and the average value of brightness, and by the geometry in region and adjacent areaInformation saves in the form of a file as colouring parameter.In addition, it will be understood by those skilled in the art that the display in attached drawing is firstThe number and shape of part are only as illustrative reference, not as the limitation to the application.
Further, in the said one of the application or multiple embodiments, the colouring parameter at least further includes in originalThe original painting lines adjacent with divided area on setting-out original text.Since the region of the adjacent two sides of original painting lines often has biggish colorDifference, therefore the classifier in neural network model of painting can improve the weight of corresponding parameter in initialization, to improveThe training effectiveness (colouring neural network model is enabled rapidly to restrain) and colouring nerve net of colouring neural network modelThe accuracy rate of network model itself.
Still further, in the said one or multiple embodiments of the application, for the original painting of different-style, in order to mentionThe applicability of height colouring neural network model, original painting data sample will be classified as multiple son training according to the Color Style of original paintingCollection, and corresponding multiple colouring neural network models are formed based on the sub- training set.At this point, the resulting colouring neural network of trainingModel will correspond respectively to different original painting colouring styles.In use, original painting to be painted will be designated corresponding colouring style,So as to more targetedly execute automatic colouring operation to the original painting of different specific requirements.
Referring to submethod flow chart shown in Fig. 4, in one or more embodiments of the application, the step S200 is alsoIncluding following sub-step:
S201) using Keras establish and initialize VGG model convolutional layer and MAXpooling layers；
S202 evaluate to VGG model is configured) with real-time testing model training performance.
Specifically, the convolutional layer of VGG model is established and initialized using Keras and at MAXpooling layers, can useThe tool that Kersa itself is provided automatically saves the weight of each classifier conveniently to be iterated training.MeanwhileEvaluate tool provided by Kersa also can be convenient ground real-time detection and assess trained colouring neural network modelTraining effect, such as the colouring whether trained accuracy rate for finishing and painting of neural network model.
Further, submethod flow chart referring to Figure 5, it is described in one or more embodiments of the applicationStep S300 further includes following sub-step:
S301 the training set being made of original painting data sample) is imported into convolutional layer；
S302 it) is exercised supervision study using SqueezeNet convolutional neural networks to training set；
S303 the weight parameter of classifier in convolutional layer and MAXpooling layers) is assessed based on evaluate, and when detectionDeconditioning when weight parameter into classifier restrains.
It is trained as mentioned previously, because evaluate tool provided by Kersa can be convenient ground real-time detection and assessColouring neural network model, check each repetitive exercise so can pass through in the training process of colouring neural network modelWhether the weight parameter variation of each classifier in front and back is greater than preset threshold value, to determine whether deconditioning.Art technologyCorresponding threshold value can be arranged in personnel according to specific training process, and the application not limits this.
Due in electronic game or cartoon making project, corresponding participant (such as software developer and outsourcing designFine arts personnel in company) geographical location can easily may modify original painting, reference relatively far apart in order to facilitate project personnelSubmethod flow chart shown in fig. 6, in one or more embodiments of the application, the step S400 includes following sub-stepIt is rapid:
S401 the colouring neural network model that training finishes) is arranged in network server, and configures colouring neural networkThe Data entries of model；
S402) original painting uploads to colouring neural network model by Data entries to execute automatic colouring.
Further, the Data entries can be the form of webpage.At this point it is possible to by with providing corresponding webpageModified original painting line original text is uploaded in corresponding network server by location by related personnel in webpage, and by network serviceDevice returns to the original painting colouring original text after colouring by network.
Referring to function structure chart shown in Fig. 7, in one or more embodiments of the application, original neural network basedDraw auxiliary coloring means may include with lower module: obtain module, for obtain multiple original paintings colouring front and back original painting line original text andOriginal painting colouring original text, and the colouring parameter of each divided area of original painting is marked, form original painting data sample；Initialization module is used forColouring neural network model is initialized using Keras, wherein colouring neural network model uses VGG model；Training module is used forIt imports original painting data sample and is used as training set, neural network model exercises supervision study to painting；Colouring module, for utilizing instructionPractice the colouring neural network model finished and automatic colouring is carried out to the original painting line original text of input.Wherein, Keras is as a kind of simple heightLayer neural network tool can construct corresponding colouring neural network model rapidly to be trained.Specifically, developerThe tool that can be provided using Keras, the TensorFlow system based on open source establish colouring neural network model.FurtherGround, the test verifying through current mainstream from image it is found that extract CNN feature, being trained using VGG model is preferred calculationMethod is more suitable for that image color lump position is imitated and learnt.Those skilled in the art can be based on related tool and calculationThe specific colouring neural network model of method theory structure configuration.
In one or more embodiments of the application, the acquisition module may include following submodule: matching module,For matching the original painting line original text and original painting colouring original text of each original painting colouring front and back, and original painting line original text and original painting colouring original text are matchedIt is quasi-；First division module forms prime area for divide to original painting based on original painting line original text；Second division module, is used forBased on hue threshold, saturation degree threshold value and luminance threshold, to prime area carry out region segmentation, formed divided area；MarkModule is put down for calculating tone in same divided area, saturation degree and the average value of brightness, and by tone average value, saturation degreeMean value, average brightness and adjacent area number are as colouring parameter.Specifically, referring to schematic diagram shown in Fig. 3, as being used forThe original painting data sample of training colouring neural network model, original painting line original text and original painting colouring original text will be first registrated, subsequentThe relative position that each divided area and line original text Central Plains draw lines is accurately determined in sub-step.Then, original painting line original text will be based onOriginal painting lines divide prime area.The division mode of prime area can be using the existing edge that is based on (for example, by using original painting lineItem is as edge) image segmentation algorithm, and by the prime area after segmentation be added to original painting colouring original text, to determine prime areaThe corresponding region on original painting colouring original text.It is then possible to based on including but not limited to hue threshold, saturation degree threshold value and luminance thresholdValue carries out region segmentation to prime area, forms divided area.Wherein, hue threshold, saturation degree threshold value and luminance threshold can be withIt is determined using the image algorithms such as least variance method or Two-dimensional maximum-entropy partitioning algorithm in existing class.Those skilled in the art can be withDetermine that above-mentioned each threshold value for being used to divide divided area, the application not limit this using appropriate algorithm as the case may beIt is fixed.It is formed by divided area finally, for segmentation, calculates being averaged for tone in corresponding divided area, saturation degree and brightnessValue, and using the geometry in region and the information of adjacent area as colouring parameter, it saves in the form of a file.In addition, abilityField technique personnel will be appreciated that the number and shape of the display element in attached drawing are only as illustrative reference, not as to thisThe limitation of application.
Further, in the said one of the application or multiple embodiments, the colouring parameter at least further includes in originalThe original painting lines adjacent with divided area on setting-out original text.Since the region of the adjacent two sides of original painting lines often has biggish colorDifference, therefore the classifier in neural network model of painting can improve the weight of corresponding parameter in initialization, to improveThe training effectiveness (colouring neural network model is enabled rapidly to restrain) and colouring nerve net of colouring neural network modelThe accuracy rate of network model itself.
Still further, in the said one or multiple embodiments of the application, for the original painting of different-style, in order to mentionThe applicability of height colouring neural network model, original painting data sample will be classified as multiple son training according to the Color Style of original paintingCollection, and corresponding multiple colouring neural network models are formed based on the sub- training set.At this point, the resulting colouring neural network of trainingModel will correspond respectively to different original painting colouring styles.In use, original painting to be painted will be designated corresponding colouring style,So as to more targetedly execute automatic colouring operation to the original painting of different specific requirements.
In one or more embodiments of the application, the initialization module can also include following submodule: establishModule, for established and initialized using Keras VGG model convolutional layer and MAXpooling layers；Configuration module, for configuringEvaluate to VGG model is with real-time testing model training performance.Specifically, it is established using Keras and initializes VGG modelConvolutional layer and at MAXpooling layers, can use Kersa itself offer tool automatically save the weight of each classifier withConveniently it is iterated training.Meanwhile evaluate tool provided by Kersa also can be convenient ground real-time detection simultaneouslyAssess the training effect of trained colouring neural network model, for example, colouring neural network model it is whether trained finish withAnd the accuracy rate of colouring.
Further, in one or more embodiments of the application, the training module can also include following submoduleBlock: import modul, for the training set being made of original painting data sample to be imported into convolutional layer；Execution module, for usingSqueezeNet convolutional neural networks exercise supervision study to training set；Evaluation module, for assessing convolution based on evaluateLayer and MAXpooling layer in classifier weight parameter, and when detect in classifier weight parameter convergence when stop instructPractice.As mentioned previously, because evaluate tool provided by Kersa can be convenient ground real-time detection and assess trained colouringNeural network model, so can be by each before and after each repetitive exercise of inspection in the training process of colouring neural network modelWhether the weight parameter variation of a classifier is greater than preset threshold value, to determine whether deconditioning.Those skilled in the art canCorresponding threshold value is arranged according to specific training process, the application not limits this.
Due in electronic game or cartoon making project, corresponding participant (such as software developer and outsourcing designFine arts personnel in company) geographical location can easily may modify original painting relatively far apart in order to facilitate project personnel, at thisIn one or more embodiments of application, the colouring module can also include following submodule: arrangement module, for that will trainThe colouring neural network model finished is arranged in network server, and configures the Data entries of colouring neural network model；It uploadsModule, for by original painting by Data entries upload to colouring neural network model to execute automatic colouring further, it is describedData entries can be the form of webpage.At this point it is possible to by providing corresponding web page address, it will be modified by related personnelOriginal painting line original text is uploaded in corresponding network server in webpage, and is led to the original painting colouring original text after colouring by network serverCross network return.
It should be appreciated that embodiments herein can be by computer hardware, the combination of hardware and software or by depositingThe computer instruction in non-transitory computer-readable memory is stored up to be effected or carried out.Standard program can be used in this methodTechnology-include realized in computer program configured with the non-transitory computer-readable storage media of computer program, whereinConfigured in this way storage medium operates computer in a manner of specific and is predefined --- according to retouching in a particular embodimentThe method and attached drawing stated.Each program can with the programming language of level process or object-oriented come realize with computer systemCommunication.However, if desired, the program can be realized with compilation or machine language.Under any circumstance, which can be compilingOr the language explained.In addition, the program can be run on the specific integrated circuit of programming for this purpose.
Further, this method can be realized in being operably coupled to suitable any kind of computing platform, wrapInclude but be not limited to PC, mini-computer, main frame, work station, network or distributed computing environment, individual or integratedComputer platform or communicated with charged particle tool or other imaging devices etc..The various aspects of the application can be to depositThe machine readable code on non-transitory storage medium or equipment is stored up to realize no matter be moveable or be integrated to calculatingPlatform, such as hard disk, optical reading and/or write-in storage medium, RAM, ROM, so that it can be read by programmable calculator, whenStorage medium or equipment can be used for configuration and operation computer to execute process described herein when being read by computer.ThisOutside, machine readable code, or part thereof can be transmitted by wired or wireless network.When such media include combining microprocessorOr when other data processors realization instruction or program of the step above, application as described herein includes that these and other are differentThe non-transitory computer-readable storage media of type.When being programmed according to methods and techniques described herein, the application is alsoIncluding computer itself.
Computer program can be applied to input data to execute function as described herein, to convert input data with lifeAt storing to the output data of nonvolatile memory.Output information can also be applied to one or more output equipments as shownDevice.In the application preferred embodiment, the data of conversion indicate physics and tangible object, including the object generated on displayReason and the particular visual of physical objects are described.
Therefore, should be with descriptive sense rather than restrictive sense understands the specification and drawings.However, by apparentIt is:, can be to the application in the case where not departing from the broader spirit and scope of the application as described in claimsMake various modifications and change.
Other modifications are in spirit herein.Therefore, although disclosed technology may be allowed various modifications and substitution structureIt makes, but has shown that in the accompanying drawings and its some embodiments shown in being described in detail above.It will be appreciated, however, that notIt is intended to for the application to be confined to disclosed one or more concrete forms；On the contrary, its intention covers such as the appended claimsDefined in fall in all modifications, alternative constructions and equivalent in spirit and scope.