Detailed Description
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
As used in the specification and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Although the present application makes various references to certain modules in a system according to embodiments of the present application, any number of different modules may be used and run on a user terminal and/or server. The modules are merely illustrative, and different aspects of the systems and methods may use different modules.
A flowchart is used in the present application to describe the operations performed by a system according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in order precisely. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Also, other operations may be added to or removed from these processes.
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
It should be noted in advance that the acquisition and processing of all information or data in the application are carried out on the premise of conforming to the national corresponding data protection regulation policy and on the premise of obtaining the authority manager to give authorization.
Aiming at the problems of large syllable rule difference, unbalanced data resources, mixed language interference and the like in multilingual place name translation, the application provides a high-precision place name translation method integrating artificial intelligence and multilingual syllable segmentation. The scheme adopts the design concept of layering decision-semantic driving, and optimizes translation accuracy by dynamically adapting syllable segmentation strategies of different language types and combining context semantic understanding.
Based on the above, in the technical scheme of the application, a high-precision place name translation method integrating artificial intelligence and multi-language syllable segmentation is provided. FIG. 1 is a flow chart of a method for high-precision place name translation integrating artificial intelligence with multilingual syllable segmentation according to an embodiment of the present application. FIG. 2 is a data flow diagram of a high-precision place name translation method integrating artificial intelligence and multi-language syllable segmentation according to an embodiment of the present application. As shown in FIGS. 1 and 2, the high-precision place name translation method integrating artificial intelligence and multilingual syllable segmentation according to the embodiment of the application comprises the steps of S1, receiving a place name character string to be translated input by a user, S2, detecting a source language of the place name character string to be translated by using a language identification model, S3, determining a syllable segmentation strategy based on the source language of the place name character string to be translated, S4, responding to the syllable segmentation strategy as a syllable segmentation method based on a deep neural network model, performing syllable segmentation on the place name character string to be translated to obtain a place name character string after syllable segmentation based on the syllable segmentation strategy, and S5, inputting the place name character string after syllable segmentation into a place name translation model to obtain a place name target language translation text.
Specifically, the S1 receives a to-be-translated place name string input by a user. Wherein the place name string to be translated is typically composed of a series of characters, representing the name of a particular geographic location, which may be letters, numbers, symbols, or a combination thereof. The place name string to be translated may contain a single language component or may contain a mixed language component (e.g., "New Delhi" combines english and hindi language elements). It is important for the translation system to accurately identify the source language of the string and determine subsequent processing steps based thereon. Specifically, in this process, a user may be allowed to input a place name desired to be translated through various devices such as a computer, a mobile phone, etc., by designing a user-friendly interface, which may be a form on a web page, an input box in a mobile application, etc.
In particular, the S2 detects a source language of the place name string to be translated using a language recognition model. Wherein the language recognition model is usually constructed based on deep learning technology, and can automatically learn and recognize characteristic patterns of different languages. After the user inputs the place name character string to be translated, the character string is sent to a pre-trained language recognition model. The model will perform a comprehensive analysis of the input character sequence to extract key information that can represent specific language features. For example, for languages with unique character sets (e.g., chinese characters, russian cyrillic letters), the model can be determined quickly based directly on character type, while for languages with the same character set but with significant differences in grammar structure or vocabulary distribution (e.g., english versus french), more complex statistical features and contextual information can be relied upon to distinguish. The language recognition model can be an advanced deep learning framework such as a Convolutional Neural Network (CNN), a cyclic neural network (RNN), a variant thereof (such as a long-short-term memory network LSTM, a gate-controlled cyclic unit GRU) or a transducer architecture. These models can effectively capture local and global features in an input string and map them into a probability distribution space representing the likelihood of different languages through multiple layers of nonlinear transformations. Finally, the model outputs a probability score for each known language that is the highest scoring source language that is considered to be the most likely correspondence of the input string.
In particular, the S3 determines a syllable segmentation strategy based on the source language of the place name string to be translated. That is, the characteristics and rules of the language are evaluated based on the source language of the place name string to be translated. For example, because the high-resource language already has a mature syllable segmentation rule base, the problem of syllable boundaries can be handled relatively directly by applying established rules, and therefore, if a high-resource language such as English, french or Spanish is detected, a rule-based syllable segmentation method is usually selected, wherein rules may include how to handle special cases such as meta-consonant combinations, irregular spelling, etc., so as to ensure that syllable structures inside each word can be accurately identified, and for those low-and medium-resource languages such as Chinese, japanese, korean, vietnam, etc., there may be insufficient labeling data to support the rule-based method. In this case, the system tends to adopt syllable segmentation strategy based on statistical model, for example, a syllable segmentation rule suitable for specific language can be automatically found and applied by using a Conditional Random Field (CRF), a Hidden Markov Model (HMM) or other models suitable for sequence labeling task, and when the place names of mixed language components are encountered, such as 'New Delhi', and examples of English and Hindi elements are contained simultaneously, a single language processing strategy is difficult to meet the requirement, at this time, the system selects syllable segmentation method based on deep neural network model so as to dynamically adapt to interaction between different languages and capture more complex language phenomena through deep architecture. Specifically, the deep neural network can learn commonalities and differences among multiple languages through training, so that high accuracy is still maintained when cross-language interference is faced. For example, when compound words or adhesive morphemes are processed, the deep neural network can identify the implicit syllable association between the root and the suffix, and enhance the local morphological characteristics near the syllable boundary, thereby effectively solving the problems of irregular spelling and cross-language interference.
Specifically, the step S4 is to perform syllable segmentation on the place name string to be translated based on the syllable segmentation strategy in response to the syllable segmentation strategy being a syllable segmentation method based on a deep neural network model, so as to obtain the place name string after syllable segmentation. In a specific example of the present application, as shown in fig. 3, S4 is shown, where the method includes S41, performing a context Wen Yuyi association coding based on a place name Token unit on a place name character string to be translated to obtain a place name semantic context association coding representation to be translated, S42, performing local semantic association reconstruction reinforcement on the place name semantic context association coding representation to be translated to obtain a place name semantic context association enhancement coding representation to be translated, and S43, determining a place name character string after syllable segmentation based on the place name semantic context association enhancement coding representation to be translated.
Specifically, the step S41 is to perform context Wen Yuyi association coding based on the place name Token unit on the place name character string to be translated to obtain a semantic context association coding representation of the place name to be translated. That is, in the embodiment of the present application, first, word segmentation processing is performed on the to-be-translated place name character string to obtain the sequence distribution of the to-be-translated place name Token units. It should be appreciated that when the input place name character stream contains a mixed writing system (e.g., latin letters, tone marks, or sticky morphemes are present as well), the continuity and unstructured nature of the original character string may prevent the model from recognizing language attribution and internal rules. For example, facing a long string of sticky phones, which may have multiple root and suffix combinations nested inside, traditional coarse-granularity space-based segmentation cannot capture implicit syllable boundaries (e.g., consonant conjunctions or vowel-to-vowel rules). At this time, the character sequence is deconstructed into Token units with independent semantic or acoustic meanings (for example, the adhesive morphemes are disassembled into a combination of 'root and suffix') by a multi-language sub-word segmentation algorithm, so that word forming logic in the language can be explicitly revealed, for example, a core semantic unit and a grammar mark are separated from each other in a compound word, or the problem of visual adhesion of conjoined characters is correctly processed in non-Latin characters. By generating Token sequence distributions, the model can transform heterogeneous input character streams into a set of units with discretized semantic boundaries, such as in languages containing tone marks, combining tone marks with underlying letters into independent tokens to preserve the phonetic features (e.g., binding tone marks "́" with vowel letters), or splitting sub-units in a dense spelling of consonants that meet the target language phoneme rules (e.g., splitting consonant affixes into legal syllable combinations). The discretization processing provides a resolvable infrastructure for subsequent semantic coding, particularly when processing non-canonical spellings (such as ancient language variants in historical place names), the word segmentation module can effectively distinguish core morphemes in language variants from interference noise through Token units generated by an adaptive strategy, for example, character blocks of different language systems are identified and isolated in mixed language components, so that segmentation errors caused by cross-language rule conflicts are avoided.
And then, carrying out semantic embedded coding on each to-be-translated place name Token unit in the sequence distribution of the to-be-translated place name Token units to obtain the sequence distribution of the to-be-translated place name Token unit semantic embedded coding vector. That is, in the technical scheme of the application, each to-be-translated place name Token unit in the sequence distribution of the to-be-translated place name Token unit is respectively embedded into an encoder through words based on Word2Vec to obtain the sequence distribution of the to-be-translated place name Token unit semantic embedded coding vector. It should be understood that in conventional natural language processing, token units are used as discrete symbols (such as subwords, roots or character combinations), which cannot directly express semantic relevance and internal rules of the language. For example, in a sticky language, if a suffix Token representing an orientation lattice and another suffix Token representing a relationship belong to each other, the model is difficult to automatically infer the similarity of the grammar functions, and in a cross-language scene, token symbols of synonymous roots (such as "mountain" and "mountain") of different language systems lack a computable relational expression. Word2 Vec-based Word embedded encoders can map each discrete Token to a low-dimensional continuous vector space by using the statistical rule of Token co-occurrence in a context window through unsupervised training, so that tokens with similar semantics or functions have geometric adjacency in the vector space (such as orientation lattice suffix vectors form cluster distribution in the vector space). Such continuous vector characterization provides a semantic basis for quantifiable computation for subsequent models.
Furthermore, the sequence distribution of the semantic embedded coding vector of the place name Token unit to be translated is subjected to place name Token context Wen Yuyi association coding to obtain a place name semantic context association coding vector to be translated, and the place name semantic context association coding vector to be translated is used as a place name semantic context association coding representation to be translated. That is, in the technical scheme of the application, the sequence distribution of the semantic embedded coding vector of the name Token unit to be translated is used for obtaining the semantic context associated coding vector of the name Token to be translated through a BiLSTM-based context Wen Yuyi associated encoder. It should be understood that the semantic embedded encoding vector of the Token unit of the place name to be translated only expresses the static semantics of the Token (such as the independent meaning of the root of the attached language), and cannot model the interaction rule between adjacent tokens (such as the constraint effect of the suffix on the structure of the stem syllable). For example, a suffix Token vector representing a position lattice, whose semantic function needs to be combined with the vector of the stem Token before the grammatical meaning of "in..location" can be fully expressed, whereas unidirectional LSTM can only capture forward dependency, and it is difficult to capture the reverse effect of the suffix on the stem Token (e.g., the change of the stem syllable accent location by the german separable verb prefix). BiLSTM respectively modeling time sequence relations in a forward direction and a reverse direction in two time dimensions through a bidirectional gating mechanism, so that the hidden state of each position is fused with history and future context information (such as the grammar function of a prefix influences the segmentation of a subsequent syllable, and the morphological characteristics of the prefix also correct the pronunciation rules of a front word stem), thereby constructing a global semantic association field. Specifically, the gating structure BiLSTM (input gate, forget gate, output gate) controls the transfer and forget of the information stream by parameterization. For example, in processing long Token sequences in a Token, a forgetting gate may dynamically determine long term memory (e.g., core semantics of the root) that needs to be preserved, while an import gate screens local features (e.g., grammatical attributes of the suffix) of the current Token. The superposition of the bidirectional mechanism enables the model to gradually accumulate the modification effect of the prefix on the stem (such as mega-representing huge) when the model encodes the structure of prefix, root and suffix, and the reverse LSTM reversely captures the morphological constraint of the suffix on the root (such as polis-representing city), finally, context representation containing bidirectional dependence is formed through the splicing of hidden states, and the semantic context association coding vector of the place name to be translated is obtained. It is worth mentioning that bi-directional context coding breaks through the limitations of local windows, enabling models to handle long distance dependencies (e.g. consonant assimilation phenomena across multiple Token), e.g. to identify core syllable boundaries in compound words that are separated into multiple modifiers, furthermore, dynamic gating mechanisms enhance the model's adaptability to irregular language phenomena, e.g. to suppress interfering noise in spelling variants (e.g. redundant characters in historic place names) by forgetting gates, while reinforcing key morphological features (e.g. the indicative effect of glottal symbols on syllable segmentation) by input gates, finally, bi-directional fused hidden states provide semantic representation for subsequent modules, e.g. distinguishing ambiguous suffixes in adhesive language, whose context correlation vectors activate specific features in different dimensions.
Specifically, the step S42 is to perform local semantic relevance reconstruction reinforcement on the semantic context-associated encoded representation of the place name to be translated to obtain the semantic context-associated enhanced encoded representation of the place name to be translated. It should be understood that, after the place names to be translated are subjected to word embedding and context semantic coding, although the global semantic dependency relationship has been partially modeled, long compound words formed by syllable superposition in the adhesive language, spelling patterns of alternative consonant clusters and vowels in the inflected language, and cross-language interference components of the mixed language place names often appear as nonlinear coupling of local semantic structures. Therefore, in order to realize feature rectification and dynamic enhancement of semantic context associated features of the place names to be translated, in the technical scheme of the application, local semantic relevance reconstruction enhancement is carried out on the semantic context associated coding vectors of the place names to be translated so as to obtain the semantic context associated enhancement coding vectors of the place names to be translated. In the process, firstly, a multiscale local structure mode is extracted on a coding vector sequence based on characteristic phase reconstruction of one-dimensional convolutional coding through a sliding window mechanism (such as a convolution kernel with the length of 3 captures morphological interaction characteristics of three adjacent Token), local phase information (namely, the relative relation between adjacent dimensions) hidden in a continuous vector space is modeled explicitly, then, as an initial phase coding set possibly contains redundant information (such as smooth transition characteristics of a non-boundary area) and noise interference (such as irregular spelling fluctuation caused by cross-language mixing), the characteristic dimension screening is carried out on the local phase coding vectors through a learning attention mechanism or sparse constraint in an information squeezing stage, for example, in a region with dense consonant clusters, the model can strengthen the dimension representing the transition steepness of consonants and inhibit the dimension representing the stability of vowels, in a postfix dense pasting language section, the dimension representing the grammar function is reserved, and then, an information density index (namely, an effective component statistics component) of each local phase coding vector is calculated through a statistical feature effective component, so that dynamic gain operator is generated. For example, when detecting that the statistics of a local phase code is significantly higher than a threshold value, the gain operator can perform nonlinear amplification on the feature of the corresponding dimension (such as generating an enhancement coefficient between 0 and 1 through a Sigmoid function), so as to highlight abrupt signals (such as consonant attachment break points or vowel weakening break points) at syllable boundaries in the feature remodeling stage, and finally, realize collaborative optimization of local structures and global contexts through phase significance remodeling. For example, when processing mixed language place names, global context coding may be difficult to locate syllable boundaries accurately due to cross-language rule conflicts, but the local phase enhancement module suppresses interference signals of conflicting language systems (e.g., long consonant sequences of a cohesive language) by amplifying local features (e.g., syllable opening preferences in a latin language system) conforming to target language system rules, so that the enhanced semantic context-dependent enhancement coding vector of the place name to be translated can still maintain stable discriminant under a cross-language interference scene. The self-adaptive enhancement mechanism essentially constructs a multi-level characteristic interaction system and provides an input representation with both global robustness and local sensitivity for a depth decoder.
Specifically, firstly, semantic unit reconstruction based on one-dimensional convolution coding is carried out on semantic context associated coding vectors of the place names to be translated so as to obtain a set of semantic feature local phase coding vectors of the place names to be translated. It should be appreciated that the semantic context-dependent encoding vector for the place name to be translated output BiLSTM includes global semantic dependencies (e.g., cross-position grammatical association of the root and suffix in the adhesive language) that may weaken microscopic interaction rules between adjacent feature dimensions (e.g., pronunciation transition features of consonant-suffix regions or acoustic continuity of vowel weakening regions). For example, in a dense semantic segment of a consonant cluster, a global encoding vector can express the overall meaning of a morpheme, but it is difficult to capture steep changes inside the consonant attachment (such as alternating patterns of stop and wipe), and such local abrupt changes are precisely key signals of syllable boundaries. Therefore, in the technical scheme of the application, semantic unit reconstruction based on one-dimensional convolution coding is carried out on the semantic context associated coding vector of the place name to be translated so as to obtain a set of semantic feature local phase coding vectors of the place name to be translated. Here, by employing multiple sets of convolution kernels of different sizes (e.g., 3/5/7 window length), the model is able to capture local phase patterns from different receptive fields, short windows focused on microscopic interactions in the immediate dimension (e.g., breaking features of a double consonant attachment), and long windows modeling a gradual change law across the dimension (e.g., tonal continuity in vowels harmony). For example, in the dense region of the suffix of the adhesive, the multi-scale convolution kernel can simultaneously capture the consonant-vowel combination rule (short window) inside the suffix and the morphological superposition trend (long window) of the suffix sequence to form a set of semantic feature local phase encoding vectors of the place name to be translated covering a plurality of possible morphologies of syllable boundaries. This multi-angle feature extraction provides a structured intermediate representation for the subsequent modules, enabling the model to distinguish between true syllable boundaries (e.g., consonant attachment breaks) and pseudo-boundaries (e.g., stable consonant combinations inside the root). In addition, the characteristic phase reconstruction significantly enhances the sensitivity of the model to implicit local laws. For example, in dealing with mixed language interference, global codes may confuse syllable boundaries due to cross-language rule conflicts, but local phase codes effectively suppress extraneous features of the interfering language (e.g., long consonant sequences of a cohesive language) by enhancing the target language specific syllable patterns (e.g., the latin syllable preference). The fine-granularity structured decoding lays an interpretable physical foundation for subsequent characteristic rectification and remodelling. In a specific example of the application, semantic unit reconstruction based on one-dimensional convolution coding is carried out on semantic context associated coding vectors of a place name to be translated by using a semantic unit reconstruction formula to obtain a set of semantic feature local phase coding vectors of the place name to be translated, wherein the semantic unit reconstruction formula is as follows:
;
Wherein,Is the semantic context associated coding vector of the place name to be translated,Is a one-dimensional convolutional encoding process,For the characteristic phase reconstruction step size,The 1 st, the 2 nd and the 2 nd of the set of the semantic feature local phase coding vectors of the place name to be translatedAnd (b)And the semantic features of the place names to be translated are locally phase coded vectors.
And then, carrying out semantic purification on each to-be-translated place name semantic feature local phase code vector in the to-be-translated place name semantic feature local phase code vector set to obtain a to-be-translated place name semantic feature extraction local phase code vector set. It should be understood that, although the set of the semantic feature local phase encoding vectors of the place name to be translated generated by the one-dimensional convolution encoding includes multi-scale structural information (such as a steep change mode of consonant attachment or a harmonious gradual change trend of vowels), dimensional redundancy (such as similar edge features extracted by adjacent convolution kernels) may exist in the set of the semantic feature local phase encoding vectors of the place name to be translated, and the set of the semantic feature local phase encoding vectors may be mixed with information (such as non-target audio feature superposition caused by cross-language interference). For example, in localized phase encoding where the consonant clusters are dense, multiple convolution kernels may simultaneously activate responses to consonant transition features, resulting in collinearity between feature dimensions, whereas in mixed language scenarios, partial convolution kernels may capture the system rules of non-target languages (e.g., the interference of long consonant sequences of an adhesive language with Latin syllable divisions), creating semantically independent noise dimensions. Semantic purification carries out subspace projection on a high-dimensional space of a local phase coding vector of semantic features of a place name to be translated through a learnable attention mechanism or sparse constraint, screens out key dimensions (such as abrupt signals of consonant breaking points or turning features of vowel weakening) which are strongly related to syllable boundary judgment, and simultaneously suppresses low-information-content dimensions (such as gentle fluctuation of a stable vowel region) and cross-language interference noise. In the process, by introducing feature importance assessment based on task driving, the model can quantify the contribution degree of each semantic feature local phase coding vector of the place name to be translated to a final syllable segmentation target, so that language-specific noise is weakened by strengthening high-value features of cross-language commonality (such as statistics rules of consonant continuous break), and generalization capability is improved. It should be noted that, the semantic purification is not simple dimension reduction, but the feature space is reconstructed through nonlinear transformation, for example, the activation intensity of each dimension is dynamically adjusted by using a gating mechanism, so that the extracted local phase encoding vector set extracted by the semantic features of the place name to be translated not only maintains the diversity of multi-scale structural information, but also has the discriminant of task adaptation. The purification and reconstruction of the feature space essentially constructs an anti-noise and high-discrimination local feature substrate for syllable boundary detection, and provides a bottom guarantee for the reliability of an end-to-end translation system. In a specific example of the application, semantic purification is carried out on each to-be-translated place name semantic feature local phase code vector in a set of to-be-translated place name semantic feature local phase code vectors by using a semantic purification formula to obtain a set of to-be-translated place name semantic feature extraction local phase code vectors, wherein the semantic purification formula is as follows:
;
Wherein,Is a norm of the vector which is the one,Squeezing out the first set of local phase-encoding vectors for semantic features of the place name to be translatedAnd extracting local phase encoding vectors by semantic features of the names to be translated.
And then, calculating statistics of the semantic features of the place names to be translated, which are extracted from the semantic features of the place names to be translated, in the set of the semantic features of the place names to be translated, and extracting the semantic features of the place names to be translated from the set of the local phase encoding vectors. It should be appreciated that, although the semantically refined set of local phase-encoding vectors has been initially filtered for redundant dimensions (e.g., low variance fluctuation features of consonant attachment regions) and noise disturbances (e.g., non-target audio patterns resulting from cross-language mixing), there is still heterogeneity in the feature dimension contribution—some dimensions may bear strong indicators of syllable boundaries (e.g., steep gradient changes of consonant break points), while others may carry only weakly relevant or poorly generalized local information (e.g., rare spelling variants unique to the particular language). For example, in processing the morphemes, the dimension characterizing the suffix morphology rules may have global universality to syllable segmentation, while the dimension characterizing the root internal consonant attachment may only be valid in a particular language. In the technical scheme of the application, the effective component statistics of the semantic features of the place names to be translated of each set of the semantic features of the place names to be translated extracted of the local phase encoding vectors are calculated, wherein the effective component statistics identify the characteristic dimension (such as a cross-language stable consonant transition mode) and the low-value dimension (such as a language specific decorative character combination) with high discriminant through quantitative analysis (such as the variance, entropy or mutual information of a task tag of a calculated dimension activation value) so as to provide an interpretable regulation basis for a subsequent gain operator. Through calculation of the effective component statistics of the semantic features of the place names to be translated, the model can convert the extracted local phase encoding vectors of the semantic features of the place names to be translated into operable numerical indexes, for example, in a consonant assimilation area, physical features (such as transition slope from gum sound to soft palate sound) with higher statistics possibly corresponding to the change of the pronunciation parts of the consonants, and environment noise (such as redundant symbols in spelling variants) with irrelevant lower statistics possibly reflecting the statistics. The quantitative control mechanism based on statistics establishes task-oriented feature importance ordering for extracting local phase coding vectors for semantic features of place names to be translated, so that subsequent significance remodeling can realize self-adaptive enhancement in a data-driven mode, and high-robustness intermediate characterization is provided for an end-to-end translation system. In a specific example of the application, calculating the statistics of the semantic features of the place names to be translated of each semantic feature of the place names to be translated of the set of the local phase encoding vectors to be extracted by the semantic features of the place names to be translated according to the following statistical formula, wherein the statistical formula is as follows:
;
Wherein,Is the firstSqueezing out the first part of the local phase code vector from the semantic features of the place names to be translatedThe characteristic value of the individual position is used,Indicating the number of active ingredients to be counted,For the trainable preset threshold value,Is thatAnd corresponding semantic feature effective components statistics of the place name to be translated.
And then, calculating the semantic feature phase remodelling gain operator of the to-be-translated place name of each to-be-translated place name semantic feature partial phase coding vector based on the statistics of the to-be-translated place name semantic feature effective components of each to-be-translated place name semantic feature extraction partial phase coding vector. It should be appreciated that after word embedding encoding and modeling of the top and bottom Wen Yuyi, noise interference (e.g., morphological conflicts of irregularly spelled consonant clusters, cross-lingual mixed components) still exists in the local semantic representation of the place name to be translated, and these interference can mask deep feature patterns that actually determine syllable boundaries. By calculating the statistics of the feature active components, the system is able to quantify the effective information density in each local phase-encoding vector that is strongly correlated with the syllable slicing task. Further, in order to construct a dynamic self-adaptive feature enhancement mechanism, in the technical scheme of the application, an initial to-be-translated place name semantic feature phase remodeling gain operator of each to-be-translated place name semantic feature local phase coding vector is calculated. Different from the traditional fixed weight feature strengthening mode, the phase remodeling gain operator fuses linguistic rules (such as consonant and meta-consonant collocation rules) with the data-driven features to form an interpretable feature regulation tool. Specifically, the operator dynamically adjusts the activation thresholds of different local phase vectors according to the feature active ingredient statistics, namely, for the coded vectors containing high-frequency active features (such as syllable boundary signals of marked suffixes in the adhesive language), the gain operator amplifies the dimension related to syllable segmentation in the characterization space, and for the low-efficiency feature areas (such as spelling conflict areas in mixed place names) with serious cross-language interference, the propagation of noise signals is restrained through nonlinear scaling. In addition, the self-adaptive characteristic remodelling remarkably improves the robustness of syllable segmentation, so that when a translation system faces to languages with fuzzy syllable boundaries such as Tibetan, the translation system can still perform phase alignment on the initial and final combination mode through a gain operator, and potential segmentation points influenced by the length of vowels and consonants can be accurately identified. In a specific example of the application, a to-be-translated place name semantic feature phase remodeling gain operator of each to-be-translated place name semantic feature local phase coding vector can be calculated by determining a suppression factor corresponding to each to-be-translated place name semantic feature extracted local phase coding vector based on a to-be-translated place name semantic feature effective component statistics of each to-be-translated place name semantic feature extracted local phase coding vector, and calculating an initial to-be-translated place name semantic feature phase remodeling gain operator of each to-be-translated place name semantic feature local phase coding vector based on each to-be-translated place name semantic feature extracted local phase coding vector. In a specific example of the application, a to-be-translated place name semantic feature phase remodeling gain operator of each to-be-translated place name semantic feature local phase coding vector can be calculated by determining a suppression factor corresponding to each to-be-translated place name semantic feature extracted local phase coding vector based on a to-be-translated place name semantic feature active ingredient statistics of each to-be-translated place name semantic feature extracted local phase coding vector, calculating an initial to-be-translated place name semantic feature phase remodeling gain operator of each to-be-translated place name semantic feature local phase coding vector based on each to-be-translated place name semantic feature extracted local phase coding vector, and performing feature phase dispersion deletion correction on the initial to-be-translated place name semantic feature phase remodeling gain operator to obtain the to-be-translated place name semantic feature phase remodeling gain operator.
In particular, when the initial to-be-translated place name semantic feature phase remodeling gain operator performs nonlinear amplification on the local phase code based on statistics (e.g., the dimension characterizing the steep gradient of consonant break points is enhanced by 1.5 times), excessive focusing on the significance of the local features may destroy the overall distribution structure of the feature space. For example, in the processing of a long suffix sequence of an attached morpheme, if the gain operators of a plurality of suffix Token independently enhance their grammatical functional dimensions, the distribution of feature vectors in space may be unevenly scattered (such as excessive separation of partial vector clusters), thereby weakening global context relevance. This phenomenon is mathematically manifested as a lack of phase-direction symmetry, i.e. an inherent consistency in the geometrical distribution of the enhanced feature vector in space deviating from linguistic laws. Thus, in a preferred example of the present application, the initial place name semantic feature phase remodeling gain operator to be translated is subjected to feature phase dispersion deletion correction to obtain the place name semantic feature phase remodeling gain operator to be translated. That is, the effect of the gain operator is constrained within the scope of maintaining spatially pure flatness by introducing a flatness resolution scale. For example, when processing consonant clusters of mixed-language place names, the modified gain operator ensures that the consonant attachment features of Latin language and the long consonant sequences of the adhesion elements maintain directional compatibility in vector space, avoiding spatial warping due to language rule conflicts. The mathematical mechanism converts the initial to-be-translated place name semantic feature phase remodelling gain operator of single-mode coupling into standard operation conforming to space geometric constraint through standard field construction, so that the feature enhancement process maintains the overall stability of feature distribution while improving local significance. The corrected semantic feature phase remodelling gain operator of the original place name to be translated enables the model to stably generalize learned acoustic rules (such as separation modes of Tibetan conjoined characters) in the scarce labeling data to unobserved language variants by keeping translation invariance of feature vectors. The mathematical correction based on the full-pure structure essentially builds the deep mapping of the linguistic rules and the feature space geometry, and provides a feature enhancement mode with both local sensitivity and global consistency for an end-to-end translation system.
In the example, based on the semantic features of each place name to be translated, extracting the statistics of the semantic features of the place name to be translated of the local phase encoding vector, and calculating the semantic feature phase remodeling gain operator of the place name to be translated of the semantic feature local phase encoding vector of each place name to be translated according to the following calculation formula, wherein the calculation formula is as follows:
;
;
;
;
Wherein,For the number of vectors in the set of semantic feature local phase encoding vectors of the place name to be translated,Is thatThe corresponding polar angle is used to determine the angle,Is thatThe corresponding inhibitory factor(s) are (are) used,The circumference ratio is indicated as such,The inverse tangent function is represented by a graph,Remodelling gain operators for semantic feature phases of the place names to be translated,Is thatCorresponding initial to-be-translated place name semantic feature phase remodelling gain operators,For the place name semantic space flattening factor to be translated,The degree rule representation factors are decomposed for the semantic flatness of the place names to be translated,And (5) remodelling a gain operator for semantic feature phases of the place names to be translated.
And further, carrying out feature phase significance remodelling on the set of the to-be-translated place name semantic feature local phase coding vectors based on the to-be-translated place name semantic feature phase remodelling gain operator of each to-be-translated place name semantic feature local phase coding vector so as to obtain the to-be-translated place name semantic context association enhanced coding vector. Here, after dynamic regulation and control are completed through statistics driving (for example, 1.5 multiplication benefit coefficients are given to consonant breaking point features) by the semantic feature phase remodeling gain operator of the place name semantic feature local phase coding vector to be translated, the essence is that the sound system rule of linguistics (for example, morphological constraint of the attached language suffix) is converted into geometric operation of feature space. For example, when processing compound words, a local phase-coded vector corresponding to a consonant assimilation phenomenon where the root and suffix are combined, by nonlinear scaling of the gain operator, a direction-specific enhancement (e.g., a particular dimension of the feature vector is stretched to amplify the sharpness of the consonant transition) is formed in vector space, thereby explicitly marking the potential location of syllable boundaries. The feature enhancement mechanism based on geometric space reconstruction essentially establishes an interpretable mapping between linguistic rules and machine-understandable feature distribution, and provides a core technical support with field knowledge embedding and data driving adaptability for an end-to-end translation system. In a specific example of the application, a set of semantic feature local phase encoding vectors of a place name to be translated is subjected to feature phase significance remodeling according to the following feature phase remodeling formula to obtain the semantic context association enhanced encoding vector of the place name to be translated, wherein the feature phase remodeling formula is as follows:
;
;
Wherein,To take the following measuresThe natural index function value of the base is,Gain weights are remodeled for semantic feature phases of the place names to be translated,And enhancing the coding vector for semantic context association of the place name to be translated.
Specifically, the step S43 is to determine the syllable-segmented place name string based on the place name semantic context-associated enhancement code representation to be translated. In other words, in the technical scheme of the application, the to-be-translated place name semantic context association enhanced coding vector is input into the syllable segmentation decoder based on the deep neural network model to obtain the place name character string after syllable segmentation. It should be understood that the semantic context-association enhanced encoding vector of the place name to be translated contains dual information of global context association and local morphological features, which essentially constructs a cross-language abstract sound system space containing both a historical track of root evolution and a dynamic distribution pattern of syllable boundaries. While traditional finite state machine-based decoders have difficulty capturing their internal complex syllable stacking logic, cross-lingual component interactions in mixed-language place names, and therefore, decoders have the flexible ability to dynamically adjust the system resolution strategy. The deep neural network decoder can build a reverse mapping from the abstract feature space to a concrete syllable sequence. Specifically, it first analyzes the burst energy distribution of the consonant cluster (such as the initial plosive combination of "pf-" in German) through multi-layer nonlinear transformation, and then dynamically focuses on the characteristic mutation region at the concatenation of the clay morphemes (such as "in Turkish language") in combination with the attention mechanism ""The join boundary of suffix and stem word"). By the method, dynamic balance can be achieved between the system rule reasoning and morphological feature analysis, and syllable segmentation accuracy breakthrough across language barriers is achieved. The decoding paradigm based on the full-pure structure constraint essentially builds a mathematical bridge of linguistic priori knowledge and a data driven model, and provides an end-to-end solution with rule rigor and self-adaptability for place name translation in globalization scenes.
Specifically, in the step S5, the place name character string after syllable segmentation is input into a place name translation model to obtain a place name target language translation text. The place name translation model is a deep learning model, and can understand and convert complex relations among different languages. Specifically, the model internally contains multiple hierarchical neural network layers, each of which is responsible for capturing different aspects of the input data, from underlying linguistic symbolic representations to higher-level semantic understanding and cross-language mapping. In the process, firstly, a model performs preliminary analysis on an input place name character string to identify the meaning of each character or Token (sub word unit) and the effect of each character or Token in a sentence, then, a deep neural network architecture such as a bidirectional long and short time memory network (BiLSTM), a Transformer and the like is utilized to start comprehensive analysis on the input character string, and the meaning of each individual word is understood, and the whole meaning of the whole place name character string and the interrelationship among all parts are also known. For example, when processing compound words or place names containing adhesive morphemes, the model needs to recognize implicit association between the root and suffix and adjust the translation strategy accordingly, and then the place name translation model reorganizes and optimizes the parsed information according to the characteristics of the target language. This includes, but is not limited to, adjusting grammar constructs, selecting appropriate synonyms or paraphraseology, taking into account cultural background differences, etc., to generate translation results that are both faithful to the original text and conform to the target language habits. It is noted that this stage may also involve the processing of special morphological features such as prefixes/suffixes, consonant clusters, etc. to ensure that the final output place name translation text is both natural and smooth and accurate. By the method, the system realizes the end-to-end effective conversion from the original place name to the target translated name, greatly improves the efficiency and accuracy of cross-language information processing, and particularly shows important value in the fields of geographic information systems, multilingual map navigation, cross-border logistics and the like.
It should be noted that, as well as the above-disclosed technical scheme of inputting syllable-segmented place name strings into place name translation models to obtain place name target language translation text, other existing technologies may be adopted to implement the technical process. For example, in another specific embodiment, the place name translation scheme in paper Neural Machine Translation for Low-Resource NAMED ENTITIES may be used to translate the syllable-segmented place name string to obtain the place name target language translation text. It should be appreciated that Neural Machine Translation for Low-Resource NAMED ENTITIES is a prior art and is not developed herein to avoid redundancy.
In summary, the high-precision place name translation method integrating artificial intelligence and multi-language syllable segmentation is clarified according to the embodiment of the application, input place names are split into sub-word Token sequences, a context coding model of a word embedding model is utilized to realize the semantic context association coding of the place names to be translated, then internal structural information and dependency relations in the semantic context of the place names to be translated are modeled in a local semantic association reconstruction strengthening mode, hidden syllable association in compound words and adhesive languages is identified, local morphological characteristics (such as prefix/suffix and consonant clusters) near syllable boundaries are enhanced, so that the problem of irregular spelling and cross-language interference is solved, and then place name character strings after syllable segmentation are decoded and output to translate the place names, so that corresponding place name target language translation texts are generated, and the end-to-end effective conversion from original place names to target translated names is realized.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.