Movatterモバイル変換


[0]ホーム

URL:


CN113326148A - Data interaction system based on micro-service - Google Patents

Data interaction system based on micro-service
Download PDF

Info

Publication number
CN113326148A
CN113326148ACN202110561734.7ACN202110561734ACN113326148ACN 113326148 ACN113326148 ACN 113326148ACN 202110561734 ACN202110561734 ACN 202110561734ACN 113326148 ACN113326148 ACN 113326148A
Authority
CN
China
Prior art keywords
service
data interaction
micro
management
log
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110561734.7A
Other languages
Chinese (zh)
Inventor
刘强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengjia Hotel Management Co ltd
Original Assignee
Chengjia Hotel Management Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengjia Hotel Management Co ltdfiledCriticalChengjia Hotel Management Co ltd
Priority to CN202110561734.7ApriorityCriticalpatent/CN113326148A/en
Publication of CN113326148ApublicationCriticalpatent/CN113326148A/en
Pendinglegal-statusCriticalCurrent

Links

Images

Classifications

Landscapes

Abstract

The invention discloses a data interaction system based on micro-services, which comprises a service gateway, a service management system, a transaction coordination system and a basic service system, wherein the service gateway performs data interaction with the service management system and the transaction coordination system and is used for route forwarding with the micro-services; the service management system is used for carrying out service period management on the accessed microservices, wherein the service period management comprises service standard management under a test environment and service running state unified management under an actual application environment; the transaction coordination system is used for coordinating the registration, submission and rollback of the micro-service distributed transaction; and the basic service system performs data interaction with the transaction coordination system and is used for realizing service application of the front-end system according to the message queue of the micro-service.

Description

Data interaction system based on micro-service
Technical Field
The invention relates to the technical field of intelligent management software, in particular to a data interaction system based on micro-service.
Background
With the pace of core cities into the inventory era, a large number of vacant or low-efficiency utilization buildings exist in cities, and a large number of idle properties are urgently needed to be activated to improve the utilization efficiency and promote the function upgrading. Meanwhile, with the increasing number of mobile population, the rising of house price and other problems brought to social economy, the establishment of a reasonable residential lease market becomes an urgent task. Under the promotion of the policies of 'lease and sale in parallel' and 'lease and sale in the same right' in the state, the cultivation and development of a house lease market represented by a long lease apartment becomes an ideal scheme for solving the house problems of urban residents from a supply side in the state, and the house lease market is developed in a 'burst mode'. In addition, due to the change of the demands of the new generation of people on the renting experience, the new generation of consumers tend to obtain the quality lodging environment and intelligent and differentiated renting products. In this context, the industry of long lessons in china has entered an accelerated development period. Standardized, normalized and digitized operation management becomes the trend of forward development of the Chinese housing rental market.
For a digital system for house leasing, how to ensure the smoothness of data interaction of micro-services is a difficult problem, and a data interaction system is urgently needed to realize the data interaction of the micro-services and provide a reliable data interaction technical guarantee for the digital development of house leasing.
Disclosure of Invention
In order to solve the existing problems, the invention provides a data interaction system based on micro-service, which comprises a service gateway, a service management system, a transaction coordination system and a basic service system, wherein,
the service gateway performs data interaction with the service management system and the transaction coordination system and is used for routing forwarding with the micro-service;
the service management system is used for carrying out service period management on the accessed micro-service, wherein the service period management comprises service standard management under a test environment and service running state unified management under an actual application environment;
the transaction coordination system is used for coordinating the registration, submission and rollback of the micro-service distributed transaction;
and the basic service system performs data interaction with the transaction coordination system and is used for realizing service application of the front-end system according to the message queue of the micro-service.
Preferably, the service gateway is implemented by a spring gateway technology, performs load balancing in a software manner, and is further used for user authentication, service routing, log recording request, fusing, and degradation processing.
Preferably, the service authentication of the service gateway is used for ensuring the security of the internal call of the service and the call between the interface and the third-party application, wherein the service authentication is realized by constructing a user authentication system by using the Oauth2 standard and providing authority control of the call between services by using jwt technology.
Preferably, the service management system at least comprises a service configuration unit, a service discovery unit and an authentication service unit;
the service configuration unit is used for providing a dynamic configuration updating function, reducing service redeployment and improving service dynamic updating capability;
the service discovery unit is used for reducing the coupling between the servers and simultaneously realizing dynamic capacity expansion and capacity reduction of the service under the condition that the service is not required to be adjusted;
and the authentication service unit is used for performing uniform security authentication on the upstream data provider and preventing the inflow of abnormal data.
Preferably, the service configuration unit uses apollo as a configuration center, and is used for at least realizing version management, gray release, version rollback, and multi-environment separation of service configuration.
Preferably, the service discovery unit provides a service discovery function by using a consul, performs remote invocation in cooperation with feign, and generates a coding experience consistent with that of invoking a local interface, so as to reduce coupling and implement dynamic capacity expansion and capacity reduction.
Preferably, the system further comprises a fusing current limiting module which interacts with the service gateway and the transaction coordination system respectively and is used for avoiding data blockage caused by service, and the service is subjected to current limiting, fusing and degrading by adopting Sentinel as a flow Sentinel.
Preferably, the transaction coordination system is a distributed transaction management system, and the distributed transaction is processed in a tcc mode, so that multi-service transactions are managed uniformly, and the data consistency requirement and the service idempotency are ensured.
Preferably, the basic service system comprises a basic service module and a log query module;
the basic service module is used for realizing the memory caching function, the relational database function, the object storage function and the message service function of the micro-service;
the log query module at least comprises a log analysis function, a log storage and retrieval function and a log query function;
the log query module is used for reducing the request pressure on the database and improving the service access speed by taking Redis as a module cache component;
the log query module is used as a log message receiving component through Kafka and used for receiving log messages;
and the log query module performs unified processing and storage of logs and establishes a retrieval platform through the ELK.
Preferably, the system further comprises a monitoring module, which interacts with the transaction coordination system;
the monitoring module is realized by adopting a combination mode of link monitoring, dimension monitoring and service monitoring and is used for helping operators to know the operation condition of the system.
The invention discloses the following technical effects:
compared with the prior art, the technical scheme of the invention has the following beneficial effects:
the method has the advantages of cross-platform, low cost and fast iteration requirement;
the invention has complete integral framework and function, achieves the level of production operation level, ensures that the change and adjustment of the later-stage business are more independent, and reduces the level of coupling among services.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments will be briefly described below.
FIG. 1 is a schematic flow diagram of a system according to the present invention;
fig. 2 is a schematic diagram of the system structure according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1-2, the present invention provides a data interactive system based on microservice, the data interactive system includes a service gateway, a service management system, a transaction coordination system, and a basic service system, wherein,
the service gateway performs data interaction with the service management system and the transaction coordination system and is used for routing forwarding with the micro-service;
the service management system is used for carrying out service period management on the accessed micro-service, wherein the service period management comprises service standard management under a test environment and service running state unified management under an actual application environment;
the transaction coordination system is used for coordinating the registration, submission and rollback of the micro-service distributed transaction;
and the basic service system performs data interaction with the transaction coordination system and is used for realizing service application of the front-end system according to the message queue of the micro-service.
The service gateway is realized by a spring gateway technology, adopts a software mode to carry out load balancing, and is also used for user authentication, service routing, log recording request, fusing and degradation processing.
And the service authentication of the service gateway is used for ensuring the internal call security of the service and ensuring the call security between the interface and the third-party application, wherein the service authentication is realized by constructing a user authentication system by using the Oauth2 standard and providing authority control of the call between services by using jwt technology.
The service management system at least comprises a service configuration unit, a service discovery unit and an authentication service unit; the service configuration unit is used for providing a dynamic configuration updating function, reducing service redeployment and improving service dynamic updating capability; the service discovery unit is used for reducing the coupling between the servers and simultaneously realizing dynamic capacity expansion and capacity reduction of the service under the condition that the service is not required to be adjusted; and the authentication service unit is used for performing uniform security authentication on the upstream data provider and preventing the inflow of abnormal data.
The service configuration unit adopts apollo as a configuration center and is used for at least realizing version management, gray level release, version rollback and multi-environment separation of service configuration.
The service discovery unit provides a service discovery function by adopting the consul, remote calling is carried out by matching with the feign, and coding experience consistent with that of calling the local interface is generated for reducing coupling and realizing dynamic capacity expansion and reduction.
The system also comprises a fusing current limiting module which is respectively interacted with the service gateway and the transaction coordination system and used for avoiding data blockage caused by service, and the service is subjected to current limiting, fusing and degrading by adopting the Sentinel as a flow Sentinel.
The transaction coordination system is a distributed transaction management system, adopts a tcc mode to process distributed transactions, is used for uniformly managing multi-service transactions, and ensures the consistency requirement of data and the idempotency of service.
The basic service system comprises a basic service module and a log query module; the basic service module is used for realizing the memory caching function, the relational database function, the object storage function and the message service function of the micro-service; the log query module at least comprises a log analysis function, a log storage and retrieval function and a log query function; the log query module is used for reducing the request pressure on the database and improving the service access speed by taking Redis as a module cache component; the log query module is used as a log message receiving component through Kafka and used for receiving log messages; and the log query module performs unified processing and storage of logs and establishes a retrieval platform through the ELK.
The system also comprises a monitoring module which interacts with the transaction coordination system;
the monitoring module is realized by adopting a combination mode of link monitoring, dimension monitoring and service monitoring and is used for helping operators to know the operation condition of the system.
Example 1: the micro-service architecture based on the SpringCloud technology stack is adopted to design and research and develop the whole system, and based on rich research and development experience accumulated in the past system development process, various service components more suitable for the actual requirements of the project are customized, and the service components include but are not limited to the following components:
the service gateway: the spring gateway technology is adopted as a service gateway technology, and the spring gateway technology is used as a unique entry of a project, and provides user authentication, service routing, log recording request, fusing and degradation processing.
And (3) service authentication: a user authentication system is constructed by using the Oauth2 standard, authority control of calling between services is enhanced by using the jwt technology, safety of calling inside the services is guaranteed, and meanwhile technical support is provided for an interface to third-party application.
Service discovery: the cusul is adopted to provide a service discovery function, a specified remote service address is not needed in calling between services, coupling is effectively reduced, meanwhile, feign is matched to carry out remote calling, coding experience consistent with calling of a local interface is achieved, and meanwhile, dynamic capacity expansion and capacity reduction of the services can be achieved without service adjustment.
Service configuration: by adopting the apollo as a configuration center, the functions of configuration version management, gray level release, version rollback, multi-environment separation and the like are realized, and meanwhile, the dynamic configuration updating function is provided, the service redeployment is reduced, and the dynamic service updating capability is improved.
Fusing and current limiting: the method has the advantages that a micro-service framework is adopted, the condition that 'avalanche' is generated in service is avoided inevitably, the Sentinel is adopted as a flow Sentinel, the service is subjected to current limiting and fusing degradation service, and the phenomenon that 'avalanche' is generated in calling between services is avoided.
Distributed transaction: after the services are split, local transactions cannot be adopted for management like the original single application, one service may simultaneously relate to a plurality of services, the data consistency cannot be ensured, the distributed transactions in a tc mode are adopted for processing, the multi-service transactions are managed in a unified mode, and the data consistency requirement and the service idempotency are ensured; idempotent of data: a hash value generated by a core service field of the metadata is used as a unique key to ensure the idempotency of the upstream data, and the storage pressure caused by a large amount of repeated data when the upstream data is pushed in a full scale is reduced.
Message middleware: and the rabbitmq and the kafka are used as message middles to provide event-based cooperative processing capability among services and provide asynchronous processing capability of the services.
Monitoring: a set of complete monitoring solution is provided by adopting a combination mode of link monitoring, dimension monitoring and service monitoring. Effective optimization suggestions and reliable Shooting can be provided according to actual monitoring conditions, and meanwhile, operators can conveniently and comprehensively know the system operation conditions.
A log base component: redis is adopted as a system cache component, so that the request pressure to a database is reduced, and the service access speed is increased. And a Kafka log message receiving component is adopted, and a unified processing storage and retrieval platform of the log is performed through the ELK.
The data interaction system has several functions:
1. data reception
(1) And (3) metadata storage: and dynamically mapping the data provided by the external upstream and the structure of the internal data storage, and directly and dynamically routing the data provided by the upstream system to the corresponding data structure for storage according to the name and the type of the data provided by the upstream system.
(2) And (4) safety authentication: and carrying out uniform security authentication on the upstream data provider to prevent the inflow of abnormal data.
(3) Idempotent of data: a hash value generated by a core service field of the metadata is used as a unique key to ensure the idempotency of the upstream data, and the storage pressure caused by a large amount of repeated data when the upstream data is pushed in a full scale is reduced.
2. Data push
(1) Data configurability: the metadata is dynamically mapped and cleaned into customized data required by a service side through a Freemarker template when being pushed downstream, and the data required by the service can be conveniently and quickly online.
(2) Data push compensation: the effectiveness of data pushing is ensured through a timing compensation mechanism, and abnormal data can timely inform technicians to check problems of service or data through mails.
3. Business driver
To ensure real-time performance and performance for large amounts of upstream data storage, as well as downstream data consumption. By using the MQ as a bridge for data reception and pushing, the reception of upstream data and the pushing of downstream data are decoupled.
On the data push side, each service can obtain data of a format type required by the service according to the requirement of the service, and can also be used for bridging the existing metadata storage in a customized manner and aggregating to obtain the required service data.
In order to improve timeliness of data requirements, metadata receiving and sending and dynamic mapping of storage are achieved, and all storage operations are achieved in a self-defining mode on the basis of JDBC.
In order to deal with the change of business, according to the business characteristics of the main data synchronization process, the whole data circulation is designed based on the state mode, different data state processes can be executed to process the data according to different requirements, and the expandability is improved.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus once an item is defined in one figure, it need not be further defined and explained in subsequent figures, and moreover, the terms "first", "second", "third", etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the present invention in its spirit and scope. Are intended to be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

CN202110561734.7A2021-05-232021-05-23Data interaction system based on micro-servicePendingCN113326148A (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN202110561734.7ACN113326148A (en)2021-05-232021-05-23Data interaction system based on micro-service

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN202110561734.7ACN113326148A (en)2021-05-232021-05-23Data interaction system based on micro-service

Publications (1)

Publication NumberPublication Date
CN113326148Atrue CN113326148A (en)2021-08-31

Family

ID=77416512

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN202110561734.7APendingCN113326148A (en)2021-05-232021-05-23Data interaction system based on micro-service

Country Status (1)

CountryLink
CN (1)CN113326148A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN114780261A (en)*2022-04-282022-07-22四川虹魔方网络科技有限公司Basic data micro-service sharing system and method in intelligent equipment operation management
CN115018325A (en)*2022-06-102022-09-06中国银行股份有限公司Service processing method and device
CN115016824A (en)*2022-08-092022-09-06飞狐信息技术(天津)有限公司Service security updating method and related equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN111130892A (en)*2019-12-272020-05-08上海浦东发展银行股份有限公司Enterprise-level microservice management system and method
CN111124670A (en)*2019-12-052020-05-08航天科工网络信息发展有限公司Tactical cloud-oriented micro-service platform
WO2020167417A1 (en)*2019-02-122020-08-20Ebay Inc.Global distributed transactions across microservices

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
WO2020167417A1 (en)*2019-02-122020-08-20Ebay Inc.Global distributed transactions across microservices
CN111124670A (en)*2019-12-052020-05-08航天科工网络信息发展有限公司Tactical cloud-oriented micro-service platform
CN111130892A (en)*2019-12-272020-05-08上海浦东发展银行股份有限公司Enterprise-level microservice management system and method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
张晶等: "一种微服务框架的实现", 《计算机系统应用》*
杨俊伟等: "基于微服务架构的电力云服务平台", 《电力信息与通信技术》*
蒋勇: "基于微服务架构的基础设施设计", 《软件》*

Cited By (6)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN114780261A (en)*2022-04-282022-07-22四川虹魔方网络科技有限公司Basic data micro-service sharing system and method in intelligent equipment operation management
CN114780261B (en)*2022-04-282022-11-29四川虹魔方网络科技有限公司Basic data micro-service sharing system and method in intelligent equipment operation management
CN115018325A (en)*2022-06-102022-09-06中国银行股份有限公司Service processing method and device
CN115018325B (en)*2022-06-102024-05-24中国银行股份有限公司Service processing method and device
CN115016824A (en)*2022-08-092022-09-06飞狐信息技术(天津)有限公司Service security updating method and related equipment
CN115016824B (en)*2022-08-092022-12-13飞狐信息技术(天津)有限公司Service security updating method and related equipment

Similar Documents

PublicationPublication DateTitle
CN113326148A (en)Data interaction system based on micro-service
US8918424B2 (en)Managing homeowner association messages
CN111614550B (en)Optimized integrated gateway of electric power marketing service system
Yang et al.Design and implementation of cloud platform for intelligent logistics in the trend of intellectualization
CN101800762A (en)Service cloud system for fusing multiple services and service implementation method
CN112256248B (en)Intelligent cloud low-code platform capable of achieving starting effect
CN101873334A (en) A state-driven executable business process execution method
CN104240070A (en)Data release service system and method
CN113468221A (en)System integration method based on kafka message data bus
CN106230985A (en)A kind of based on the big data processing method of Internet of Things, system and service processing end
CN115391361A (en)Real-time data processing method and device based on distributed database
CN116628033A (en)Cache preheating and data processing method, service device, electronic equipment and medium
CN113608952B (en)System fault processing method and system based on log construction supporting environment
CN115757552B (en)Bank historical data management system based on distributed microservice
CN112580199A (en)Electric power system multidimensional data unified construction system based on CIM model
CN114489761B (en)Service integration and application integration method based on container cluster
CN102801876B (en)Business information centralized processing system and method
CN118200140B (en)Data processing method, management system and related equipment
CN118535302A (en) Task scheduling and processing method, device, computer equipment and storage medium
CN116193382A (en) Service reminder method, device, electronic device and storage medium
CN111401819B (en)Intersystem data pushing method and system
CN115794922A (en)External data query method, device, equipment and storage medium
CN115686497A (en)Business development data management method, development engine, electronic device, and storage medium
CN113542245A (en)Data flow monitoring method and device, computer equipment and storage medium
CN113191621A (en)Intelligent integrated voice service management platform based on data and service fusion

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination
RJ01Rejection of invention patent application after publication
RJ01Rejection of invention patent application after publication

Application publication date:20210831


[8]ページ先頭

©2009-2025 Movatter.jp