Disclosure of Invention
The embodiment of the specification provides a map updating method, a map updating device, electronic equipment and a map updating storage medium based on 2D-SLAM, and the technical scheme is as follows:
In a first aspect, an embodiment of the present disclosure provides a map updating method based on 2D-SLAM, applied to a vehicle, where the vehicle is installed with a radar, and the vehicle constructs an original map according to original point cloud data acquired by the radar, including:
responding to the real-time point cloud data acquired by the radar, identifying static point clouds which accord with target characteristics in the point cloud data, and determining the rest point clouds as real-time dynamic point clouds;
determining the position of the vehicle in the original map according to the static point cloud, and determining an original dynamic point cloud matched with the real-time dynamic point cloud in the original point cloud data according to the position of the vehicle;
And replacing the original dynamic point cloud with the real-time dynamic point cloud to obtain target point cloud data, and updating the original map according to the target point cloud data.
In a second aspect, an embodiment of the present disclosure provides a training method for a point cloud identification model, including:
acquiring a training sample set, wherein each training sample in the training sample set comprises a history point cloud and a real identification result which is manually marked and matched with the history point cloud;
Inputting the training sample set into a model to be trained so that the model to be trained can identify the history point cloud to obtain a prediction identification result;
and comparing the predicted recognition result with the real recognition result, and optimizing the model to be trained according to the comparison result.
In a third aspect, an embodiment of the present disclosure provides a map updating apparatus based on 2D-SLAM, applied to a vehicle, the vehicle being mounted with a radar, the vehicle constructing an original map according to original point cloud data acquired by the radar, including:
The identification unit is used for responding to the real-time point cloud data acquired by the radar, identifying static point clouds which accord with target characteristics in the point cloud data, and determining the rest point clouds as real-time dynamic point clouds;
The determining unit is used for determining the position of the vehicle in the original map according to the static point cloud and determining an original dynamic point cloud matched with the real-time dynamic point cloud in the original point cloud data according to the position of the vehicle;
And the updating unit is used for replacing the original dynamic point cloud with the real-time dynamic point cloud to obtain target point cloud data, and updating the original map according to the target point cloud data.
In a fourth aspect, an embodiment of the present disclosure provides a training apparatus for a point cloud identification model, including:
The acquisition unit is used for acquiring a training sample set, wherein each training sample in the training sample set comprises a history point cloud and a real identification result which is manually marked and matched with the history point cloud;
The input unit is used for inputting the training sample set into a model to be trained so that the model to be trained can identify the history point cloud to obtain a prediction identification result;
and the comparison unit is used for comparing the predicted recognition result with the real recognition result and optimizing the model to be trained according to the comparison result.
In a fifth aspect, embodiments of the present disclosure provide an electronic device comprising a processor and a memory; the processor is connected with the memory; the memory is used for storing executable program codes; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory for performing the steps of the method according to the first or second aspect of the above embodiment.
In a sixth aspect, embodiments of the present disclosure provide a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the steps of the method of the first or second aspects of the embodiments described above.
The technical scheme provided by some embodiments of the present specification has the following beneficial effects:
The point cloud is divided into a static point cloud and a dynamic point cloud, wherein the static point cloud corresponds to an object which cannot easily change in a real environment, and the dynamic point cloud corresponds to an object with a time length change in the real environment. In the process of updating the map, the static point cloud is not required to be updated, only the dynamic point cloud is required to be updated, and the accuracy of map updating is considered, and meanwhile, the updated data volume is reduced, so that the efficiency of map updating is improved.
Detailed Description
The technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
The 2D-SLAM (Simultaneous Localization AND MAPPING ) technique is a technique for building a map and determining its own location in real time in an uncertain environment. It builds a map by collecting environmental data using sensors (e.g., cameras, lidar, etc.) and extracting feature points in the environmental data. Over time, feature points in the map may be subject to errors due to inaccuracies in the robot motion model or other factors. Therefore, the map needs to be optimized regularly.
In the related art, it is necessary to re-collect surrounding environment data through a sensor, add new feature points to a map, recognize the feature points, and adjust the positions and attitudes of existing feature points. However, in the above process, the re-acquisition of the environmental data and the feature recognition are both time consuming and computationally intensive, and the map updating is inefficient.
In order to solve the problems in the related art, the present specification proposes a map updating method based on 2D-SLAM.
Referring to fig. 1, fig. 1 shows a flow chart of a map updating method based on 2D-SLAM provided in an embodiment of the present disclosure, and as shown in fig. 1, the map updating method based on 2D-SLAM is applied to a vehicle, where the vehicle is equipped with a radar, and the vehicle constructs an original map according to original point cloud data acquired by the radar, and may at least include the following steps:
And 102, responding to the real-time point cloud data acquired by the radar, identifying static point clouds conforming to target characteristics in the point cloud data, and determining the rest point clouds as real-time dynamic point clouds.
The method is applied to a radar-mounted vehicle, and of course, may be other radar-mounted devices, and this is not a limitation of the present specification.
The point cloud data is a data set of spatial points scanned by the radar apparatus. The radar scan can obtain image data of the surrounding environment. After the data are resolved in two or three dimensions in space, two-dimensional or three-dimensional point cloud data are formed. The point cloud data can provide high-precision spatial information and can be used for carrying out stereoscopic modeling and spatial analysis. Acquisition of point cloud data typically uses a lidar for scanning. Lidar is a device capable of emitting laser light and measuring reflected light, by measuring the time that the laser light is reflected back from a target object, the size, shape, and position of the target object can be determined. By scanning the laser radar, a large amount of space point data can be obtained, and the data is processed to form a point cloud.
The original point cloud data may be point cloud data used for constructing an existing original map acquired before this map update. The real-time point cloud data is the point cloud data obtained by the map updating.
The static point cloud is a point cloud corresponding to a static and reliable object, and the dynamic point cloud is a point cloud corresponding to a dynamically updated object. In a real-world environment, there are stationary reliable objects that are relatively stationary and not easily moved, and there are also dynamically updated objects that are not stationary and easily moved. For example: the buildings or lighthouses on the port yard are all static and reliable objects, while the container or vessel is a dynamically updated object. In the map updating process, static point clouds are not required to be updated, and dynamic point clouds are important points of updating.
The target features include at least one of: the reflection intensity of the point cloud reaches a preset threshold, and the corner characteristics of the point cloud accord with preset characteristics. The specific process of target feature and point cloud identification will be described in detail later, and will not be repeated here.
Step 104, determining the position of the vehicle in the original map according to the static point cloud, and determining an original dynamic point cloud matched with the real-time dynamic point cloud in the original point cloud data according to the position of the vehicle.
And 106, replacing the original dynamic point cloud with the real-time dynamic point cloud to obtain target point cloud data, and updating the original map according to the target point cloud data.
The position of the vehicle can be positioned by comparing static point clouds in the real-time point cloud data, and after the position is determined, the real-time dynamic point clouds can be used for replacing the original dynamic point clouds, so that the space information of the object dynamically updated in the map is updated. Because the static and reliable object corresponding to the static point cloud is not easy to change, the static and reliable object can be ignored when the map is updated, and only the dynamic point cloud data is processed, so that the processed data volume is reduced.
In this embodiment, the point cloud is divided into a static point cloud corresponding to an object that does not easily change in the real environment and a dynamic point cloud corresponding to an object that changes in the real environment in a time period. In the process of updating the map, the static point cloud is not required to be updated, only the dynamic point cloud is required to be updated, and the accuracy of map updating is considered, and meanwhile, the updated data volume is reduced, so that the efficiency of map updating is improved.
As previously mentioned, the target features include at least one of: the reflection intensity of the point cloud reaches a preset threshold, and the corner characteristics of the point cloud accord with preset characteristics.
The reflection intensity of the point cloud refers to the light intensity reflected by the laser pulse after striking the surface of the target object in the scanning process of the laser radar. The reflected intensity signal carries important information about the properties of the target object and can be used in various application scenarios, such as three-dimensional modeling, obstacle detection, material classification, etc.
When the laser pulse encounters an object, a portion of the energy is absorbed and another portion is reflected back. This reflected light intensity depends on a number of factors including the material, color, roughness of the object surface, and the angle of the incident light. Different materials have different reflectivities. For example, metals typically reflect a large amount of laser energy with less absorption, and therefore their reflected intensity is relatively high. Conversely, absorbing more material (e.g., wood or plastic) may have lower reflection intensity.
In practical cases, the reflecting plate can be installed on a static and reliable object, so that the reflecting intensity of the point cloud corresponding to the object is stronger. Therefore, a threshold may be preset, and if it is identified that the reflection intensity of the point cloud exceeds the preset threshold, the point cloud is a static point cloud.
Point cloud corner features refer to those points in the point cloud data that have a particular position and orientation that generally represent critical turning points of the boundary or shape of an object in two or three dimensions. Corner features are critical for modeling, object recognition, scene reconstruction, etc. applications, as they contain important information about the object.
Extracting corner features of a point cloud generally comprises the following steps:
Pretreatment: the method comprises the steps of filtering noise points, removing redundant points and the like, so that the subsequent processing is more accurate; and (3) feature detection: common methods are geometry-based and vision-based; geometry-based methods: identifying corner points by utilizing geometrical properties of point clouds, such as curvature, normal, edge detection and the like; vision-based methods: referring to corner detection algorithms in image processing, such as Harris corner detection, shi-Tomasi corner detection and the like, the methods identify corners by calculating local characteristics of the points, and different corners form certain semantics by a clustering method; characterization: to represent the corner points more accurately, it is often necessary to generate feature descriptors for the corner points, such as local coordinate systems, main directions, etc.; corner matching: in multi-view point clouds, corresponding corner points need to be found, which usually involve feature matching algorithms, such as FLANN, brute-Force matching, etc.; corner optimization: the accuracy and the robustness of the angular points are improved through an optimization algorithm, for example, a RANSAC algorithm can be used for optimizing a plane model in the point cloud, so that the angular points are accurately positioned.
Further, in the case that the target feature includes that the corner feature of the point cloud is matched with a preset template, the identifying the static point cloud conforming to the target feature in the point cloud data includes: inputting the real-time point cloud data into a point cloud identification model, so that the point cloud identification model extracts angular point characteristics of each point cloud in the real-time point cloud data through a neural network, and outputs static point clouds in the real-time point cloud data according to the extracted angular point characteristics; and/or extracting the angular point characteristics of each point cloud in the real-time point cloud data, and determining that the corresponding point cloud is a static point cloud under the condition that the extracted angular point characteristics hit any preset template in the characteristic template set.
Still take the port and dock as an example, the shelf is a static and reliable object, the shapes of the shelf are generally similar, corresponding templates can be set for the shelf, and when the corner features of the point cloud are matched with the templates of the shelf, the point cloud is a static point cloud.
In identifying corner features by machine learning models, a cluster analysis model, such as a K-means model, may be used. The model can form certain semantic corner features, such as semantic corner features of a cylindrical wall, semantic corner features of a shelf and semantic corner features of a window metal edge, by a clustering method aiming at different corner points.
The training process of the K-means cluster analysis model mainly comprises the following steps: initialization, allocation, update, iteration, evaluation. Initializing: the number of clusters K needs to be determined first, and then K data points are randomly selected as initial cluster centers. And (3) distribution: for each data point in the dataset, its distance from each cluster center is calculated and assigned to the cluster represented by the nearest cluster center. Updating: after all data points are assigned, the center point of each cluster is recalculated, typically the center point of a cluster is the average of all data points in the cluster. Iteration: the steps of assigning and updating are repeated until a stop condition is met. The stopping condition may be that the change of the cluster center point is smaller than a certain threshold, a preset upper limit of iteration times is reached, or a cohesiveness index (such as a contour coefficient) reaches a certain threshold. Evaluation: at the end of the training process, it is often necessary to evaluate the quality of the clustering results. Some evaluation indexes, such as contour coefficients, internal distances, external distances, etc., can be used to measure the performance of the clusters. In this embodiment, the initial clustering center may be semantic corner features of a plurality of static reliable objects (for example, semantic corner features of a cylindrical wall, semantic corner features of a shelf, and semantic corner features of a window metal edge) preset manually, and the cohesion index may be the accuracy of comparison between an analysis result of model analysis and an analysis result of manual analysis.
When the angular point features are identified through the templates, a specific matching method can use an OpenCV-hu invariant moment matching method to perform template matching on the point cloud in an image mode, so that whether the point cloud is a static point cloud or not is determined.
The two methods for identifying whether the point cloud is a static point cloud according to the angular point features, wherein the point cloud is identified according to the machine learning model, a template is not required to be arranged, and the efficiency of identifying the point cloud can be improved; the method is based on preset template recognition, whether the trained model is accurate or not is not needed to be considered, and accuracy of point cloud recognition can be improved.
As described above, the static point cloud is a point cloud corresponding to a static and reliable object, and the dynamic point cloud is a point cloud corresponding to a dynamically updated object. For example: the buildings or lighthouses on the port yard are all static and reliable objects, while the container or vessel is a dynamically updated object. In the map updating process, static point clouds are not required to be updated, and dynamic point clouds are important points of updating.
In an embodiment, the vehicle is further loaded with a local communication module, further comprising the steps of: determining an updated real-time map, and sending the real-time map to other communication models through the local communication module; the other communication modules are communication modules loaded by other vehicles in the same vehicle team with the vehicle.
The communication means of the communication modules between the vehicles in the fleet may be through near field communication (such as bluetooth, NFC, etc.), may also be through cloud (such as blockchain), or may even be through wired communication, which is not limited in this specification.
In the embodiment, only one or a few vehicles in a vehicle team are required to have the map updating function, so that the map optimization of the vehicles in the whole vehicle team can be realized, and the cost is reduced.
The specification also provides a training method of the point cloud identification model. Referring to fig. 2, fig. 2 is a flow chart illustrating a training method of a point cloud identification model according to an embodiment of the present disclosure, and as shown in fig. 2, the training method of the point cloud identification model may at least include the following steps:
step 202, acquiring a training sample set, wherein each training sample in the training sample set comprises a history point cloud and a manually marked real recognition result matched with the history point cloud;
step 204, inputting the training sample set to a model to be trained, so that the model to be trained identifies the history point cloud to obtain a prediction identification result;
And 206, comparing the predicted recognition result with the real recognition result, and optimizing the model to be trained according to the comparison result.
In this embodiment, the point cloud is divided into a static point cloud corresponding to an object that does not easily change in the real environment and a dynamic point cloud corresponding to an object that changes in the real environment in a time period. In the process of updating the map, the static point cloud is not required to be updated, only the dynamic point cloud is required to be updated, and the accuracy of map updating is considered, and meanwhile, the updated data volume is reduced, so that the efficiency of map updating is improved.
Based on the map updating device based on 2D-SLAM described in the embodiments of the present disclosure, it is known that the point cloud is divided into a static point cloud and a dynamic point cloud, where the static point cloud corresponds to an object that cannot easily change in a real environment, and the dynamic point cloud corresponds to an object that changes in a real environment in a time period. In the process of updating the map, the static point cloud is not required to be updated, only the dynamic point cloud is required to be updated, and the accuracy of map updating is considered, and meanwhile, the updated data volume is reduced, so that the efficiency of map updating is improved.
Referring to fig. 3, fig. 3 is a block diagram of a map updating apparatus based on 2D-SLAM according to an embodiment of the present disclosure. The device comprises:
The identifying unit 302 is configured to identify a static point cloud conforming to a target feature in the point cloud data in response to the real-time point cloud data acquired by the radar, and determine the rest point clouds as real-time dynamic point clouds;
A determining unit 304, configured to determine, according to the static point cloud, a position of the vehicle in the original map, and determine, according to the position of the vehicle, an original dynamic point cloud that matches the real-time dynamic point cloud in the original point cloud data;
and the updating unit 306 is configured to replace the original dynamic point cloud with the real-time dynamic point cloud to obtain target point cloud data, and update the original map according to the target point cloud data.
Optionally, the target feature includes at least one of: the reflection intensity of the point cloud reaches a preset threshold, and the corner characteristics of the point cloud accord with preset characteristics.
Optionally, in the case that the target feature includes that the corner feature of the point cloud matches with a preset template, the identifying unit 302 is specifically configured to:
Inputting the real-time point cloud data into a point cloud identification model, so that the point cloud identification model extracts angular point characteristics of each point cloud in the real-time point cloud data through a neural network, and outputs static point clouds in the real-time point cloud data according to the extracted angular point characteristics; and/or the number of the groups of groups,
Extracting angular point characteristics of each point cloud in the real-time point cloud data, and determining that the corresponding point cloud is a static point cloud under the condition that the extracted angular point characteristics hit any preset template in a characteristic template set.
Optionally, the static point cloud is a point cloud corresponding to a static and reliable object, and the dynamic point cloud is a point cloud corresponding to a dynamically updated object.
Optionally, the vehicle is further loaded with a local communication module, and further includes the steps of:
A sending unit 308, configured to determine the updated real-time map, and send the real-time map to other communication models through the local communication module; the other communication modules are communication modules loaded by other vehicles in the same vehicle team with the vehicle.
Referring to fig. 4, fig. 4 is a block diagram of a training device for a point cloud recognition model according to an embodiment of the present disclosure. The device comprises:
An obtaining unit 402, configured to obtain a training sample set, where each training sample in the training sample set includes a history point cloud and a manually-labeled real recognition result that matches the history point cloud;
The input unit 404 is configured to input the training sample set to a model to be trained, so that the model to be trained identifies the history point cloud to obtain a predicted identification result;
And the comparison unit 406 is configured to compare the predicted recognition result with the real recognition result, and optimize the model to be trained according to the comparison result.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are mutually referred to, and each embodiment mainly describes differences from other embodiments. In particular, for the 2D-SLAM-based map updating apparatus embodiment, since it is substantially similar to the 2D-SLAM-based map updating method embodiment, the description is relatively simple, and the relevant points are referred to in the partial explanation of the method embodiment.
Please refer to fig. 5, which illustrates a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
As shown in fig. 5, the electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.
Wherein the communication bus 502 may be used to enable connectivity communication of the various components described above.
The user interface 503 may include keys, and the optional user interface may also include a standard wired interface, a wireless interface, among others.
The network interface 504 may include, but is not limited to, a bluetooth module, an NFC module, a Wi-Fi module, and the like.
Wherein the processor 501 may include one or more processing cores. The processor 501 utilizes various interfaces and lines to connect various portions of the overall electronic device 500, perform various functions of the electronic device 500, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and invoking data stored in the memory 505. Alternatively, the processor 501 may be implemented in at least one hardware form of DSP, FPGA, PLA. The processor 501 may integrate one or a combination of several of a CPU, GPU, modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 501 and may be implemented by a single chip.
The memory 505 may include RAM or ROM. Optionally, the memory 505 comprises a non-transitory computer readable medium. Memory 505 may be used to store instructions, programs, code sets, or instruction sets. The memory 505 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described various method embodiments, etc.; the storage data area may store data or the like referred to in the above respective method embodiments. The memory 505 may also optionally be at least one storage device located remotely from the processor 501. An operating system, a network communication module, a user interface module, and a map update application program based on 2D-SLAM may be included in the memory 505 as one type of computer storage medium. The processor 501 may be used to invoke the 2D-SLAM based map updating application stored in the memory 505 and perform the 2D-SLAM based map updating and formulation steps mentioned in the previous embodiments.
Embodiments of the present disclosure also provide a computer-readable storage medium having instructions stored therein, which when executed on a computer or processor, cause the computer or processor to perform the steps of one or more of the embodiments shown in fig. 2-4 above. The above-described constituent modules of the electronic apparatus may be stored in the computer-readable storage medium if implemented in the form of software functional units and sold or used as independent products.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present description, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted across a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (Digital Subscriber Line, DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disk (DIGITAL VERSATILE DISC, DVD)), or a semiconductor medium (e.g., a Solid state disk (Solid STATE DISK, SSD)), or the like.
Those skilled in the art will appreciate that implementing all or part of the above-described embodiment methods may be accomplished by way of a computer program, which may be stored in a computer-readable storage medium, instructing relevant hardware, and which, when executed, may comprise the embodiment methods as described above. And the aforementioned storage medium includes: various media capable of storing program code, such as ROM, RAM, magnetic or optical disks. The technical features in the present examples and embodiments may be arbitrarily combined without conflict.
The above-described embodiments are merely preferred embodiments of the present disclosure, and do not limit the scope of the disclosure, and various modifications and improvements made by those skilled in the art to the technical solutions of the disclosure should fall within the protection scope defined by the claims of the disclosure without departing from the design spirit of the disclosure.